Plausible isn't compliant

Is that a lot? A little? Is it a dose, a patient count, a lab value, a day of the trial?

You can't tell, and neither can the US Food and Drug Administration (FDA). That's why its data integrity guidance uses this exact example: the number 23 means nothing until you know it is "23 mg." For those with a scientific background that may seem rather obvious. However, AI is trained on data from every perspective imaginable and often writes in shorthand (we're looking at you, Claude Fable 5.1), with the assumption the reader knows what it's talking about. The FDA calls that extra information metadata, "the contextual information required to understand data."

We would call it context. And that's the whole reason KIRHA exists.

The rules haven't changed

The prevailing idea is that AI will force regulators to write new rules for trust. It won't. The rules already exist, and they are older than any language model.

For decades, pharma has worked to a simple standard called ALCOA. Data must be attributable, legible, recorded at the time, original (or a true copy), and accurate. If you can't say where a number came from, who recorded it and when, it doesn't count.

When the FDA published guidance on using AI for drug decisions in January 2025, it didn't lower that bar. It said the data behind an AI model must be "fit for use." That means relevant, with the right data and the right patients, and reliable, meaning "accurate, complete, and traceable."

In July 2026, researchers put GPT-5.4 and Gemini 2.5 through the World Health Organization–Uppsala Monitoring Centre (WHO-UMC) causality assessment on 337 drug–event pairs from the FDA Adverse Event Reporting System (FAERS), the most routine exercise in any pharmacovigilance (PV) department. Depending on the model and the prompt, they agreed with the expert assessors between 58% and 80% of the time, and on the "possible" category, the one that generates most of the work, performance collapsed. The authors' verdict was that AI is useful for triage, not for independent decisions. The FDA has run into the same wall. Its own AI assistant, Elsa, launched in June 2025 to summarise adverse events and compare labels, was citing studies that don't exist within weeks, according to six agency staff. One of them put it in a sentence any PV professional will recognise: "Anything that you don't have time to double-check is unreliable."

In January 2026, the European Medicines Agency (EMA) and the FDA went further and agreed on ten shared principles for AI in drug development. Principle six reads:

Data source provenance, processing steps, and analytical decisions are documented in a detailed, traceable, and verifiable manner, in line with GxP requirements.

In other words: where the data came from, what you did to it, and why you decided what you decided. All of it written down, all of it checkable.

That isn't an AI problem. It's a context and traceability problem. Even the strictest draft on the table lands in the same place from the opposite direction. The EU's proposed Good Manufacturing Practice (GMP) Annex 22 would keep generative AI out of critical manufacturing decisions altogether. When the EMA workshopped it with industry in June, nobody asked for the bar to be lowered. They asked for AI to be judged on its documented intended use, with human oversight proportionate to the risk. However the final text lands, the questions a regulator will ask are the same: what is this tool for, where did its answer come from, and who checked it?

The FDA has already asked them once. In April 2026 it sent a warning letter to a manufacturer that had used AI agents to write its product specifications, procedures and batch records. Nobody had reviewed what the agents produced, and the firm told investigators it had skipped process validation because the AI never told it that validation was required. The letter's wording is worth keeping: "if you use AI as an aid in document creation, you must review the documents it generates to ensure they are accurate and actually compliant with CGMP."

The FDA's first warning letter over AI misuse (Purolea Cosmetics Lab, 2 April 2026).

Plausible isn't compliant

Language models are very good at sounding right. They give you the most probable next word according to their inputs. In most jobs, sounding competent is a decent start. In regulated work, it's a liability. Without all the variables and considerations needed, a large language model just returns whatever is most likely to come next.

The EMA put it bluntly in its 2024 reflection paper on AI: generative models "are prone to include plausible but erroneous or incomplete output." Its answer is that every generated sentence must be checked before it goes anywhere near a regulator.

The usual fix is to bolt on web search, or "retrieval," so that the model cites real things. That helps, up to a point. A citation proves the model found a page, not that the page was the right source, that it is still current, or that anyone could rebuild the answer from it. And in life sciences, web search brings problems of its own:

• It doesn't know where you are. A drug can be approved in the US and not in France. A clinical decision support tool that recommends it to a French doctor is simply wrong. That's why our clinical agents run on localised data like Vidal, the French drug reference, so every recommendation is checked against what is actually authorised and available in France. Pas mal non ? C'est français 🥖
• It doesn't know when it is. Labels change. Guidelines change. Safety signals appear spontaneously. A page indexed six months ago may be dangerous today.
• It doesn't expose its methodoly. A search result is a link, not an audit trail. You can't rebuild how an answer was made from a list of URLs, and often you can't tell where in that link the information came from.
• Data quality is questionable. Most reliable data sources sit behind paywalls and block AI crawlers and Model Context Protocol (MCP) access, so an AI web search is often limited to the lowest-quality sources: blogs and websites where anyone can say anything.

Professionals don't trust, they verify. So any system they use for regulated work has to make verifying not just possible, but practical too.

What it actually takes to make AI useful

Making AI usable in regulated work isn't magic or complicated. It just requires several boring things done properly and consistently, and the easiest way to show them is a single trace.

To demonstrate this, we asked our pharmacovigilance agent about pseudoephedrine, a decongestant found in many over-the-counter cold medicines, including what the safety data says about cardiovascular risk. The chart below is one piece of its answer: a reporting odds ratio for every Medical Dictionary for Regulatory Activities (MedDRA) System Organ Class, with anything above 1.0 flagged as a signal.

See the full sample report on pseudoephedrine, prepared in minutes.

Useful AI in PV requires high-quality, curated data. That's why every data provider on KIRHA is chosen, licensed and maintained. When a pharmacovigilance agent asks about a drug, it asks the reference that regulators and pharmacists already rely on, not whatever ranked well on a search engine that morning.

A drug dictionary, a trial registry and a literature database each structure their data differently, so flattening everything into one blob of text loses the exact metadata the FDA asks about. Our context graph keeps each source in its native shape, so "23" stays "23 mg, oral, daily, as of the label dated this month."

Regulators often refer to the "context of use" of an AI agent: what is this tool for, what is it not for, and what happens if it's wrong? So we don't build one agent that does everything. We build a suite of agents, each with dedicated tasks: clinical decision support, regulatory affairs, pharmacovigilance and clinical trial data programming.

Our PV agent ensures that every answer points back to its sources and the steps taken to reach it. If an auditor asks "where did this come from?", there's an actual answer. Not "I dunno, the model just said so." We show you the query used, where the data was pulled from, how it was done and why.

Under the French health data hosting rules (HDS) and the Health Insurance Portability and Accountability Act (HIPAA) in the US, you can't send patient data to a random AI service and hope for the best. That's why we built our own sandbox for running agents, and why sensitive workloads can run in a mode where nothing is ever written to disk. Your data never leaves your environment, and the outputs are yours.

None of this makes for a flashy demo, but all of it is what makes a useful AI solution for pharmacovigilance and regulated workflows.

What it unlocks

Doing the boring work properly is what makes impressive results possible and trustworthy. Beyond our PV agent, we extend these rules to our clinical coding agent. Clinical trial data must be converted into the standardised format the FDA expects before submission, and that used to take an experienced team weeks. Our trial coding agent suggests the mappings and writes the conversion code in SAS, in a day. You can see every step and verify the logic. A data manager still reviews every dataset, program and decision before approving the study outputs. The trail is what makes that review fast, not what replaces it.

Speed you can't defend is just going nowhere fast. Great if you've nowhere to be, but pointless if you do. Speed with a defensible trail behind it is worth everything. Traceable analysis lets your teams go faster and focus on interpreting results rather than gathering information.

The industry keeps asking which model is best for life sciences. It's entirely the wrong question. The models will keep getting better, and the gap between them will keep shrinking. What will never shrink is the gap between an answer that sounds right and an answer you can prove. In regulated work, context isn't a feature. It's the product.

---

Sources

• FDA, Data Integrity and Compliance With Drug CGMP: Questions and Answers (Dec 2018): ALCOA definition, the "23 mg" metadata example.
• FDA, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products, draft guidance (Jan 2025): "fit for use", "accurate, complete, and traceable".
• Ha YM et al., Large Language Models for WHO-UMC Drug–Adverse Event Causality Assessment Using FAERS Cases: Comparative Performance Study, J Med Internet Res (8 Jul 2026): 58% to 80% agreement with expert assessors; "Possible" category F1 0.29; "supportive tools… preliminary case triage".
• Owermohle S., FDA's artificial intelligence is supposed to revolutionize drug approvals. It's making up studies, CNN (23 Jul 2025): Elsa launched June 2025; six current and former FDA staff on non-existent studies; "Anything that you don't have time to double-check is unreliable."
• GMP Publishing, Annex 22: Results of the EMA Workshop (9 Oct 2026): the draft still excludes generative AI and large language models from critical GMP uses; industry asked for a risk-based approach judged on intended use, with human oversight "proportionate to risk."
• EMA and FDA, Guiding Principles of Good AI Practice in Drug Development (14 Jan 2026): principle 6 quote.
• EMA, Reflection paper on the use of AI in the medicinal product lifecycle (Sept 2024): "plausible but erroneous or incomplete output".
• FDA, warning letter to Purolea Cosmetics Lab (2 April 2026): the first over AI misuse.
• KIRHA blog, Spring brings Life (Sciences) (May 2026) and Monthly recap, March 2026: Vidal, sandbox, trial coding in a day, 95% time saved on literature reviews.