Humans at the Helm, Not Just in the Loop

by Jason Gardner, EVP, Head of Medical, Global Medical Communication

Why the future of medical publications depends on who’s accountable, not how fast the machine runs

Generative AI can draft a manuscript for a medical journal in minutes. Impressive! But with this comes the risk of automating away precisely what is most critical for safeguarding its quality: author intent, scientific expertise and judgment, and accountability. Here’s the line we draw, and why.

There is a very real expectation running through many conversations about AI in medical publications that the goal is time saving. Faster congress abstracts and journal manuscripts; fewer medical writer hours needed to create content on behalf of the guiding expert authors. It is an easy expectation to have, because speed is the thing AI most obviously delivers.

But using AI for speed without keeping humans in the driver’s seat carries risk. Automating the typing while leaving the thinking unsupported doesn’t solve the problem. In medical publishing, the stakes are too high. Medical publications establish scientific understanding, influence treatment decisions, support regulatory and payer conversations, and shape the evidence base on which future research builds. Errors are not simply editorial mistakes; they can influence clinical interpretation and decision-making.

We should work to a different premise. AI should be an accelerator for decision-making and an amplifier for human expertise, not a replacement. It should compress the mechanical work – synthesizing landscapes, structuring evidence, drafting first passes – so that human experts spend their time where only humans can: on scientific rigor, on context, on accountability. This philosophy is foundational to how we build our AI solutions within Real Chemistry ANATOMI, including our publications workflows. It’s our craft that guides the code. The more we can decouple our people from repeatable tasks and allow them to focus on healthcare-specific, scientific judgement, the more we strengthen client impact and ensure patients ultimately benefit from better-informed decisions.

A different role for AI in Medical Publications

That distinction has a practical name. The industry talks about keeping a “human in the loop”; but that term implies a passive, informed role in a process the humans did not shape. We design for something stricter: “humans at the helm”: a lesser-known term that we think should become the standard. The authors’ intent sets the direction. AI works inside that intent – guided by expert medical writers who curate the content, shape the narrative, review and approve at defined decision gates throughout – not once at the end, but at every point where a wrong assumption would compound into a confident error. This is how we take human accountability of the whole content creation – from evidence interpretation and publication strategy to scientific statements and data verification — not just the end product.

We’re aware of the critique, and it’s fair. Automation bias is real: even experts defer to a confident machine when the clock is ticking and the output looks fluent. That is exactly the failure mode we design against. “Humans at the helm” only means something if the human holds real authority, not borrowed time. So, every decision gate carries the following guarantees: the medical writer has standing to adapt or return AI output; review time is protected, not squeezed by whatever throughput the AI makes possible; and outcomes are tracked. This means that content changes are assessed at each gate, tracked in the same audit log as the content itself. If content never changes at a gate, that isn’t oversight succeeding. That’s the signal to look harder.

Furthermore, we know that LLMs are advancing quickly, and maintaining a human eye on changes that could destabilize existing workflows is just as important as making sure the outputs are scientifically sound. When version updates happen, model behavior isn’t necessarily reproduced in exactly the same way, and workflows built around one behavior can drift under another without warning. Keeping humans at the helm means someone is watching for that shift, capturing the gains when a model improves, but also catching it before an undetected change compounds into an error.

Three principles follow from that, and they are non-negotiable in how we build AI-augmented medical publications through Real Chemistry ANATOMI.

1. Human intent governs the workflow.

We do not start from “what can the AI create?”, have AI generate content based on the data, then check it with humans who take credit for it. That risks reducing human contributors to reviewers of machine-generated content rather than what they are – true authors and scientific stewards of the work. Instead, we start from what the authors are trying to achieve and what GPP and ICMJE compliance requires,^1,2 and we constrain the AI to serve that. Transparent, auditable, compliant – that is the deliberate design of Real Chemistry ANATOMI Publications.

2. AI tools must expose uncertainty, not conceal it.

In healthcare, confidence without evidence isn‘t useful. Surfacing uncertainty is a non-negotiable trait of AI purpose-built for medical publishing. If a Real Chemistry AI tool cannot verify a finding against its source, it reports the gap and asks for input. It does not invent information to fill a hole. This sounds obvious; but we all know that hallucinations are a fundamental feature of unchecked AI. Integrating internal checks before enabling human verification is critical – ensuring quality standards remain paramount.

3. Every claim must be traceable to evidence.

A human reader should never take AI’s word for it! Because in healthcare communications, trust is built on evidence, transparency and accountability. Just like the manual publication process, each sourced statement carries a cross-reference back to the primary material, in every output format. Confidence in the source is scored by AI for interrogation, adjudication, iteration of the content, and approval by medical writers – with audit log available for inspection by authors and sponsors.

The AI acceleration of content creation is real and substantial. What changes in AI-augmented content creation is where the human time goes: out of manually typing and formatting of content, and into the “first mile” judgment that determines the sources and the intent, and into the “last mile” validation of the content.³ But to take true accountability of the product, we maintain that human decision-making is critical during the “middle mile” of content creation. With this model, we have certainty that the output is accurate, trustworthy, and carries itself with integrity.

That trust is not a feature you add at the end. It is the thing you design the whole system around.

The future of medical publications will not be defined by how quickly AI can draft content. It will be defined by trust, and whether organizations can scale scientific communication without weakening accountability. The technologies will continue to evolve, but responsibility for authorship, judgement, and scientific accuracy still rests with people. That’s why we believe the future belongs not to humans in the loop, but to humans at the helm.

---

---

References

1. DeTora LM, et al.. Good Publication Practice (GPP) Guidelines for Company-Sponsored Biomedical Research: 2022 Update. Ann Intern Med. 2022 Sep;175(9):1298-1304. doi: 10.7326/M22-1460.
2. https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html
3. D’Souza F. The Great Decoupling. https://recognize.com/the-great-decoupling/