When GenAI shapes clinical decisions in oncology, does it matter which model you ask?

By Sarah Frazer, Head of Medical Strategy

There is no longer a question of whether oncologists use GenAI as part of the work that they do. In February and September of this year, we surveyed oncologists and found GenAI use among them increased from 64% to 91%.

The question then shifted to how that use had evolved, including where it is showing up in their working day, and what they are doing with the information it provides. Our surveys found that use for clinical decision support increased from 39% to 57%, and in September, 94% of oncologists reported taking at least one action based on something they had learned from GenAI. In some cases, those actions extended to patient management: 32% reported updating a patient’s treatment plan and 23% reported recommending a clinical trial.

And as with all data, that gave us another question to explore.

As GenAI becomes more embedded in the clinical workflow, how much do we understand about the evidence it puts in front of clinicians?

The same question can surface different evidence pictures

Using Real Chemistry’s large language model (LLM) Generative Engine Optimization (GEO) tool, HealthGEO, we assessed responses to 30 survey-informed questions in non-small cell lung cancer across ChatGPT, Claude, Gemini and Google AI Overviews. The questions reflected the kinds of tasks surveyed oncologists told us they were using GenAI for, including keeping up with new evidence, treatment selection and sequencing, biomarkers, safety and preparation for patient visits.

Across 180 pairwise platform comparisons, 19% contained a potentially meaningful difference. At first glance, that might sound like a story about AI models disagreeing with each other, but it was more nuanced than that. Where responses differed, the platforms selected different practice-changing trials or guideline updates in 65% of comparisons, and different biomarkers or eligibility criteria in 47%. The issue was not necessarily that one platform was recommending one course of action and another something completely different – it was that the evidence in view could be different.

Oncologists are bringing far more to a decision than an AI response, including the individual patient, their own expertise, guidelines, the wider evidence base and multidisciplinary input. But as GenAI becomes another way into that evidence base, what it chooses to put in front of the oncologist matters.

What is sitting behind the answer?

We also saw clear differences in where the platforms were drawing their information from.

Claude cited 567 references and had the broadest scientific journal footprint. Gemini cited 254 references and Google AI Overview 245, with both drawing more heavily on oncology news. ChatGPT cited 178 references, drawing primarily from government and regulatory sources and a smaller number of leading journals.

More references do not automatically mean a better answer, and our findings do not show that one platform is clinically superior to another. What they do show is that the route from a clinical question to an AI-generated answer can look quite different depending on the platform.

Language and the question itself mattered too. Variation was greatest when the questions related to evidence synthesis and staying current. Matched English and French prompts surfaced broadly similar leading sources, but French prompting increased the visibility of France-specific guidance. Open-ended questions tended to amplify evidence differences, while more targeted questions converged more closely on the same core recommendation.

Our work does not answer the technical “why” behind these differences, which evidence gets seen and which evidence does not. But as GenAI becomes more embedded in clinical workflows, it is a question the healthcare community needs to start asking and answering.

This is a shared healthcare challenge

The obvious response could be to put the emphasis back on the oncologist: ask a more specific question, look at the sources and check important information against the underlying evidence. Those things matter, but they are only part of the answer.

As GenAI is now established as another interface between clinicians and scientific evidence, there is a role for the wider healthcare community too.

For technology developers, that means helping us better understand the evidence pathway, including why particular sources are surfaced, how that changes between models and questions, and what relevant evidence might be missing.

For pharmaceutical leaders and the scientific community, it means thinking beyond the publication. Increasingly, evidence may be retrieved and synthesized by an AI before a clinician encounters it from more traditional sources. We therefore need to understand whether authoritative evidence is visible and appropriately represented in these new information environments.

And for healthcare organizations, it means helping clinicians understand the AI tools being provided to them as they adopt them, including how to get the most out of prompting and what an AI response represents.

If GenAI is shaping which evidence reaches clinicians, then making sure the right evidence comes into view cannot sit with clinicians alone. It is becoming a shared responsibility across the healthcare community.

Who decides what evidence gets seen? 

None of this means GenAI is making clinical decisions. Clinicians are, and their judgment remains central.

But our findings suggest that route is not neutral: that the platform used, the question asked and the language chosen can all influence what comes into view. That means the next challenge is not simply whether an AI-generated answer is accurate. It is whether the healthcare community can see and scrutinize the evidence-selection layer forming between published science and clinical judgement. 

If we do not understand how evidence is selected, prioritized or omitted, a plausible answer can easily be mistaken for a complete view. That is why LLM evidence selection can no longer be treated as an invisible technical process occurring in the background.

When GenAI influences what clinicians see, the way it selects evidence becomes part of wider healthcare decision-making, whether we acknowledge it or not. 

---

Source

Frazer S, Rafferty H, Voorhorst K, et al. Generative AI (GenAI) in oncology practice: global usage patterns and the evidence large language models (LLMs) surface. Poster 242eP. Presented at ESMO Congress 2026, Madrid, Spain, 23–27 October 2026.