BMJ Best Practice: helping to shape the future of clinical AI
As healthcare organisations explore the safe use of artificial intelligence, BMJ Best Practice is increasingly being used as a trusted global source of evidence based, point of care clinical information.
In 2025, researchers from Google Research and Google DeepMind selected BMJ Best Practice as one of only two core clinical guideline resources used to develop and evaluate AMIE, a conversational AI system designed to support clinical management of disease progression, therapeutic response, and safe medication prescription. The other resource was guidance from the UK’s National Institute for Health and Care Excellence.
BMJ Best Practice recommendations informed the development of 100 simulated patient scenarios used to assess AMIE’s clinical management responses, including investigations, treatments, and follow-up decisions.
The study found that AMIE’s responses were non-inferior to those of primary care physicians, showing how evidence based clinical knowledge can support safe clinical management plans aligned with current evidence and guidelines.
This places BMJ Best Practice among the authoritative evidence sources against which clinical management decisions were assessed, and shows how it goes beyond supporting clinicians at the point of care today; it is also helping define the evidence base used to develop and evaluate the next generation of clinical AI systems.
Towards Conversational AI for Disease Management, Nature
Abstract
While large language models (LLMs) have shown promise in diagnostic dialogue1, their capabilities for effective management reasoning—including disease progression, therapeutic response, and safe medication prescription—remain under-explored. We advance the previously demonstrated diagnostic capabilities of the Articulate Medical Intelligence Explorer (AMIE)1−3 through a new LLM-based agentic system optimized for multi-visit clinical management and dialogue. To ground its reasoning in authoritative clinical knowledge, AMIE leverages Gemini’s long-context capabilities4, combining in-context retrieval with structured reasoning to align its output with up-to-date clinical practice guidelines and drug formularies. In a randomized, blinded virtual Objective Structured Clinical Examination (OSCE) study, AMIE was compared to 21 primary care physicians (PCPs) across 100 multi-visit case scenarios designed to reflect UK NICE Guidance and BMJ Best Practice guidelines. AMIE was non-inferior to PCPs in management reasoning as assessed by specialists and scored better in both preciseness of treatments and investigations, and in its alignment with and grounding in clinical guidelines. To benchmark medication reasoning, we developed RxQA, a multiple-choice question benchmark derived from two national drug formularies (US, UK) and validated by board-certified pharmacists. Though AMIE and PCPs both benefited from the ability to access external drug information, AMIE outperformed PCPs on higher difficulty questions. While further research would be needed before real-world translation, AMIE’s strong performance across evaluations marks a significant step towards conversational AI as a tool in disease management.
Liévin, V., Palepu, A., Weng, WH. et al. Towards Conversational AI for Disease Management. Nature (2026). https://doi.org/10.1038/s41586-026-10764-5
- BMJ Best Practice was selected as one of only two core clinical guideline resources used in the study, alongside National Institute for Health and Care Excellence guidance. This places BMJ Best Practice among the authoritative sources against which clinical management decisions were assessed.
- The researchers built 100 multi-visit patient scenarios specifically around NICE guidance and BMJ Best Practice recommendations. The “ground truth” management plans for diagnosis, investigation, treatment, and follow-up were derived from these sources.
- Both physicians and the AI system were given access to a corpus containing 527 NICE guidelines and 100 BMJ Best Practice topics. The AI’s ability to produce safe and guideline-aligned management plans depended in part on retrieval and reasoning over this content.
- One of the key outcome measures was how well management plans aligned with NICE and BMJ Best Practice guidance. AMIE was found to be non-inferior to primary care physicians and scored higher on guideline alignment and grounding.
BMJ Best Practice: evidence based, clinical decision support information for the multidisciplinary team
BMJ Best Practice takes you quickly and accurately to the latest evidence-based information, whenever and wherever you need it. We are the only point-of-care tool to support the management of single conditions and patients with more complex comorbidities.
Rated 4.8 on Google Play and 4.9 on Apple Store, making BMJ Best Practice one of the most highly rated clinical decision support tool apps in the world.
Related posts
-
Innovative device tackles racial bias in health monitoring, ensuring accurate readings for all skin tones -
Building a powerful advocacy voice -
How Ugandan research shaped covid-19 treatment -
Practical lessons for publishers
