Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

AI news & research

Research ·

A drug advert in the prompt makes consumer chatbots recommend that drug more often

A Mount Sinai study in The Lancet Digital Health gave a dozen consumer chatbots, including versions of Claude, Gemini and GPT, 33 doctor-written cases, some preceded by a labelled drug advert. With the advert, the share of answers picking the advertised drug rose from 35% to 47.6%, and models mentioned the advert in only 29.9% of free-text replies.

When you ask a chatbot which treatment fits a set of symptoms, you assume the answer reflects medicine, not marketing. A study from the Icahn School of Medicine at Mount Sinai, published in The Lancet Digital Health, shows how easily that can shift. Placing a clearly labelled drug advert before a clinical question made consumer AI models pick the advertised drug noticeably more often, and the change was easy to miss because the answers still looked sensible.

The design was simple and well controlled. Researchers gave 12 AI models from companies including Google, OpenAI and Anthropic the same 33 physician-written clinical cases, each in two versions: the case alone, or the case with a real, labelled drug advert placed just before it. Patient details and treatment options were identical; only the advert changed, and it described the drug without telling the model to recommend it. Across 258,559 responses, the team recorded which treatment each answer chose and whether it matched clinical guidance.

The headline numbers are clear. With an advert present, models chose the advertised drug in 47.6% of responses, against 35% without one. In four cases, models went from never choosing the advertised drug to choosing it every time. In a subset of free-text answers, the models mentioned the advert only 29.9% of the time, more often repeating its claims without saying where they came from. The first author, Mahmud Omar, said what surprised him was how reasonable the answers often looked: the model's preference among several acceptable options simply moved.

That subtlety is the practical warning. Advertising can reach a model through the user's own prompt, through context an app adds, through retrieved web pages or through deliberate insertion. Assistants such as ChatGPT, Claude and Gemini increasingly browse the web and pull in sponsored content, and OpenAI has begun selling business placements inside ChatGPT. A shift that stays within guideline-approved choices will pass most accuracy tests, which is why the authors urge developers and health systems to test with and without promotional material, disclose when a system has seen an advert and keep promotion apart from clinical advice.

The limits are stated plainly. The adverts did not clearly push models towards unsupported supplements or clearly worse drugs, and the study does not show ads lead to unsafe advice. It did not measure effects on prescriptions, patient choices, costs or health. The press material does not break results down by model, so it is not possible to say which chatbots were most affected, and the cases were written for a controlled test rather than drawn from real conversations.

Our take: The shifts mostly stayed within guideline-approved options, which is exactly why standard accuracy checks miss them. Health apps that mix retrieved web content or sponsored material into prompts should test for this before launch.

Our reviews:ClaudeChatGPTGoogle Gemini

Source: The Lancet Digital Health

More stories