AI this week: invented answers, AI doctors, a free AI school

This week: why your chatbot invents things on purpose, a panel of AI doctors that just passed peer review, and a free OpenAI school that gives you a badge when you pass.

Your AI's made-up answers started as a feature

You ask a chatbot for a book recommendation, it sounds perfect, and then you search for the title and it doesn't exist.

It's tempting to call that a bug the next update will fix. Benjamin Lindquist, a historian at MIT, traces it back to the 1950s instead, when researchers put randomness into their machines deliberately. A computer that only gave the expected answer wasn't impressive; they wanted one that could surprise them. Many early neural networks were built by psychologists such as Frank Rosenblatt, who hoped the machines' odd mistakes would show how our own brains come up with ideas.

Today you use AI because you expect it to be right, and AI companies work hard to stop it saying anything strange. Lindquist calls that a "sanding down" of strangeness, and argues the strangeness is still there underneath, only suppressed. His paper is in the journal Past & Present.

Our take: the streak that gives you a surprising idea and the one that invents a book are probably the same streak. Let a chatbot wander when you want ideas. When you are going to act on a fact, ask for the source and open it.

A panel of AI doctors on medicine's hardest cases

When a symptom doesn't fit anything obvious, getting an answer can take months, one specialist ruling something out before sending you to the next.

Microsoft's Diagnostic Orchestrator (MAI-DxO) puts a whole room on the case. It runs a panel of virtual doctors: each suggests what might be wrong and which question or test should come next, and they argue over several rounds as results arrive. Like a real doctor, it starts with a short summary and has to ask for each further detail.

On 304 difficult cases from the New England Journal of Medicine, it reached the right diagnosis 83% of the time. Generalist doctors got 22% from memory and 36% when they could look things up. On more ordinary emergency-department cases it scored 80% against the doctors' 47%, often with fewer tests. The study is now published in Science.

The catch: these were written case files, not patients in a room, and the comparison was with generalists rather than specialists or a full hospital team. Every author works for Microsoft. The next step matters more than the score: a study at Beth Israel Deaconess Medical Center in Boston where patients talk to the system directly, measured against doctors' own judgement.

Our take: hard cases drag on because one doctor only has so many hours. A panel like this is most useful as the second opinion your doctor never had time to get, with the decision still theirs.

A free school for getting better at AI

Learning something new usually means a bill and a timetable that isn't yours.

OpenAI Academy has neither. The courses are free and self-paced for anyone with a ChatGPT account, and in September OpenAI grew it to four learning paths and 14 courses: everyday work, building with Codex and the API, leading AI adoption, and teaching and studying. Most courses take between half an hour and three hours. Pass the end-of-course test with 80% or more and you get a badge.

The catch: OpenAI says plainly that badges are not certifications. They show you finished a course, not that an employer should treat you as qualified.

Our take: using AI every day teaches you what works for the jobs you already give it. A course shows you the jobs you never thought to try, which is where the bigger time savings tend to hide. The 70-minute foundations course is a cheap way to find out which kind of user you are.