AI Can Calculate Medical Probabilities, But It Cannot Answer the Question That Matters Most

Artificial intelligence is moving deeper into everyday health decisions, yet it remains unable to address what physicians and researchers describe as one of the most important questions in modern healthcare: "What matters most to you?" In a commentary published in the Los Angeles Times, two physicians and researchers argue that while AI can calculate probabilities, it cannot determine what those probabilities mean to a particular person.

The authors recount a familiar scenario: a longtime friend called after receiving conflicting guidance from two doctors, an MRI, an online AI tool, and a stack of articles. "Everything tells me something different. The AI says I might need surgery. What should I do?" The authors' response — "What matters most to you?" — was met with a long pause, which they describe as "one of the most important moments in modern healthcare — and it is exactly the question artificial intelligence is unable to address."

The Limits of Probability in Preference-Sensitive Decisions

The central argument is that for many common conditions, the medical evidence does not point to a single "right" answer. "The biology is often close. What determines the success of an outcome is whether the choice fits the person making it," the authors write. They illustrate this with examples: some patients with back pain (搜索)搜索该疾病 want the fastest possible return to physically demanding work, even if it means surgery, while others want to avoid an operation at almost any cost, even if recovery takes longer. "The scan may look the same. The lives behind the scan are not."

This distinction matters because some of the most common and most expensive medical decisions are not purely biological. The authors point to questions such as whether someone with low-risk prostate cancer (搜索)搜索该疾病 should choose surgery, radiation, or careful monitoring; whether a person with atrial fibrillation (搜索)搜索该疾病 should undergo a procedure or manage the condition with medication; and whether a patient with chronic knee or back pain (搜索)搜索该疾病 should operate now or try months of physical therapy first.

In these situations, the medical differences between options are often small or uncertain. "What makes the biggest difference is whether the treatment aligns with the patient's goals: tolerance for risk, willingness to undergo recovery, ability to adhere to long-term therapy or simply what kind of life they want to live."

What AI Knows — and What It Cannot See

The authors acknowledge that artificial intelligence may know more medicine than any individual physician, synthesizing millions of scientific papers, clinical studies, and patient records in seconds. "Yet it knows remarkably little about the person sitting across from it. AI does not know a patient's goals, fears, obligations, tolerance for risk or personal definition of a good outcome."

A second patient story underscores this blind spot. A retired teacher was referred after an AI-based symptom checker flagged a heart rhythm abnormality and "favored" an invasive procedure. The patient arrived frightened, convinced there was one correct path. In conversation, it became clear that what mattered most was avoiding a long recovery and staying healthy enough to travel to see grandchildren. Medication and monitoring — "less dramatic, but well-supported by evidence" — fit those goals better. "The AI wasn't wrong. It just didn't know what mattered."

The Economic and Clinical Stakes

The authors frame this blind spot as far from trivial. "Roughly a quarter of U.S. healthcare spending flows through decisions in which patient preferences meaningfully affect outcomes." When those preferences are ignored — by people or by algorithms — care becomes misaligned, which can mean unnecessary procedures, poor adherence, regret, and rising costs without better health.

For consumers facing a recommendation from an app, portal, or "smart" tool, the authors propose three questions: "Best for whom?" — asking whether a tool means best on average or best for someone with the patient's specific priorities; "What does this system not know about me?" — since AI can see lab values and imaging results but not a patient's job, family responsibilities, fears, or what they are trying to get back to; and "What happens if I wait or choose differently?" — noting that many important medical decisions are not emergencies.

A Broader Evidence Question

The commentary's caution aligns with a separate study published in Nature Medicine titled "Is AI actually improving healthcare?" As summarized in STAT News, the study answers its own question affirmatively — "Yes, it is" — while offering a significant caveat: "In many cases, we do not know." Many AI tools are so new that it remains unclear whether they improve patient outcomes.

STAT News argues that treating AI as a monolith obscures this distinction. Asking "Is AI actually improving health care?" is compared to asking "Do lasers improve surgery?" In the hands of a skilled surgeon using a validated tool, lasers allow for lifesaving precision; in an unproven setting, the question of efficacy remains open.

The Los Angeles Times authors conclude that AI is becoming a powerful partner in medicine, capable of helping explain options, surface evidence, and reduce confusion. "But it should inform human decisions, not replace them." They close with a pointed observation: "AI may know more medicine than any physician. It knows far less about any patient. And it knows least about the conversation between them. The most important variable in your healthcare is not in any algorithm. It is you."