AI in Healthcare Risks Encoding and Amplifying Health Inequities, Experts Warn
核心洞察
Canada's AI for All strategy lacks an equitable implementation framework, risking that AI health tools will disproportionately benefit well-resourced institutions while failing marginalized communities.
AI systems trained on under-representative datasets perform worse for Black, Indigenous, and racialized patients, embedding patterned and predictable equity failures into clinical decision-making.
Research shows AI is not neutral but mirrors human psychological biases and gender stereotypes, leading to women's symptoms being dismissed and reinforcing existing healthcare inequalities.
As artificial intelligence accelerates its penetration into healthcare systems, a growing chorus of experts is warning that without deliberate equity-focused implementation, AI tools risk encoding and amplifying the very disparities they might help address. The concern is not hypothetical: AI systems are already being deployed in Canadian healthcare, and the frameworks governing their rollout may be insufficient to prevent harm to the populations most in need.
Canada's recently launched AI for All strategy — an ambitious five-year national plan targeting $200 billion in economic growth, 250,000 new jobs, and a surge in AI adoption from 12 percent to 60 percent by 2034 — has drawn both enthusiasm and pointed criticism. Dr. Kannin Osei-Tutu, a hospitalist physician and the inaugural senior associate dean for health equity and systems transformation at the Cumming School of Medicine (搜索), University of Calgary, described reading the strategy with "genuine enthusiasm and an urgent concern: has equitable implementation even been considered?"
The stakes, Osei-Tutu argues, could not be higher. "In health systems, those stakes are life and death."
The Two Gaps: Adoption and Equity
Canada's AI adoption rate sits at just over 12 percent, one of the lowest among peer nations. The AI for All strategy identifies this as a risk. But Osei-Tutu points to a second, equally critical gap: within that 12 percent, AI tools are not being adopted uniformly. They cluster in institutions with existing resources and technical infrastructure, while being designed, trained, and validated on datasets that under-represent Black, Indigenous, and racialized Canadians.
"When the diagnostic algorithm performs worse for a Black or Indigenous patient, when the clinical decision tool reflects a history of exclusion — that failure is not random," Osei-Tutu writes. "It is patterned and predictable. An equity failure, by definition."
This concern is echoed in broader research on AI and healthcare bias. AI systems draw on large datasets to "learn," but women's health issues are under-researched and under-represented in published literature, and women from marginalized groups are especially under-represented. This leads to gaps in accuracy and reliability that can affect clinical decision-making through biased AI outputs influencing diagnoses and treatment recommendations. The result is "inequitable performance — an AI-driven healthcare system that works better for some groups than others."
The Myth of AI Neutrality
A central problem identified by researchers is the persistent myth that AI is a neutral tool. Research increasingly suggests that AI reflects the same psychological biases, stereotypes, and stigmas that shape human cognition. "There is nothing truly creative about AI — it does what it is told, as interpreted through the programmer's lens," notes research published in Psychology Today. When used in healthcare, existing biases impact what AI does and have important consequences, particularly for women.
Human beings rely on mental shortcuts known as heuristics to process information quickly, but these shortcuts also lead to systematic biases and stereotypes. Gender stereotypes are among the most powerful. As some illnesses are repeatedly associated with women while others are overlooked for this population, it shapes not only public understanding and clinical expectations but also affects how symptoms are interpreted and which diagnoses are considered likely.
"Gender stereotypes that associate women with emotion can lead to situations where women's symptoms are taken less seriously or misinterpreted," the research notes. "This reflects a long history in medicine, where women's pain or illnesses have often been dismissed or attributed as a psychological manifestation." AI systems mirroring such biases perpetuate this pattern, embedding it more deeply into healthcare practice.
Design Choices That Reinforce Stereotypes
A review of research on gender stereotypes in AI highlights how widespread biased patterns are across technologies including chatbots, robots, and virtual assistants. Digital assistants are frequently given female names and voices, positioning them as helpful, supportive, and deferential, while roles associated with authority or expertise are more likely to be represented in masculine ways. These design choices shape how users perceive and interact with the system: people tend to expect female-coded AI to be warm and emotionally intelligent, while male-coded AI is seen as more competent and authoritative.
Even when AI designers attempt to create gender-neutral systems, users often infer gender from subtle cues such as tone of voice or the types of tasks the AI performs. This reflects the strength of gender schemas in human cognition.
The impact extends to AI-generated images in healthcare, which have a powerful influence on understanding — shaping expectations, guiding attention, and influencing memory. When AI-generated images consistently present narrow, biased views of healthcare roles, they reinforce stereotypes, shaping how care is delivered, who is taken seriously, and what is considered "typical" in medicine. AI systems that reproduce stereotypes can make patients feel misunderstood or marginalized, undermining patient trust. When people perceive healthcare bias, they are less likely to seek help, follow advice, or engage with professionals.
Stereotype Threat and Clinical Consequences
A related psychological effect is stereotype threat: when individuals become aware that they belong to a negatively-stereotyped group, they can experience anxiety and reduced confidence, which impairs communication and decision-making. This can lead to poorer healthcare interactions, delayed care, and worse outcomes. AI systems that signal or reinforce stereotypes intensify these effects.
Taken together, the findings point to a broad conclusion: AI is not an impartial observer of reality but a mirror of human cognition. It reflects ways in which people categorize, simplify, and stereotype, and because it operates on a large scale, it can amplify these patterns far beyond individual interactions.
Building Equity Into Implementation
At the Cumming School of Medicine (搜索), Osei-Tutu established the Health Equity AND Systems Transformation (HEST) Innovation Lab, which hosts implementation scientists, clinicians, equity scholars, and community knowledge-holders under the scientific direction of Dr. Nonsikelelo Mathe, who teaches that "implementation failure is equity failure." With investigator funding exceeding $1.16 million across six active awards, the lab's platforms include the Black Health Equity Network initiative — a bilingual digital infrastructure co-developed with the Association of Faculties of Medicine of Canada (搜索) to dismantle anti-Black racism in Canadian medicine — and SafeSpeak, an AI-enabled, trauma-informed chatbot interface for safer disclosure of racism and discrimination during medical training.
Osei-Tutu has three specific recommendations for those shaping the AI framework. First, require equity impact assessments for all health AI deployments receiving federal funding: "No diagnostic tool or clinical algorithm should reach scale without evaluating performance across race, income, language and geography." Second, fund equitable implementation science alongside technical development, recognizing that adoption is not the same as implementation — the latter being "the structured process of embedding solutions into complex human systems effectively, sustainably and equitably." Third, build equity infrastructure into the National AI Literacy Initiative, ensuring that training one million students in AI includes the history of algorithmic bias and risks for historically excluded communities.
The cost of inaction, Osei-Tutu warns, is not stasis but acceleration of harm. "Every month without an equitable implementation framework is a month in which health AI tools are trained, validated and deployed without equity as a design requirement. Disparities will not simply persist. They will be encoded, accelerated and entrenched at a scale Canada has never seen before."
Improving AI in healthcare, the research suggests, requires more than technical and ethical fixes. It demands an understanding of the complex psychological processes that shape both human thinking and its machine reflections — and a commitment to building equity into implementation before inequity becomes embedded at scale.