Ongoing studies reveal AI underdiagnoses women, offers less empathetic care to minorities, prompting regulatory efforts to manage discrimination risks.
Recent research in the US has been showing that AI tools used in healthcare have increasing shown significant biases against women and ethnic minorities, raising concerns about discriminatory outcomes as these systems become more integral to clinical decision-making.
Studies from institutions such as MIT; the London School of Economics; and Emory University reveal that widely used AI models — including OpenAI’s GPT-4, Meta’s Llama 3, and healthcare-specific tools such as Palmyra-Med — produce recommendations that under-diagnose women and give less empathetic responses to Black and Asian patients.
One example cited involved identical case notes portraying an elderly man with “complex medical history” and poor mobility, while the same notes for a woman depicted her as “independent”, illustrating gender bias in assessments.
Google’s Gemma AI, used by UK local authorities, was found to undervalue women’s health issues, and patients with informal language or uncertain symptoms were often advised against seeking care, impacting those less comfortable with technology or English.
The problem of embedded data biases
These revelations contradict recent high-profile claims by companies like Microsoft, which in mid-2025 asserted that its AI-powered diagnostic tool was four times more accurate than physicians in handling complex cases with 85% accuracy.
Medical experts express skepticism, emphasizing that performance metrics alone do not guarantee fair treatment across diverse populations. MIT’s associate professor Marzyeh Ghassemi has highlighted that AI models could base guidance on perceived race, leading to unequal care with some patients receiving less supportive recommendations. Such biases stem largely from AI systems learning from historical medical data that already contain embedded prejudices, risking the reinforcement of existing healthcare disparities.
The problem of reactive AI policy
In response to these concerns, the US Department of Health and Human Services (HHS) had implemented a final rule in May 2024 under Section 1557 of the Affordable Care Act, legally requiring healthcare organizations to manage discrimination risks associated with AI tools.
This regulation mandates “reasonable efforts” to identify whether AI decision support tools use protected characteristics such as race or sex and to mitigate associated risks. Alongside federal oversight, several US states now require physician review of AI health insurance claim denials to protect patient interests.
Google and OpenAI are stating they are refining models to reduce bias. However, experts warn that without fundamental changes in training data and development practices, AI risks perpetuating longstanding inequities in healthcare delivery.
