Algorithmic Bias and Health Equity in Clinical AI: Taxonomy, Evidence, Audit Frameworks, and Mitigation Strategies

The Health Equity Stakes of Clinical AI Adoption

Clinical AI is no longer a research-stage technology. As of May 2024, the FDA had authorized 882 AI/ML-enabled medical devices, and a 2026 KFF analysis found that 84% of health insurers were using AI in clinical or administrative functions. These are not projections — they describe a deployed infrastructure that is actively shaping patient care across risk stratification, diagnostic imaging, scheduling, and clinical decision support.

The scale of deployment makes algorithmic bias a systemic health equity concern, not a marginal edge-case problem. When a model trained on historically unrepresentative data is applied to millions of patients, its performance gaps are not statistical artifacts — they are operational mechanisms that can systematically under-serve specific demographic groups at population scale.

The 2026 regulatory context sharpens this concern. Federal equity mandates that previously required algorithmic fairness analysis have been rescinded. The governance burden has shifted substantially onto health systems themselves, at precisely the moment when AI deployment is accelerating. Understanding where bias originates, how to detect it, and which mitigation strategies are evidence-supported is no longer optional for health system leaders and clinical informaticists — it is a core institutional responsibility.

This article is organized into four substantive parts: a formal taxonomy of bias types mapped to the AI lifecycle; domain-specific empirical evidence of disparate clinical AI performance; three major audit frameworks operating at distinct governance levels; and technical and systemic mitigation strategies, with their documented trade-offs.

A Taxonomy of Bias Types Across the AI Lifecycle

Bias in clinical AI does not originate in a single place. A 2025 narrative review by Hasanzadeh et al. in npj Digital Medicine organizes bias across three origin categories — human, data, and algorithmic/deployment — and maps each subtype to the AI model lifecycle phase where it enters. This structure is essential because the mitigation entry point for each bias type is determined by where it originates, not where it surfaces.

One foundational framing principle applies throughout: race and ethnicity are social constructs historically used to categorize and differentiate groups, not biological variables. Whether to include race or ethnicity as a model feature is a contextual and ethical design decision, not a technical default. The same principle applies to any demographic variable used as a proxy for biological or clinical characteristics.

Human-Origin Biases

Human-origin biases enter the AI lifecycle through the decisions made by the people who design, develop, and deploy systems. Three subtypes are clinically significant:

• Implicit bias: Unconscious assumptions held by developers, clinicians, or annotators that shape problem framing, feature selection, and labeling decisions — often without awareness.
• Systemic bias: Structural inequities embedded in the healthcare system itself — differential access, documentation practices, and care quality — that are then encoded into training data as if they represent neutral ground truth.
• Confirmation bias: The tendency to favor evidence that confirms existing clinical assumptions during model evaluation and interpretation, which can cause disparate performance to go unrecognized or be rationalized away.

Data Biases

Data biases arise from the composition, collection, and measurement properties of training datasets. They are among the most extensively documented bias sources in clinical AI literature, but they are not the only source — and larger datasets do not automatically resolve them.

• Representation bias: Demographic groups present at lower rates in training data than in the target clinical population, leading to models that generalize poorly to underrepresented groups.
• Selection/sampling bias: Non-random data collection that systematically excludes certain populations — for example, datasets drawn from academic medical centers that do not reflect community hospital or safety-net populations.
• Measurement bias: Differential accuracy or completeness in how clinical variables are recorded across demographic groups, such as pulse oximetry readings that systematically overestimate oxygen saturation in patients with darker skin tones.
• Aggregation bias: Treating heterogeneous subgroups as a single homogeneous population during model training, obscuring subgroup-specific patterns and producing models that perform adequately on average but poorly for specific groups.

Algorithmic and Deployment Biases

• Feature selection bias: Including features that are correlated with protected attributes — even when those attributes are not explicitly modeled — allowing demographic disparities to influence predictions indirectly.
• Proxy variable bias: Using variables that serve as proxies for race, ethnicity, socioeconomic status, or other protected characteristics. Healthcare cost, zip code, insurance type, and prior no-show rates are common examples — each encodes historical disparities and can reproduce them algorithmically.
• Automation bias: Clinicians over-relying on AI outputs and under-applying independent judgment, which can amplify model errors and reduce the chance that disparate predictions are caught before affecting care.
• Feedback loop bias: When model outputs influence future data collection — for example, a risk score that determines which patients receive follow-up testing — biased predictions generate biased future training data, compounding the original disparity over time.
• Dismissal/alert fatigue: High alert volumes cause clinicians to override AI recommendations indiscriminately, which can disproportionately affect patients whose conditions are flagged at higher rates by biased models.

Documented Evidence of Disparate Clinical AI Performance

The evidence base for disparate clinical AI performance has grown substantially. A 2024 systematic review of 30 AI studies conducted over a ten-year period found a significant association between AI utilization and exacerbation of racial disparities in health outcomes, including longer wait times, lower accuracy in predicting mental health outcomes, and underdiagnosis — disproportionately affecting Black and Hispanic patients. A separate analysis by Kumar et al. (2023) found that 50% of sampled AI studies were at high risk of bias. Using the PROBAST framework, Chen et al. examined 555 neuroimaging-based psychiatric AI models and rated 83% at high risk of bias, with 97.5% of studies including only subjects from high-income regions and only 15.5% including external validation.

The following domain-specific cases each illustrate a distinct bias subtype from the taxonomy above and a corresponding mitigation entry point.

Risk Stratification and Population Health Management

Obermeyer et al.'s 2019 analysis in Science remains the most widely cited demonstration of proxy variable bias in clinical AI. A widely deployed population health management algorithm used healthcare cost as a proxy for health need. Because Black patients historically receive less care per equivalent health burden — a product of systemic access inequities — the algorithm systematically assigned Black patients lower risk scores than White patients with the same underlying conditions. Black patients were therefore less likely to be enrolled in care management programs.

When the algorithm was recalibrated to use direct chronic condition counts rather than cost as the target variable, the proportion of Black patients enrolled in high-risk care management programs nearly tripled — rising from 17.7% to 46.5%. This case illustrates that the bias was not in the model's mathematical structure but in the choice of what to optimize, a decision made during model design. The mitigation entry point is proxy variable scrutiny at the conception and design phase.

Medical Imaging: Chest X-Ray and Cardiac MRI

Seyyed-Kalantari et al.'s 2021 study in Nature Medicine evaluated chest X-ray AI models across multiple large public datasets and found systematic underdiagnosis of pathology in Black patients, Hispanic patients, and patients with low socioeconomic status. The underdiagnosis pattern was consistent across datasets and models, suggesting that it reflects training data composition rather than model-specific implementation choices. This is a representation and aggregation bias operating at the data collection phase.

A cardiac MRI deep learning segmentation model (nnU-Net) trained on UK Biobank data demonstrated a Dice Similarity Coefficient (DSC) of 93.5% for White subjects but only 84.5% for Black and Mixed-race subjects — a nearly 9-percentage-point performance gap attributable to the training dataset's demographic composition. Applying stratified batch sampling during training improved DSC for Black subjects from 85.88% to 93.07%, substantially closing the gap. This case provides a direct demonstration of how a pre-processing mitigation technique resolves a representation bias at its source.

Mental Health NLP

Natural language processing models used for depression severity prediction have shown lower performance for Black patients relative to White patients. This pattern reflects both representation bias in training corpora — which tend to draw from clinical notes generated in settings that disproportionately serve White patients — and measurement bias, since documentation practices for mental health conditions vary across clinical settings and patient populations. The mitigation entry point is dataset diversification and measurement instrument validation by demographic subgroup.

Appointment Scheduling Algorithms

A machine learning scheduling algorithm studied in the KFF 2026 analysis produced wait times for Black patients that were 33% longer than for White patients. The algorithm used socioeconomic indicators — employment status, zip code, insurance type, and prior no-show rates — as scheduling inputs. Each of these variables is a proxy for race and socioeconomic status, and each encodes historical disparities in healthcare access and employment. The bias subtype is proxy variable bias; the mitigation entry point is feature audit during model design, combined with disparate impact analysis post-deployment.

Race-Corrected Clinical Algorithms

Several established clinical calculators — including eGFR (estimated glomerular filtration rate), spirometry reference equations, and VBAC (vaginal birth after cesarean) success prediction — historically incorporated race as a correction variable, treating it as a biological modifier. These corrections have been shown to systematically disadvantage Black patients: the race-adjusted eGFR formula assigned Black patients higher kidney function estimates than their actual function warranted, delaying referral for nephrology evaluation and transplant listing. The underlying error is categorical: race is a social construct, not a biological variable, and its inclusion as a clinical correction factor encodes historical measurement inequities rather than capturing genuine physiological differences. Most major clinical guidelines have now removed race corrections from these algorithms.

Three Audit Frameworks Operating at Different Governance Levels

Three major frameworks have emerged to structure algorithmic equity assessment in clinical AI: HEAAL, the FDA Total Product Lifecycle (TPLC) equity framework, and STANDING Together. These are not interchangeable tools. They operate at structurally distinct governance levels — health system adoption decisions, device regulatory lifecycle, and health dataset documentation respectively — and address different accountability questions. Using one does not substitute for the others.

HEAAL: Health System Adoption Level

The Health Equity Across the AI Lifecycle (HEAAL) framework, published by Kim et al. in PLOS Digital Health (2024), was co-designed with 77 representatives from 10 U.S. healthcare delivery organizations. It is designed for use by health systems evaluating whether to adopt, continue, or discontinue an AI solution — not for device developers or dataset curators.

HEAAL assesses five health equity domains — accountability, fairness, fitness for purpose, reliability and validity, and transparency — across eight AI adoption decision points. It contains 37 step-by-step procedures for evaluating existing AI solutions and 34 for new ones, providing concrete procedural guidance rather than abstract principles.

The framework's most important conceptual contribution is its explicit decoupling of algorithmic fairness from health equity impact. HEAAL models four distinct scenarios that can arise when an AI system is deployed:

• Scenario A: Algorithm performs well across subgroups, and the clinical setting can act on outputs equitably. Expected result: health equity improvement.
• Scenario B: Algorithm performs well across subgroups (technically fair), but is deployed in an under-resourced setting that cannot act on model outputs with equal effort for disadvantaged patients. Result: health equity worsened despite technical fairness. This scenario demonstrates that a technically fair algorithm can worsen real-world equity.
• Scenario C: Algorithm appears to perform poorly on historical data for a disadvantaged subgroup — but the poor historical performance reflects underdiagnosis of that group in the training data, not true absence of disease. Prospective implementation with proactive outreach to this group could improve equity. This is the 'inequitable underdiagnosis' scenario: an apparently unfair algorithm may identify previously invisible patients.
• Scenario D: Algorithm performs poorly across subgroups and the setting cannot compensate. Expected result: health equity worsened.

FDA Total Product Lifecycle (TPLC) Equity Framework: Device Regulatory Level

Abramoff et al.'s TPLC equity framework, published in npj Digital Medicine (2023), extends the FDA's Total Product Lifecycle model to embed equity analysis at each of six phases: conception, design, development, validation, access and marketing, and monitoring. It is addressed primarily to AI device developers and regulatory reviewers, not health system procurement teams.

The framework's defining principle is phase independence: the equity impact at each lifecycle phase is independent of all other phases. Even when all potential bias has been mitigated in earlier phases, the next phase can still introduce new equity effects. This means that a device that passes equity review at the development and validation stages can still produce disparate outcomes at the access and marketing stage — for example, if it is deployed exclusively in well-resourced health systems that serve predominantly White patients.

For measuring bias, the TPLC framework recommends subgroup-disaggregated performance testing and population-achieved sensitivity and specificity — a metric that captures the combined impact of access inequities and diagnostic performance disparities. This is particularly relevant for identifying 'invisible populations' who lack routine healthcare access and are therefore underrepresented in both training data and validation cohorts.

STANDING Together: Dataset Documentation Level

The STANDING Together consensus recommendations, published simultaneously in NEJM AI and Lancet Digital Health (2025), were developed through an international Delphi process involving more than 350 representatives from 58 countries. The initiative — established in 2021 as part of the NHS AI Lab's AI Ethics initiative — produced 29 consensus recommendations organized in two parts: documentation of health datasets and use of health datasets.

STANDING Together operates at a different level than HEAAL or TPLC. Its recommendations are addressed to dataset curators, data custodians, and researchers who create and share health datasets used to train AI systems. The central premise is that no dataset is free of limitations, and that transparent communication of data limitations should be treated as a quality signal — not a liability — while absence of this information should itself be treated as a limitation.

For health systems and researchers using AI tools built on external datasets, STANDING Together provides a checklist framework for evaluating whether the datasets underlying a model have been documented with sufficient transparency to support equity assessment.

Supporting Governance References: NIST AI RMF and CHAI Blueprint

The NIST AI Risk Management Framework (AI RMF) and the Coalition for Health AI (CHAI) Blueprint for Trustworthy AI Implementation Guidance provide governance scaffolding that complements the three primary frameworks above. Neither is a bias audit tool in itself, but both provide organizational risk management structures within which HEAAL procedures, TPLC equity analysis, and STANDING Together documentation requirements can be embedded. Health systems building institutional AI governance programs should treat these as the organizational container, with HEAAL providing the health equity-specific procedural content.

Technical Mitigation Strategies: Pre-Processing, In-Processing, and Post-Processing

Technical bias mitigation strategies operate at three stages of the model development pipeline. A 2023 scoping review by Cary et al. in Health Affairs examined 109 articles and found that almost every study reported some success using these strategies — but also that this success sometimes came at a cost to other statistical performance measures. The fairness-accuracy trade-off is real, context-dependent, and requires clinical benefit-harm analysis rather than a blanket technical resolution.

Pre-Processing Strategies

Pre-processing interventions modify the training dataset before model training begins. They address representation and sampling biases at their source.

• Stratified sampling: Ensuring that training batches include proportionally representative samples from each demographic subgroup. The cardiac MRI case demonstrates its effectiveness: stratified batch sampling improved DSC for Black subjects from 85.88% to 93.07%, nearly matching White subject performance (93.5%).
• Reweighting: Assigning higher loss weights to underrepresented or historically disadvantaged subgroups during training, so the model pays proportionally more attention to their error patterns.
• SMOTE (Synthetic Minority Oversampling Technique): Generating synthetic training examples for underrepresented groups by interpolating between existing samples, increasing effective representation without requiring new data collection.
• Disparate impact remover: A preprocessing transformation that modifies feature distributions to reduce correlation with protected attributes while preserving predictive information.

In-Processing Strategies

In-processing strategies modify the model training procedure itself to incorporate fairness objectives alongside predictive accuracy.

• Fairness-constrained objective functions: Adding fairness penalties or constraints directly to the loss function, requiring the model to satisfy demographic parity or equalized odds conditions during optimization.
• Adversarial debiasing: Training a secondary adversarial network to predict protected attributes from model representations, then penalizing the primary model for producing representations that enable this prediction.
• Stratified cross-validation: Ensuring that validation folds preserve demographic subgroup proportions, so that performance estimates reflect subgroup-level variation rather than aggregate averages.
• Federated learning: Training models across distributed datasets at multiple sites without centralizing patient data, enabling training on more demographically diverse populations while preserving privacy.
• Red teaming: Structured adversarial testing by teams specifically tasked with identifying failure modes and disparate performance patterns before deployment.
• Cost-sensitive learning: Assigning asymmetric misclassification costs to reflect the differential clinical consequences of errors for different patient groups.

Post-Processing Strategies

• Subgroup threshold recalibration: Setting different decision thresholds for different demographic subgroups to equalize sensitivity or specificity across groups. This is the most operationally accessible post-processing approach but requires careful clinical justification for threshold choices.
• Varying cutoff points: Adjusting prediction cutoffs based on subgroup-specific calibration data, particularly relevant when base rates differ across demographic groups.
• Disparate impact analysis: Systematic post-hoc testing of model outputs for statistically significant performance differences across protected attribute subgroups, used both for initial deployment review and ongoing monitoring.

Fairness Metrics Used to Evaluate Mitigation

Evaluating whether a mitigation strategy has succeeded requires selecting an appropriate fairness metric. The choice of metric encodes a normative judgment about what constitutes equitable treatment and is not a purely technical decision.

Governance and Systemic Mitigation: Beyond Technical Fixes

Technical mitigation strategies address bias at specific points in the model pipeline. But the Cary et al. scoping review of 109 articles found no consensus on a single best technical practice, and concluded that optimal mitigation depends on patient population, clinical setting, algorithm design, and bias type. This is not a hedge — it is a structural conclusion with direct governance implications: context-specificity requires institutional decision-making infrastructure, not just algorithmic adjustment.

Governance and systemic mitigation strategies address the organizational conditions that allow bias to enter and persist. They are structurally equal in importance to technical approaches, and in some cases are the only effective intervention for biases that originate in human decisions or systemic inequities rather than in data composition.

• Professional diversity on AI development teams: Teams that include clinicians, patients, community members, ethicists, and representatives from affected demographic groups are better positioned to identify implicit bias assumptions during problem framing and feature selection — the lifecycle phases where human-origin biases most commonly enter.
• Health equity by design: Treating equity as a core development requirement rather than a post-hoc audit criterion. This means specifying equity objectives alongside predictive performance objectives at the outset of model development, and evaluating progress against both throughout the lifecycle.
• Proxy variable scrutiny: Systematic review of all model features for correlation with protected attributes before training. Socioeconomic variables — zip code, insurance type, employment status, prior utilization — require particular scrutiny in clinical applications.
• Auditable algorithm requirements: Procurement standards that require vendors to provide subgroup-disaggregated performance data, documentation of training dataset demographics, and access to model feature importance information sufficient to support proxy variable review.
• Participatory design with affected communities: Involving patients and community members from groups historically disadvantaged by clinical AI in the design, evaluation, and governance of AI systems — not as symbolic consultation but as substantive co-design participants with decision-making input.
• Transparent reporting of training dataset demographics: Requiring that all AI systems deployed in clinical settings be accompanied by documentation of the demographic composition of their training and validation datasets, consistent with STANDING Together recommendations.
• Governance structures supporting algorithmic accountability: Institutional AI governance committees with explicit health equity mandates, defined escalation pathways for flagged performance disparities, and authority to suspend or modify deployed systems.

The 2025–2026 U.S. Regulatory Context and the Self-Governance Imperative

The federal regulatory landscape governing algorithmic equity in healthcare AI has shifted substantially since January 2025. Understanding this shift is essential for health systems building AI governance programs, because it determines where accountability currently rests.

Prior Regulatory Framework

Under the Biden administration, a series of executive orders and regulatory actions established federal requirements and expectations around equitable AI in healthcare. HHS Section 1557 proposed rulemaking addressed algorithmic discrimination in health programs receiving federal financial assistance. These mandates created a baseline federal accountability structure for algorithmic equity.

Executive Orders Reshaping Federal AI Governance

Executive Order 4148 (January 2025) rescinded the Biden-era equitable AI mandates that had required federal agencies and contractors to assess and mitigate algorithmic discrimination. Executive Order 14179 shifted the federal government's stated AI policy focus from algorithmic fairness to innovation promotion, removing equity analysis requirements from federal AI procurement and deployment guidance.

Executive Order 14365 (December 2025) directed the Department of Justice to establish an AI Litigation Task Force, operational in January 2026, with the stated purpose of challenging state laws governing AI that are inconsistent with federal policy. The practical effect has been to create legal uncertainty around state-level algorithmic accountability requirements.

State-Level Fragmentation: Colorado SB 24-205

Colorado's Consumer Protections for Artificial Intelligence Act (SB 24-205), which would have established algorithmic impact assessment requirements for high-risk AI systems, has faced implementation delays due to legal challenges. The DOJ AI Litigation Task Force's mandate to challenge state AI laws inconsistent with federal policy creates ongoing uncertainty about whether state-level algorithmic equity requirements will survive legal scrutiny.

The Governance Vacuum and Health System Accountability

The combination of rescinded federal equity mandates, challenged state requirements, and accelerating clinical AI deployment has created a governance vacuum. In the absence of enforceable federal or state algorithmic equity requirements, health systems are the primary accountability holders for the equity performance of AI tools they deploy.

This is not a temporary gap that will be resolved by future federal action. The current policy trajectory suggests that institutional self-governance — supported by frameworks like HEAAL, the TPLC equity model, and STANDING Together — will remain the primary mechanism for algorithmic equity assurance in U.S. healthcare for the foreseeable future. Health systems that have not yet built institutional AI governance capacity with explicit equity mandates are operating without the infrastructure to detect or respond to the documented disparities described in this article.

• Federal equity mandates rescinded: EO 4148 and EO 14179 removed federal algorithmic fairness requirements from AI procurement and deployment guidance.
• State-level requirements under legal challenge: DOJ AI Litigation Task Force (January 2026) is actively challenging state AI laws; Colorado SB 24-205 implementation delayed.
• FDA device authorization does not address equity: FDA 510(k) clearance and De Novo authorization do not require subgroup-disaggregated performance demonstration as a clearance condition in most cases. Clearance is not an equity certification.
• Health system accountability is primary: In the current regulatory environment, institutional AI governance programs with explicit equity mandates are the operative accountability mechanism.

Research Gaps and Practical Recommendations for Health Systems

The evidence base for algorithmic bias in clinical AI has grown substantially, but it retains significant structural gaps that limit the translation of research findings into institutional practice.

Key Research Gaps

• Geographic concentration: The PROBAST analysis cited by Hasanzadeh et al. found that 97.5% of neuroimaging psychiatric AI studies included only subjects from high-income regions. This pattern likely extends across clinical AI domains and means that bias findings from U.S. and European datasets may not generalize to other healthcare contexts, and that bias patterns specific to low- and middle-income settings are largely undocumented.
• Underrepresentation of external validation: Only 15.5% of neuroimaging studies included external validation. Without external validation on demographically distinct populations, the true generalizability of reported fairness improvements cannot be established.
• Absence of post-deployment surveillance standards: No consensus standards exist for how frequently deployed clinical AI systems should be re-audited for subgroup performance drift, or what triggers should initiate a formal review. Feedback loop bias can compound over time without detection.
• Lack of consensus mitigation benchmarks: The Cary et al. review of 109 articles found no consensus on a single best mitigation practice. This is partly a function of legitimate context-specificity, but it also reflects the absence of standardized reporting frameworks that would enable cross-study comparison of mitigation effectiveness.
• Underrepresentation of non-racial demographic axes: The published literature focuses heavily on racial and ethnic disparities. Disparities along axes of sex, gender identity, disability status, socioeconomic status, and rural/urban geography are less systematically studied.

Practical Recommendations for Health Systems

The following recommendations translate the frameworks and evidence reviewed in this article into institutional action. They are organized by organizational role.

1. Adopt a lifecycle audit framework matched to organizational role. Health systems evaluating AI adoption decisions should implement HEAAL procedures. AI device developers should embed the TPLC equity framework at each development phase. Dataset curators and researchers should apply STANDING Together documentation recommendations. Using a framework designed for a different governance level provides false assurance.
2. Require subgroup-disaggregated performance data as a procurement standard. Before deploying any AI system, require vendors to provide performance metrics (sensitivity, specificity, AUROC, calibration) stratified by race/ethnicity, sex, age group, and insurance type at minimum. Aggregate performance figures are insufficient for equity assessment.
3. Implement health equity impact assessment alongside algorithmic fairness testing. Use the HEAAL four-scenario model to assess whether the deployment context — not just the algorithm — will produce equitable outcomes. A technically fair algorithm deployed in an under-resourced setting can worsen equity (scenario B); this requires a deployment context assessment, not just a model performance review.
4. Establish ongoing post-deployment monitoring for subgroup performance drift. Define monitoring cadence and trigger thresholds for re-audit before deployment, not after a problem is identified. Feedback loop bias compounds over time; early detection requires prospective monitoring infrastructure.
5. Audit proxy variables in all deployed clinical AI systems. Review feature lists for variables correlated with race, socioeconomic status, or other protected characteristics. This includes cost-based variables, geographic variables, and utilization history — all of which encode historical disparities.
6. Build institutional AI governance capacity with explicit equity mandates. In the current regulatory environment, institutional governance is the primary accountability mechanism. This requires a governance committee with equity expertise, defined escalation pathways, and authority to modify or suspend deployed systems — not just a policy document.
7. Do not treat FDA clearance as an equity certification. FDA authorization establishes that a device meets safety and effectiveness standards for its intended use in the cleared population. It does not certify equitable performance across all demographic subgroups in all deployment contexts. Procurement decisions require independent equity evaluation.