Ä¢¹½ÊÓƵ

Healthcare systems are steadily adopting technologiesÌýdriven byÌýdata analytics. artificial intelligence (AI) and machine learning (ML) will increasingly play a role in patient care. This is, of course, cause for excitement—these technologies have the potential to advance healthcare in various subtle and revolutionary ways.Ìý

But there's also cause for concern. The racial, gender, and socioeconomic biases too oftenÌýpresentÌýin AI models have the potential to harm already marginalized populations. Left unaddressed, models based on biased data may reinforce or magnify the inequities in the healthcare system.

To create an ethical AI ecosystem, organizations must commit to analyzing AI for bias and fairness and ensuring transparency and accountability. As AI and healthcare continually evolve, there's no one-size-fits-all solution. Instead, the problem must be addressed through interdisciplinary collaboration and from multiple angles including federal regulation, data governance, and data science accountability.

We propose the following controls as an initial foundation and guideline for arriving at a comprehensive approach for eliminating bias in AI:Ìý

Interdisciplinary Collaboration with Patient Advocates

As University of Chicago Medicine’s (UCM) attempt to develop a length of stay algorithm suggests, developers should begin their bias mitigation efforts by seeking partnerships with two key stakeholders: diversity, equity, and inclusion professionals, and physicians with a deepÌýunderstanding ofÌýthe patient population(s) in question.Ìý

UCM’s story is far from the only one that supports this point. In another case, a for early identification of neurological diseases such as Alzheimer’s, Parkinson’s, and multiple sclerosis. The model was trained exclusively on speech data from English-speaking Canadians. When the tool was trialed with clinicians on a broader set of subjects, the model incorrectly classified patients from certain groups as likely to have early onset neurological disease. This included individuals who spoke a dialect other than Canadian English and those who were learning English as a second language.

Clinicians were immediately able to discern the differences between variable dialects and actual signs of neurological disease but this early version of the model was not. This type of collaboration between developers and clinicians should be established as a requirement prior to commercialization and clinical integration of applicable AI technologies.Ìý

Federal Regulation on Quality Assurance of Health Data Protocols

Federal agenciesÌýnow have a vital opportunityÌýtoÌýlead the way in demonstrating how toÌýincorporateÌýdiversity, equity, and inclusionÌýas coreÌýcomponents of the quality assurance protocols for modelÌýdevelopment, operations, and maintenance.ÌýIn June 2019,Ìýthe American Medical Association toÌýdo just that, asking the agency to developÌýbetter guardrailsÌýagainst bias in AI that may exacerbate health disparities, particularly when deploying AI tools on vulnerable populations.

In December 2019, Senators Cory Booker and Ron WydenÌý to the Centers for Medicare and Medicaid Services (CMS) and the Federal Trade Commission (FTC) requesting insights into what practices the agencies were implementing to combat errors related to AI bias.ÌýThose same senators teamed with Rep. Yvette Clarke to introduce the , which directs the FTC to enforce private sector implementation of bias assessments.ÌýWhile these actions are valuable steps forward, we mustÌýcontinue to strengthen partnerships between private industry and regulatorsÌýto ensure that the pace of technological integration does not outpace federal responses.

Data Governance to Ensure Diversity

Often, the data used to develop algorithms is not representative of diverse populations.ÌýIn genomics research, representation bias is an issue that has been acknowledgedÌýfor over a decade. In 2009, 96% of participants in Ìýwere of European descent. By 2016, representation of non-European ancestry had increased by 20 percent, but that boost largely came from Asian participants. African and Latin American ancestry representation had barely changed in the seven years since the initial analysis.

AI models, genomics research, precision medicine, and other technologies must be built upon representatively diverse datasets if we are to use them to diagnose and treat diverse patient populations. The fact that it takes so long to achieve even modest gains in the diversity of genomic data demonstrates the potential risks of pushing forward with broad AI integration before these challenges are resolved.Ìý

Make Data Science AccountableÌý

To address AI bias in a meaningful way, we must use quality control strategies to prevent unfair differential outcomes between diverse populations. To this end, theÌýAlan Turing Institute recommends several that can help find potential biases during the development cycle. Google researchers have contributed to the effort with published research around and addressing geographic diversity in unrepresentative datasets.

Tools like these are a good start but they do not specifically address the challenges of AI in the healthcare industry. To map a comprehensive path for ethical and equitable AI development, we’ll need to expand upon such frameworks. The result of such efforts will be an AI ecosystem that reduces the risk for disparate impacts on vulnerable populations and limits the risk of legal and financial implications related to poorly developed and implemented AI.Ìý

What Does AI-enabled Healthcare Look Like in the Future?

To date, the integration of AI technologies hasn't reliably narrowed the disparities in our healthcare system. Producing solutions that can fulfill the promise of AI in healthcare will require a sustained effort to develop comprehensive controls. Unbiased AI has the potential to improve care quality, foster patient engagement, predict optimal therapeutic options, and more. AI could enable us to make great leaps in reducing inequality and improving care across diverse and marginalized communities. If, instead, it merely places a veneer of objectivity onto care and diagnosis decisions, we will have failed.