This suggests that the idea of biased AI is neither new nor surprising, but troubling. Other previous research, such as an analysis in Nature Medicine, has shown that the implementation of AI can be influenced by demographics, including race. He said that “there is growing concern that such AI systems may reflect and amplify human biases, and reduce the quality of their performance in historically underserved populations such as female patients, black patients or patients of low socioeconomic status”. This bias often leads to embarrassing underdiagnosis by AI algorithms.
The Lancet study indicates that several factors could lead to bias, such as the use of data that does not represent an entire patient population. Another factor that can lead to AI bias is artificial intelligence learning traits that might be present in the population, like certain phenotypes, or characteristics, like bone density.
The research team wanted to see if AI models could determine race from chest X-rays alone. They used three large datasets that included a large and diverse population and what they found was staggering. AI could accurately predict a patient’s race based on X-ray alone, something even experts were unable to do.
The AI could also determine the course even when the images were “very degraded or cropped to one ninth of the original size, or when the resolution was changed to such an extent that the images were barely recognized as X-rays” .
Regarding the factors
To avoid phenotypic racial bias, researchers used other non-chest X-ray datasets, including mammograms, chest computed tomography (CT) scans, and cervical spine X-rays. The AI was still able to determine the race of the person.