TESFA Lab logo
TESFA Trustworthy, Equitable, Safe and Fair AI for Medical Imaging

RESEARCH THEME

Fairness & Bias

One of our main focuses is fairness and bias in AI for medical image analysis.

All Research Fairness & Bias Quantification Prediction Segmentation Uncertainty

OUR WORK IN FAIR AI

Fairness in CMR segmentation

In many computer vision applications, artificial intelligence (AI) models have been known to exhibit bias in performance against protected groups that were underrepresented in the data used to train them. In this work we investigated whether AI semantic segmentation models could also exhibit such bias. We trained AI models for the task of segmenting the chambers of the heart from short axis cine cardiac MR images, and found significant racial bias in performance against minority races. This bias could potentially lead to higher misdiagnosis rates for heart failure, which is typically based on the patient's ejection fraction, as estimated from segmentations of the cardiac structures from cardiac MR.

Publications:

Subsequent work has attempted to uncover the root causes of this bias. Through a series of carefully-designed experiments we were able to discover that most of the distributional shift between White and Black subjects' CMR images (which can lead to bias) stems from areas outside the heart, such as subcutaneous fat. This knowledge opened up the possibility to design a new "understanding-informed" method for bias mitigation, which resulted in improved segmentation performance for both White and Black subjects with no accuracy-fairness trade-off.

Publications:

Sample segmentations from biased model for different races
(a) Sample segmentations from biased model for different races.
Relationship between training set imbalance and segmentation performance
(b) Relationship between training set imbalance and segmentation performance: white vs. black.
Figure: Investigation of racial bias in CMR segmentation.

Fairness in brain MR analysis

We investigated the potential for bias in the brain MR segmentation task. We systematically varied the level of protected group imbalance in the training set of a FastSurfer segmentation model and observed both sex and race bias in the resulting model performance. The bias was localised to specific regions of the brain and was stronger for race (white vs black) than sex.

Publications:

We have also investigated different sources of bias in AI based unsupervised anomaly detection (UAD) from brain MR. UAD involves training an AI model to learn the normal distribution of a training set. Subsequently, any sample outside of this distribution is considered to be abnormal, or an anomaly. However, previous work has assumed that there were no other sources of domain shift that could cause the abnormality. In this work, we found that both race and sex, as well as MR scanner vendor, could lead to such shifts, and UAD models trained using imbalanced data could exhibit biases in a similar way to supervised models.

Publications:

AI has been proposed for diagnosing Alzheimer's disease from brain MR. We have investigated if such models can learn demographic-related shortcuts and hence exhibit biased behaviour. We found that AI could identify both race and sex from brain MR images with a high level of accuracy and that the features exploited for this task could act as shortcuts when training for Alzheimer's disease diagnosis with race or sex imbalanced training sets. A method for quantitatively analysing the severity and nature of this shortcut learning was proposed.

Publications:

Quantitative analysis of shortcut learning in brain MR classification
Figure: Quantitative analysis of shortcut learning in brain MR classification.

Fairness in prostate MR segmentation

As well as CMR and brain MR segmentation, we have also investigated the potential for race bias in AI-based prostate segmentation from MR imaging. Here, we found a smaller, but still significant race bias effect — when training with data acquired from White subjects the model showed significant bias in performance against Black subjects. Publicly available datasets of prostate MR images do not provide information about the race of their subjects, but they are likely racially imbalanced, so this raises concerns about possible biases in AI segmentation models trained using these data.

Publications:

Fairness in breast cancer characterisation

There has recently been increasing interest in the use of AI to determine characteristics of tumours from radiological images. We have investigated the use of AI for determining breast tumour molecular subtype from dynamic contrast enhanced MR images (DCE-MR). We found that a random forest model trained with radiomics features was able to classify the race of the subject from such data (White vs Black) and furthermore that models for predicting tumour subtype could exhibit race bias when trained with imbalanced data.

Publications:

Fairness in dermatology image analysis

In dermatology, bias by skin tone is a well-known phenomenon: when training AI skin lesion diagnosis models using data from mostly light-skinned subjects, the resulting model will be biased against dark-skinned patients. In these studies, the Fitzpatrick Skin Tone (FST) scale is commonly used to represent skin colour. In this work we specifically investigated the impact that the granularity of FST values has on efforts to eliminate this bias. We found that dark-skinned patients have more coarse granularity in FST values than light-skinned patients and that this disparity negatively impacts the performance of AI models.

Publications:

FAIMI Initiative

Prof King and Dr Puyol-Antón are founding members and organisers of FAIMI, an independent academic initiative aimed at promoting research into Fairness of AI in Medical Imaging. FAIMI holds an annual free online symposium to showcase the best and latest research into Fair AI in medical imaging. They also run a workshop at the MICCAI conference. Check out the FAIMI web site for recordings of events, details of future events and useful resources on fairness research.

FAIMI — Fairness of AI in Medical Imaging

Interested in our work?

Explore our research or get in touch with the group.

Contact the group →