Face recognition algorithms are biases towards black people and women

Joy Buolamwini, a MIT PhD candidate, presented her research at the inaugural FAT Conference in New-York. She had previously given a TED talk and been invited to the White House to present her work on algorithmic fairness.
In the paper she presented at the FAT conference Joy showed how face recognition algorithms can be biased towards certain segment of the population and in fact lead to discrimination.

Here are the 2 main contributions of this research

From a technical point of view there are 2 important contributions of Joy’s research :

• a dataset of 1270 individuals in 6 countries representing a more balanced view of how skin colors vary across countries
• a benchmark of existing face recognition algorithms (IBM Watson, Microsoft, Face++) based on this dataset

How the research was done 

The researchers assessed an official dataset of public subjects’ faces (the IJB-A dataset, the download of which is currently suspended) and discovered over-representation of lighter skin males and under-representation of darker skin subjects.
They subsequently decided to develop their own, more balanced, dataset which they used to the accuracy of 3 commercial face recognition tool : IBM, Microsoft and Face++

The results show clearly that commercial tools are biased (see table below for compared accuracy of the 3 algorithms).

Why this research is important

This research is important because it sheds light on real biases that are embedded in commercial algorithms, biases that are likely to touch a significant percentage of the population.
It’s also important because of the reaction of the manufacturers. Joy explained that she notified all 3 manufacturers (IBM, Face++ and Microsoft); yet only IBM reacted constructively and worked with her towards a better version of its algorithm.
The improved version of the IBM face recognition algorithm was dramatically better than the previous one, showing accuracies of 98%, 96.5%, 99.8%, and 100% for darker males, darker females, lighter males, lighter females respectively.

Key learning points

Here are some key leaning points from Joy’s research.

• calibration of many systems is biased towards lighter skin types. I learned for instance that digital cameras sensors are calibrated on lighter skin which can result in poorly illuminated darker-skin subjects
• face recognition algorithms are gender biased : they show very high (almost perfect) results with male faces but are less accurate with female faces
• commercial face recognition algorithmic tools were extremely biased and showed poor results when exposed to darker-skin subjects
• IBM was the only company to work with Joy on improving and debiasing their algorithms. Microsoft and Face++ politely referred to their terms and conditions.
  even official dataset can be biased : don’t trust them blindly

Image : Shutterstock

By Pierre-Nicolas Schwab Pierre-Nicolas est Docteur en Marketing et dirige l'agence d'études de marché IntoTheMinds. Ses domaines de prédilection sont le BigData l'e-commerce, le commerce de proximité, l'HoReCa et la logistique. Il est également chercheur en marketing à l'Université Libre de Bruxelles et sert de coach et formateur à plusieurs organisations et institutions publiques. Il peut être contacté par email, Linkedin ou par téléphone (+32 486 42 79 42)