Skip to content

← Home
Artificial Intelligence

The face it couldn't see

Why a machine can score high and still fail you · 13 min

A graduate student at MIT had to hold up a white mask before her own project could find her face. Why face software works brilliantly on average and badly for some people, how chocolate makers helped change colour film, and three questions to ask any machine that judges your face.

Takeaways

  • A face system learns what a face is from the examples it is given, and is judged by the tests it is given: in the 2018 Gender Shades study, three commercial systems guessing gender erred for lighter-skinned men at most 0.8% of the time, and for darker-skinned women up to about a third of the time.
  • An accuracy score is always a score on somebody: one company reported over 97% accuracy on a test set more than 77% male and 83% white, and in NIST's 2019 tests, some algorithms developed in Asian countries showed no dramatic gap between Asian and white faces.
  • This week: for one machine that judges your face, ask who it was shown, who it was tested on, and what the human route is when it's wrong, and find that route before you need it.

Chapters

  1. The white mask0:17
  2. The film that couldn't see chocolate2:12
  3. The average that hides4:05
  4. Two selfies, two stakes6:38
  5. Hear it done8:06
  6. Build yours9:24
  7. The answer11:03

References

  1. Buolamwini, J. — How I'm fighting bias in algorithms (TEDxBeaconStreet talk)(2016)
  2. Buolamwini, J., & Gebru, T. — Gender Shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR 81:77-91(2018)
  3. Hardesty, L. (MIT News) — Study finds gender and skin-type bias in commercial artificial-intelligence systems(2018)
  4. Raji, I. D., & Buolamwini, J. — Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products (AIES '19)(2019)
  5. Grother, P., Ngan, M., & Hanaoka, K. (NIST) — Face Recognition Vendor Test (FRVT) Part 3: Demographic effects (NISTIR 8280)(2019)
  6. Roth, L. — Looking at Shirley, the ultimate norm: Colour balance, image technologies, and cognitive equity. Canadian Journal of Communication 34(1)(2009)
  7. Michigan Public — 'It didn't make sense at all': Wrongful facial recognition arrest in Detroit leads to landmark settlement(2024)