Stop worrying about The Matrix and start worrying about what happens when a bunch of middle-aged white men get to decide what’s fed to our AI technology. Fei Fei Li, the inventor of the ImageNet image recognition technology that became one of the fundamental building blocks of AI, recently warned that while the dangers of AI are frequently represented in overblown dystopian sci-fi scenarios, the real danger at present lies in the data we’re feeding to the algorithms.
Speaking at the AI Hearing, With Great Power Comes Great Responsibility, Li warned, “When someone is using the term “doomsday scenario”... to me, I think if we wake up 20 years from now, whatever year it is, and we see the lack of diversity in our technology and leaders and practitioners... that would be my doomsday scenario.” Indeed, AI is a nascent field and yet we’re already starting to see biases. The Guardian warned we must act fast if we don’t want AI to become “the ultimate expression of masculinity.”
Examples of bias in AI are proliferating
A 2016 study by Bolukbasi et al. found that word embedding in Google News articles returned matches such as “man=computer programmer, woman=homemaker.” Safiya Umoja Noble, in her book, “Algorithms of Oppression: How Search Engines Reinforce Racism,” discusses how search engine results are a direct reflection of the gazes held by the most powerful people in our society. This has led to skewed representation of marginalized groups, such as hypersexualization of minority women or Google mistakenly identifying black people as gorillas.
More recently, the Gender Shades study found worrying discrepancies in facial recognition technology: while the technology had a 99% success rate in determining the gender of light-skinned men, the error rate soared as high as 34% when asked to determine the gender of dark-skinned men.
What are the consequences of bias in AI technology?
So, why should we be worried? Well, the problems seem small at first: facial recognition software rejecting a passport application because the applicant is Asian and the AI thinks his eyes are closed; a soap dispenser failing to detect a black hand. Joy Buolamwini, a dark-skinned researcher dedicated to fighting algorithmic bias, caused a stir when she explained how her webcam only recognized her face when she wore a white mask.
But the problems get bigger: think about the implications of racist facial recognition software in the justice system, or that awkward moment when sexist AI suggests higher-paying positions to male job searchers vs. female ones; or when your wristband heart rate monitor doesn’t work on you because your skin is too dark. These problems could have been avoided if researchers had been more proactive about including different demographics in the AI training process.
AI learns from the data it’s fed
We must act now if we want to avoid amplifying these biases. It’s not 100% the fault of the programmers. AI learns from data, and if it’s fed an overwhelming amount of data from white men, it will come to its own conclusions. We can assume nobody is purposely programming racist, sexist AI. AI is created by humans, and it mirrors what we put into it. But as a society, we need to take some urgent steps before the bias gets out of hand.
People are already taking steps to combat bias in AI. Along with several colleagues, Li co-founded AI4All, a nonprofit organization “dedicated to increasing diversity and inclusion in AI education, research, development and policy.” After her experiences with the racist facial recognition software, Buolamwini went on to found the Algorithmic Justice League. And artist Stephanie Dinkins was inspired by a dark-skinned robot she encountered called Bina48, created by Martine Rothblatt and modeled after her wife, Bina Rothblatt. Since meeting Bina48, Dinkins has been active in promoting diversity in AI development. Other organizations like Black in AI are doing their utter best to promote diversity in AI.
How late is too late to remove the bias from AI? We’ve already seen massive leaps and bounds in AI research and development over the past decade. Is it possible to retrain all the algorithms we’ve already taught?