Amazon Hiring Tool Downgraded Resumes with Womens, Reflecting Gender Bias
“1. Amazon Hiring Tool; Internal testing revealed that the system downgraded résumés containing the word “women’s,” reflecting gender bias present in the training corpus (Raghavan, 2018).”
Summary
Internal testing confirmed that Amazon’s AI hiring tool downgraded résumés containing the word “women’s,” such as “women’s chess club captain,” revealing gender bias in the system. This bias was documented in 2018 and prompted Amazon to discontinue the tool.
Sources 60 searched
- Insight - Amazon scraps secret AI recruiting tool that showed bias against women | Reuters
It penalized resumes that included the word "women's," as in "women's chess club captain." And it downgraded graduates of two all-women's colleges, according to people familiar with the matter.
- Amazon scrapped 'sexist AI' tool
By 2015, it was clear that the ... submitted to the firm mostly from males, Reuters claimed. The system started to penalise CVs which included the word "women"....
- The Problem With Amazon’s AI Recruiter | Robert H. Smith School of Business
Amazon.com scrapped its machine-learning resume-reader when it demonstrated a bias against female applicants. Here's what that tells us.
- Why Amazon’s Automated Hiring Tool Discriminated Against Women | American Civil Liberties Union
It similarly downgraded resumes that included the word “women’s” — as in “women’s rugby team.” And it privileged resumes with the kinds of verbs that men tend to use, like “executed” and “captured.”
- Amazon’s sexist hiring algorithm could still be better than a human - IMD business school for management and leadership courses
Although Amazon is at the forefront of AI technology, the company couldn’t find a way to make its algorithm gender-neutral. But the company’s failure reminds us that AI develops bias from a variety of sources.
- Amazon ditched AI recruitment software because it was biased against women | MIT Technology Review
Although Amazon made changes to make these terms neutral, the company lost confidence that the program was indeed gender neutral in all other areas. Why it matters: We can’t treat artificial intelligence as inherently unbiased. Training the systems on biased data means the algorithms also ...
- Amazon reportedly scraps internal AI recruiting tool that was biased against women | The Verge
“They literally wanted it to ... with the work told Reuters. When the company realized the software was not producing gender-neutral results it was tweaked to remove this bias....
- Demystifying bias in AI: Lessons from Amazon's sexist ...
It was to take care of reviewing ... in 2018, Amazon made headlines for the wrong reasons. Their AI recruiting tool showed a bias against women in technical roles....
- Incident 37: Amazon’s Experimental Hiring Tool Allegedly Displayed Gender Bias in Candidate Rankings
Amazon discovered a problem with using artificial intelligence to hire: their AI was biased against women. The Seattle-based company developed computer programs designed to filter through hundreds of resumes and surface the best candidates,…
This fact check is free and donation-based. $1 powers ~30 fact-checks.
