The American Civil Liberties Union (ACLU) just spent US$12.33 to test a system that could quite literally cost people their lives.
On Thursday, the nonprofit organization, which focuses on preserving the rights of US citizens, published a blog post detailing its test of Rekognition, a facial identification tool developed and sold by Amazon.
Using Rekognition's default setting of 80 percent confidence (meaning the system was 80 percent certain it was correct when it signaled a match), the ACLU scanned a database of 25,000 publicly available mugshots looking to match them to photos of every sitting representative in Congress, in both the Senate and the House of Representatives.
Rekognition matched 28 Congresspeople to mugshots. It was wrong, but it found matches anyway.
Not only did Rekognition mistakenly believe that those 28 Congresspeople were the same people in the mugshots, the people it wrongfully matched were disproportionately people of colour; while people of colour make up just 20 percent of Congress, according to the ACLU, they accounted for 39 percent of the false matches.
"People of colour are already disproportionately harmed by police practices, and it's easy to see how Rekognition could exacerbate that," wrote the ACLU in its post.
"An identification - whether accurate or not - could cost people their freedom or even their lives."
An Amazon spokesperson told The Verge that poor calibration was likely the reason Rekognition falsely matched so many members of Congress.
"While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn't be appropriate for identifying individuals with a reasonable level of certainty," said the spokesperson.
They said Amazon recommends at least a 95 percent confidence threshold for any situation where a match might have significant consequences, such as when used by law enforcement agencies.
While anyone can use the system relatively cheaply (Rekognition, according to its web site, "charges you only for the images processed, minutes of video processed, and faces stored"), like the ACLU did, Amazon is actively marketing Rekognition to government and law enforcement agencies. Several are already using the service.
The ACLU isn't the only organization actively petitioning against this use; Amazon shareholders and employees have urged the company to stop providing Rekognition to government agencies. So have dozens of civil rights groups and tens of thousands members of the public.
So far, Amazon has not indicated that it plans to comply with these requests.
But perhaps, if members of Congress see just how flawed the system really is, they'll be compelled to take action, placing a halt on any law enforcement use of facial recognition software, as the ACLU requests in its blog post.
Science AF is ScienceAlert's new editorial section where we explore society's most complex problems using science, sanity and humor.
This article was originally published by Futurism. Read the original article.