Lost in Transcription: Experimental Findings on Ethnic and Age Biases in AI Systems

Abstract

Artificial intelligence (AI) has revolutionized industries and improved our lives in various ways. However, AI systems’ potential to amplify existing biases in society has become a major concern. This thesis explores the concept of bias in AI and how it can lead to discrimination, focusing specifically on the performance of Automatic Speech Recognition (ASR) systems in relation to the ethnicity (accent) of participants. The study collected 187 recordings from proficient English speakers of 55 ethnic groups.These recordings were transcribed via ASR systems and evaluated by the word error rate (WER) metric. The ASR systems selected for the study were Gboard (Android) by Google, Apple keyboard (iOS), and Whisper by Open AI. The study results show that ethnicity significantly impacts the performance of ASR systems, with some ethnic groups experiencing substantially higher error rates than others. The study provides evidence that ASR systems may not be equally accurate for all users. To address ethnic bias in AI systems, it is essential to take a multi-faceted approach involving technical and societal solutions. The findings highlight the importance of addressing bias in AI systems to ensure fairness, transparency, and equity for all users, regardless of ethnicity.

Keywords: automatic speech recognition; bias in AI; digital ageism; digital equity; ethnic bias