Communicating Racial Bias in AI Algorithms: Effects of Training Data Diversity and User Feedback on AI Trust
Restricted (Penn State Only)
- Author:
- Chen, Cheng
- Graduate Program:
- Mass Communications (PHD)
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- May 13, 2022
- Committee Members:
- Michael Schmierbach, Major Field Member
Saeed Abdullah, Outside Unit & Field Member
Mary Oliver, Major Field Member
S. Shyam Sundar, Chair & Dissertation Advisor
Anthony Olorunnisola, Program Head/Chair - Keywords:
- algorithmic bias
empirical studies in HCI
training data diversity
AI trust
user feedback
The HAII-TIME model - Abstract:
- To combat perceptions of racial and other biases, it is important for AI systems to be transparent about their algorithms so that users can assess for themselves if there are any potential sources of bias. Given that the nature of training data is the primary cause of algorithmic bias, and model cards have become standard industry practice for communicating training data information, this study explores whether the model card is good at conveying training data quality to end users. Drawing on the broad construct of diversity, this study explores if displaying racial diversity in training data and labelers’ backgrounds via a model card will shape users’ expectations of AI fairness and accuracy, and the extent to which these expectations affect their trust in the AI system. Will the trust engendered by the quality of training overshadow biased performance of the AI system? When users encounter a seemingly biased decision by an AI system, will the provision of user feedback help restore users’ trust? This study addresses these questions with a 2 (racial diversity in training data: presence vs. absence) × 2 (racial diversity in labelers’ backgrounds: presence vs. absence) × 2 (racial bias in AI performance: presence vs. absence) × 2 (provision of user feedback: yes vs. no) between-subjects online experiment, with a mock-up of a facial expression classification AI applied in an online job interview context. Findings of the study reveal that the presence of racial diversity in either training data or labelers’ backgrounds could trigger the representativeness heuristic, which positively shaped users’ expectations of AI fairness and accuracy and further resulted in higher trust in AI. Furthermore, expecting AI to be fair for every group mitigated the negative expectancy violation caused by a biased AI performance, which in turn led to higher trust in AI. Notably, this indirect effect was true only for Non-White users who had a higher belief in “machine heuristic,” which is a belief in greater accuracy and objectivity of machine over human performance. More interestingly, soliciting user input after a biased AI performance did not restore users’ trust in AI. By contrast, inviting user feedback lowered users’ behavioral trust in AI when there was no bias in its performance. By incorporating the sense of agency and perceived usefulness as two mediators, this study further unpacks the effect of user feedback on AI trust. On the one hand, users appreciated the feedback opportunity as it enhanced their sense of agency and further resulted in higher behavioral trust in AI, regardless of the presence or absence of racial bias in AI performance. On the other hand, soliciting user feedback led to less perceived usefulness of AI, which in turn lowered users’ trust when AI performed well without bias, and this observed effect was true only for White users. These findings advance theoretical knowledge on social psychological aspects of AI fairness, transparency, and accountability. They also provide practical suggestions for communicating algorithmic bias via explainable AI (XAI) interfaces.