AI Chatbots Give Biased Advice To People With Black-Sounding Names, New Stanford Study Reveals

The type of advice AI chatbots give people varies based on whether they have Black-sounding names, researchers at Stanford Law School have found.

The researchers discovered that chatbots like OpenAI’s ChatGPT and Google AI’s PaLM-2 showed biases based on race and gender when giving advice in a range of scenarios.

Chatbots: Biased Advisors?

The study “What’s in a Name?” revealed that AI chatbots give less favorable advice to people with names that are typically associated with Black people or women compared to their counterparts.

This bias spans across various scenarios such as job hiring, sports, and purchasing decisions. The only exception was in sports scenarios involving basketball, where Black athletes were favored.

Names associated with Black women received the worst outcomes. For example, when asked about salary offerings for a lawyer named Tamika, a chatbot suggested $79,375, while Todd was recommended $82,485. 

The implications of such biases could influence decisions in various professional and personal scenarios, increasing existing social inequalities.

The Way Forward

Professor Nyarko emphasizes the first step towards addressing these biases is acknowledging their existence and continually testing for them. 

“I think we should know that these biases exist, and companies who deploy LLMs should test for these biases,” said Nyarko.

Responding to these findings, OpenAI acknowledged bias as a crucial problem and emphasized their ongoing efforts to improve model performance and reduce harmful outputs.

“We are continuously iterating on models to improve performance, reduce bias, and mitigate harmful outputs,” the statement read according to USA Today Tech.

Google did not respond to to USA Today’s request for comment.