February 22, 2024

“Are You Blacker than ChatGPT?” Quiz Highlights AI’s Cultural Blind Spots

Are you blacker than CHAT GPT

A recent development in the tech industry is the quiz game “Are You Blacker than ChatGPT?” created by creative ad agency McKinney. 

Are You Blacker Than ChatGPT?

This interactive game challenges players to test their knowledge of Black culture against ChatGPT, OpenAI’s language model.

The game’s inception traces back to a creative brainstorm at McKinney, led by copywriter Meghan Woods and a Black-led team. 

It took a year to develop, with the underlying goal of pointing out ChatGPT’s limitations in grasping the nuances of Black culture. 

The deficiency stems from many aspects of Black culture being orally transmitted or not widely documented online, leading to a gap in ChatGPT’s algorithmic learning.

Game players encounter questions like, “What does it mean when someone says, ‘Not too much on them, now’?” or “What is your response if you are invited to an event?” 

ChatGPT’s Blind Spots

These seemingly simple queries on Wood’s quiz reveal ChatGPT’s struggles with cultural specifics. 

“It’s interesting because it’s billed as this bot that knows everything, and it’s like, clearly, you don’t know everything, especially when it comes to things that aren’t white-specific,” Woods told Techcrunch.

“The blind spots can be pretty upsetting. It’s pretty dangerous.”

ChatGPT has been hit with allegations in the past, such as content moderators petitioning the country’s lawmakers to investigate the nature of their work.

Textio’s CEO also found that ChatGPT’s performance reviews could also turn out to be both sexist and racist.

“A lot of nuances can be addressed by including diverse perspectives at every level,” said Gerald Carter, founder of Destined AI, a company that helps detect and mitigate AI bias.

He confirmed that the McKinney quiz does a good job at gamifying and bringing more awareness to these AI gaps.

“For AI to reach its full potential, it needs to work for everyone, everywhere.”

Sara Keenan

Tech Reporter at POCIT. Following her master's degree in journalism, Sara cultivated a deep passion for writing and driving positive change for Black and Brown individuals across all areas of life. This passion expanded to include the experiences of Black and Brown people in tech thanks to her internship experience as an editorial assistant at a tech startup.