The Hidden Biases Behind ChatGPT
This article by Christian Ilube was first published on Medium.
What is ChatGPT?
Open AI is an artificial intelligence research company that aims to create AI to benefit all of humanity. In 2020, they announced GPT-3 (Generative Pre-trained Transformer 3), an AI capable of any reading/writing task e.g., question answering, summarising text, or continuing a story. It was created as a step on the way to Open AI’s grand mission of solving general intelligence, creating an AI able to learn any intellectual task that a human can do.
Despite not being general intelligence, GPT-3 still has many uses. A variety of apps that use it for different tasks have spun out of it as a result. The latest one is called ChatGPT, created by Open AI themselves. It is a chatbot designed for dialogue.
Open AI sells some of their products, for instance allowing users to pay to run their programs through APIs. However, their aim is to develop AI in a transparent way, making much of their code and research open source. This means they don’t generate much profit and are heavily reliant on investment, for example, the billion-dollar deal they made with Microsoft shortly after they released GPT-3. The applications they develop are needed to generate hype around their products and attract more investment.
Hidden Biases
An important consequence of this, is that Open AI do their best to hide any flaws in their systems, that may scare away investors. We saw an example of this recently when they made their DALL.E AI available to the public. DALL.E takes natural language prompts as input and generates pictures of what is written. It was trained on huge datasets of images available on the internet. The pictures in these datasets likely featured a much higher percentage of white males and initially this was reflected in the pictures that DALL.E generated.
In order to avoid accusations of bias and maintain hype, Open AI secretly altered the prompts inputted by users by adding extra text such as ‘black female’ at the end of prompts. These were visible to the AI but not the user. Researchers discovered this when they started using prompts such as “a person holding a sign that says” and received images of people holding signs saying, “Black Female”, implying that the words “black female” had secretly been added to the prompt.
This makes the AI appear less biased without actually fixing its inherent problems. Open AI essentially tried to sweep the problem of bias under the rug. This is dangerous because these technologies have the potential to have massive impacts on society, so transparency is needed for us to accurately assess how safe they are to use.
GPT-3 suffers from many of the same issues as DALL.E. GPT-3 was trained on a curated version of the common crawl dataset, a dataset made by algorithms that scrape the internet for text data. Experts believe that most of the text on the internet is generated by white males in western countries. It is likely that GPT-3’s training data overrepresents this demographic. Open AI’s own research discovered that the sentiment of the text it generated, varied when talking about different racial groups.
ChatGPT will not engage in any conversations it deems to be inappropriate. This is because Open AI are using their Moderation API, another application made using GPT-3, to block certain types of unsafe content. While this helps to stop mass production of harmful content, it leaves us in the dark about how an unfiltered GPT-3 would respond to certain questions. it is a good thing that Open AI are aware of these limitations and are actively working to fix them, but it may also mean that the public is unaware of the technology’s potential danger to society. Awareness of these dangers will be especially significant when policy makers consider GPT-3 based applications.
The moderated responses of ChatGPT, make language AI seem safe to start using today. However, we cannot be so confident in the text generated by Language AI, when they are applied to less structured tasks and given more freedom. There is still work to be done before we start using them everywhere.
How many languages can ChatGPT speak?
Most of the text data on the internet is in English. Therefore, GPT-3 performs much better in English than any other language. In the brief conversation I had with ChatGPT, I experimented by talking to it in various languages and although it is capable of conversing in an impressive range of languages, it acknowledges that its far more capable in English.
White males from western countries have had the largest role in creating this technology. They make up some of the key decision makers behind its creation but more importantly they have contributed the most data to its training set. Similarly, English speaking internet users will benefit the most from this technology. This is a relatively small and affluent section of society so it is fair to say that this technology will widen the inequality gap.
Future research
Language AI research has made incredibly fast progress in recent years. Applications using GPT-3 have demonstrated impressive capabilities so the hype that surrounds them is not without substance. However, we still have a long way to go before we can be sure that these technologies will benefit all of humanity. It is paramount that the world is aware of the limitations they have, regardless of the effect this may have on investment.
This article by Christian Ilube was first published on Medium.
