Large Language Models Generating False Information

The new AI chat bots (such as GPT-3 and GPT-4) sometimes create answers with clearly false information (which is commonly called "hallucinating"). Since they are trained from a corpus derived from the Internet where there is a huge amount of false information, I don't find that surprising in the least. Nonetheless, I thought I'd ask GPT-4 about Large Language Models and their production of responses containing false information:

Sometimes Large Language Models such as GPT-3 are accused of "hallucinating" where they write stuff that sounds correct but is simply not true. One example of this is when these models generate citations that simply don't exist. Why does this happen?

GPT-4's response:

Large language models like GPT-3 can sometimes "hallucinate" information, generating text that appears plausible but is actually incorrect or entirely fabricated. This occurs for a few reasons:

1. Training data: These models are trained on vast amounts of text data, learning to predict and generate text based on patterns and structures they observe during training. They don't possess a deep understanding of the content; instead, they rely on statistical associations. Consequently, they might generate plausible-sounding but inaccurate statements or citations.

2. Lack of fact-checking: Language models do not have an inherent mechanism to verify the accuracy of the information they generate. Their primary goal is to produce text that is coherent and contextually appropriate, which can sometimes result in incorrect or fabricated information.

3. Ambiguity and over-optimization: During training, language models may encounter ambiguous or conflicting information. They learn to optimize for generating text that appears coherent and reasonable, which can sometimes lead to them producing "safe" but false statements.

4. Bias in training data: If the training data contains incorrect or fabricated information, the model might learn to reproduce similar content. This is a reflection of the biases present in the data it was trained on.

5. Demand for specificity: When users prompt the model for specific information or details, the model may generate a response even if it doesn't have the exact knowledge. It's attempting to fulfill the user's request, which can result in the model creating nonexistent citations or false statements.

In summary, the phenomenon of "hallucination" in large language models like GPT-3 arises from the limitations of their training data, lack of fact-checking mechanisms, ambiguity in training data, biases, and the pressure to generate specific responses. These models are powerful tools for generating human-like text, but users should be aware of their limitations and always verify the information generated by them.

But don't the reasons for generating bad responses also apply to people? It seems to me the above reasons cause people to write (and believe!) things that just aren't so as well. So I asked ChatGPT to comment on that:

In your most recent response to me, you listed five reasons that Large Language Models sometimes "hallucinate" information. Humans and their writings and speech may also be subject to some or all of those five constraints that you listed which may also cause them to put forth false information. Please comment on how the five reasons you listed might also affect humans.

Here's the response:

Indeed, the five reasons listed for large language models "hallucinating" information can also affect humans in their writings and speech. Let's examine how each of these factors applies to humans:

1. Limited knowledge: Humans, like language models, possess limited knowledge based on their experiences, education, and exposure to information. When discussing a topic, they might draw on incomplete or inaccurate knowledge, leading to the dissemination of false information.

2. Lack of fact-checking: Humans may also share information without verifying its accuracy, whether due to time constraints, cognitive biases, or simply being unaware of the need to fact-check. This can result in the spread of false information, rumors, or misleading claims.

3. Ambiguity and over-optimization: Humans can encounter ambiguous or conflicting information and may form opinions based on incomplete understanding or personal biases. They might simplify complex issues, leading to oversimplified or incorrect conclusions. Additionally, people sometimes over-optimize their communication to persuade others or maintain social harmony, which may involve bending the truth.

4. Bias in sources: People are exposed to various sources of information, such as news outlets, social media, and personal networks, which may have their own biases. As a result, individuals may adopt and perpetuate these biases, consciously or unconsciously, in their speech and writings.

5. Demand for specificity: In conversations, humans may feel pressured to provide specific information or details even if they lack sufficient knowledge. To avoid appearing uninformed, they might speculate, guess, or fabricate information, potentially leading to the spread of false information.

In summary, the reasons that large language models like GPT-3 may "hallucinate" information can also be applicable to humans. It is essential to recognize these limitations, be aware of biases, and strive to verify information before sharing it with others.

So there are similarities to why both Large Language Models and people end up writing stuff that is simply and verifiably wrong. The internals are a bit different, for example AIs don't even have a concept of "appearing uninformed," but the end result is the same. Garbage in, garbage out.