Introduction
As ChatGPT and enterprise applications with Gen AI see rapid adoption, one of the
common downside or gotchas commonly expressed by the GenAI (Generative AI)
practitioners is to do with the concerns around the LLMs or Large Language Models
producing misleading results or what are commonly called as Hallucinations.
A simple example for hallucination is when GenAI responds back with reasonable
confidence, an answer that doesn’t align much with reality. With their ability to generate
diverse content in text, music and multi-media, the impact of the hallucinated responses
can be quite stark based on where the Gen AI results are applied.
This manifestation of hallucinations has garnered substantial interest among the GenAI
users due to its potential adverse implications. One good example is the fake citations
in legal cases.
Two aspects related to hallucinations are very important.
1) Understanding the underlying causes on what contributes to these hallucinations
and
2) How could we be safe and develop effective strategies to be aware, if not prevent
them 100%
What causes the LLMs to hallucinate?
While it is a challenge to attribute to the hallucinations to one or few definite reasons,
here are few reasons why it happens:
Sparsity of the data. What could be called as the primary reason, the lack of sufficient
data causes the models to respond back with incorrect answers. GenAI is only as good
as the dataset it is trained on and this limitation includes scope, quality, timeframe,
biases and inaccuracies. For example, GPT-4 was trained with data only till 2021 and
the model tended to generalize the answers from what it has learnt with that. Perhaps,
this scenario could be easier to understand in a human context, where generalizing with
half-baked knowledge is very common.
The way it learns. The base methodology used to train the models are ‘Unsupervised’
or datasets that are not labelled. The models tend to pick up random patterns from the
diverse text data set that was used to train them, unlike supervised models that are
carefully labelled and verified.
In this context, it is very important to know how GenAI models work, which are primarily
probabilistic techniques that just predicts the next token or tokens. It just doesn’t use
any rational thinking to produce the next token, it just predicts the next possible token or
word.
Missing feedback loop. LLMs don’t have a real-time feedback loop to correct from
mistakes or regenerate automatically. Also, the model architecture has a fixed-length
context or to a very finite set of tokens at any point in time.
What could be some of the effective strategies against
hallucinations?
While there is no easy way to guarantee that the LLMs will never hallucinate, you can adopt some effective techniques to reduce them to a major extent.
Domain specific knowledge base. Limit the content to a particular domain related to an industry or a knowledge space. Most of the enterprise implementations are this way and there is very little need to replicate or build something that is closer to a ChatGPT or BARD that can answer questions across any diverse topic on the planet. Keeping it domain-specific also helps us reduce the chances of hallucination by carefully refining the content.
Usage of RAG Models. This is a very common technique used in many enterprise implementations of GenAI. At purpleScape we do this for all the use cases, starting with knowledge base sourced from PDFs, websites, share point or wikis or even documents. You are basically create content vectors, chunking them and passing it on to a selected LLM to generate the response.
In addition, we also follow a weighted approach to help the model pick topics of most relevance in the response generation process.
Pair them with humans. Always. As a principle AI and more specifically GenAI are here to augment human capabilities, improve productivity and provide efficiency gains. In scenarios where the AI response is customer or business critical, have a human validate or enhance the response.
While there are several easy ways to mitigate and almost completely remove hallucinations if you are working in the Enterprise context, the most profound method could be this.
Unlike a much desired human trait around humility, the GenAI models are not built to say ‘I don’t know’. Sometimes you feel it was as simple as that. Instead they produce the most likely response based on the training data, even if there is a chance of being factually incorrect.
Bottomline, the opportunities with Gen AI are real. And, given the way Gen AI is making its presence felt in diverse fields, it makes it even more important for us to understand the possible downsides.
Knowing that the Gen AI models can hallucinate, trying to understand the reasons for hallucination and some reasonable ways to mitigate those are key to derive success. Knowing the limitations and having sufficient guard rails is paramount to improve trust and reliability of the Gen AI results.
