Large language models (LLMs) have gained great momentum due to their inherent advantages for organizations building gen AI applications. And while gen AI continues to impress as a shiny new area, it evolves daily with new models in the market.
While ChatGPT, LLama, Mistral and more get all the attention, we also need to consider how effective these LLMs perform for a specific task. And occasionally, LLMs produce incorrect responses that are presented confidently. This is one of the major challenges in using these LLMs in building applications — for various reasons, they hallucinate. The good news? There is a way to reduce these incidents.
Let’s dive deeper into LLM hallucinations, and understand how you can efficiently reduce this phenomenon with certain techniques.
What is hallucination in Large Language Models?
Hallucination is a phenomenon where an AI model produces text that, while often grammatically correct and plausible, is not rooted in the given input — or may even be factually incorrect.
LLMs have completely revolutionized the way we interact with machines by making them more wise and efficient to complete different custom tasks. But, it is the inherent property of any LLM to sometimes generate incorrect answers that are correct — including ones with biases or factually incorrect responses. This happens due to several factors including the data they are trained on, low quality input, poor prompts, etc.
You might be wondering how bad the impact of these hallucinations can be. The following screenshot shows Google’s Bard (now Gemini) generating hallucinated answers and responses. When one user asked about a query, Bard falsely stated something regarding the James Webb Space Telescope.
This hallucination from Bard cost Google a nearly $100B drop in valuation.
Why do hallucinations occur in LLMs?
As previously discussed, language models can hallucinate and produce outputs that include made-up or incorrect responses. These errors demonstrate the limitations of AI, highlighting the importance of human oversight and cross-checking against reliable sources for verification. But assigning a human to verify each response is not feasible or scalable. We will talk about hallucination mitigation strategies in a minute, but first let's see why hallucinations occur in LLMs:
- Insufficient training data. A model that hasn’t encountered diverse data during training might not establish accurate correlations between inputs and appropriate outputs, leading to hallucinated content.
- Inadequate supervision. Without proper guidance, a model might rely too heavily on its internal logic, leading to outputs that appear to hallucinate.
- Model overfitting. Overfitting to training data can cause a model to produce outputs that mirror the training set, but are misaligned with new or different inputs.
- Knowl
edge cutoff. LLMs like ChatGPT have a knowledge cutoff date and thus, are unaware of any information past that date. They may unknowingly respond to your question with out-of-date information that is no longer relevant.
Types of LLM hallucinations
We can divide these hallucination types into three main categories:
- Factual inaccuracies. This type of hallucination occurs when a language model presents information that is not true or correct, but is framed as factual. This includes dates, events, statistics or statements that are verifiably false. It can happen due to various reasons including misinterpretation of input data, low quality data and training methodologies, reliance on outdated or incorrect sources or the blending of information from different contexts that leads to an inaccurate output.
- Generated quotations or sources. This occurs when a language model fabricates quotes or citations. It might generate a statement and incorrectly attribute it to a real person, or create a fictitious source that does not exist at all. This is problematic because it leads to misinformation, falsely attributed statements and confusion.
- Logical inconsistencies. This includes generating responses that are internally inconsistent or logically flawed. After generating a response for a user query, the LLM can contradict itself in further responses. It occurs when a model makes a series of statements that, when taken together, are incoherent or conflicting — challenging the credibility of the model's outputs and confusing users who rely on its consistency.
In all of these cases, the language model is not intentionally misleading but is exhibiting its own limitations from various reasons that might include its training data, quality of data, knowledge cut-off date, poor fine-tuning, etc. And as a result, AI researchers are now coming up with various frameworks and tools to mitigate these hallucinations.
LLM hallucination mitigation strategies
Researchers are developing various approaches to make sure the response generated by LLMs is accurate. Some strategies need human intervention like reinforcement learning through human feedback (RLHF); others need fresh and custom data to train the model, known as fine-tuning. While Retrieval Augmented Generation (RAG) can help reduce hallucinations generated by LLMs, let's look at some more in-depth approaches we can use.
Along with RAG, we can divide these strategies into two parts: Pre-generation and post-generation strategies.
Pre-generation strategies prevent AI from generating incorrect or misleading information in the first place. These include:
Prompting
- Chain-of-Verification (CoVe). This involves self verification of responses by models. Multiple stages of verification makes it more efficient.
- Optimization by PROmpting (OPRO). This is where LLMs tend to optimize their own prompts, correcting the prompt inputs.
- System 2 Attention (S2A). This approach improves LLM reasoning. An instruction-tuned LLM is used here to identify, analyze and extract the most relevant parts of the input context, mitigating the influence of unnecessary information.
- EmotionPrompt. This technique uses emotional cues through prompts to LLMs so they can have more context and sentiment.
- Step-Back Prompting. This is an approach used to improve LLM reasoning and problem-solving skills.
- Rephrase and Respond (RaR). This technique allows LLMs to rephrase and expand the questions/prompts posed by humans, helping LLMs gain insightful context.
Retrieval Augmented Generation (RAG)
- Self-RAG. Self-RAG empowers LLMs to dynamically fetch relevant passages until the entire context is captured, all within the specified window.
- Active-RAG. Active-RAG improves passive RAG by fine-tuning the retriever based on feedback from the generator during multiple interactions.
- Multimodal RAG. Multimodal RAG gives a deeper understanding of context by augmenting text data with images and other media, enabling more accurate and relevant responses.
Post-generation strategies deal with verifying and correcting the AI's outputs after they have been generated. These include:
- Fact checking. Implementing human-in-the-loop (HITL) and knowledge bases to verify the accuracy of the information provided by the LLMs.
- Preference alignment. Using human feedback mechanisms (RLHF) to align the LLM’s outputs with human values and preferences.
These strategies are aimed at enhancing the reliability of AI systems, improving the quality of their outputs and ensuring they are aligned with human values and factual accuracy.
Mitigating hallucinations in your LLM apps with SingleStore
In this tutorial, we will take a publicly available news dataset, storing it in our SingleStore database and retrieving the information through hybrid search, a classic RAG approach. Through this method, we aim to mitigate instances of hallucination, ensuring precise and relevant results.
We will use SingleStore’s Notebook feature to complete this tutorial. If you haven’t already, activate your free SingleStore trial to use Notebooks and create a database to store our embeddings (vector) data.
Once you sign up, you land on the SingleStore dashboard. Click on ‘Develop’ to create a Notebook.
Create a new Notebook, naming it whatever you want.
Start with installing and importing the required libraries.
!pip3 install wget --quiet!pip3 install openai==1.3.3 --quiet!pip3 install sentence-transformers --quietimport jsonimport osimport pandas as pdimport wget
Download the model.
from sentence_transformers import SentenceTransformermodel =SentenceTransformer('flax-sentence-embeddings/all_datasets_v3_mpnet-base')
Import data from the csv file (AG News is a subdataset of AG's corpus of news articles).
cvs_file_path ='https://raw.githubusercontent.com/openai/openai-cookbook/main/examples/data/AG_news_samples.csv'file_path = 'AG_news_samples.csv'if not os.path.exists(file_path):wget.download(cvs_file_path, file_path)print('File downloaded successfully.')else:print('File already exists in the local file system.')df = pd.read_csv('AG_news_samples.csv')df
This is what you should see after running the preceding code to read the contents of the csv file.
You can see the data here:
data = df.to_dict(orient='records')data[0]
The next step is setting up the database to store our data.
shared_tier_check = %sql show variables like 'is_shared_tier'if not shared_tier_check or shared_tier_check[0][1] == 'OFF':%sql DROP DATABASE IF EXISTS news;%sql CREATE DATABASE news;%%sqlDROP TABLE IF EXISTS news_articles;CREATE TABLE IF NOT EXISTS news_articles (title TEXT,description TEXT,genre TEXT,embedding BLOB,FULLTEXT(title, description));
Get embeddings for every row based on the description column
descriptions = [row['description'] for row in data]all_embeddings = model.encode(descriptions)all_embeddings.shape
Merge embedding values into data rows:
for row, embedding in zip(data, all_embeddings):row['embedding'] = embedding
Here is an example of one row of the combined data:
data[0]
Now, let’s populate the database with our data.
%sql TRUNCATE TABLE news_articles;import sqlalchemy as safrom singlestoredb import create_engine# Use create_table from singlestoredb since it uses the notebook connectionURLconn = create_engine().connect()statement = sa.text('''INSERT INTO news_articles (title,description,genre,embedding)VALUES (:title,:description,:label,:embedding)''')conn.execute(statement, data)
Let’s run semantic search, and get scores for the search term ‘Aussie.’
search_query = 'Aussie'search_embedding = model.encode(search_query)query_statement = sa.text('''SELECTtitle,description,genre,DOT_PRODUCT(embedding, :embedding) AS scoreFROM news_articlesORDER BY score DESCLIMIT 10''')# Execute the SQL statement.results = pd.DataFrame(conn.execute(query_statement,dict(embedding=search_embedding)))print(results)
Now, let’s run a hybrid search to find articles about Aussie captures.
hyb_query = 'Articles about Aussie captures'hyb_embedding = model.encode(hyb_query)# Create the SQL statement.hyb_statement = sa.text('''SELECTtitle,description,genre,DOT_PRODUCT(embedding, :embedding) AS semantic_score,MATCH(title, description) AGAINST (:query) AS keyword_score,(semantic_score + keyword_score) / 2 AS combined_scoreFROM news_articlesORDER BY combined_score DESCLIMIT 10''')# Execute the SQL statement.hyb_results = pd.DataFrame(conn.execute(hyb_statement,dict(embedding=hyb_embedding, query=hyb_query)))hyb_results
By leveraging targeted mitigation tools and strategies like RAG, we can significantly diminish instances of hallucinations. Reducing hallucinations enhances the reliability and accuracy of LLM-powered applications — and also propels us closer to harnessing the full capabilities of generative AI. The journey toward perfecting these models is ongoing, but with constant effort and innovative solutions like SingleStore, we can maximize their benefit for a wide array of applications.
If you are interested in understanding more about the concept of LLM hallucinations, check out on-demand webinar.
