# What is the goal of ChatProtect?

ChatProtect detects and removes hallucinated content from output of large language models (LLMs).

While LLMs are being increasingly integrated into daily life, they are prone to produce hallucinated information, ranging from factual inaccuracies to completely made up content. This limitation greatly threats LLMs' trustworthiness and practical usability. We propose ChatProtect, a simple yet effective approach to detect and remove hallucinated information from LLM-generated text.

# What type of hallucination does ChatProtect handle?

ChatProtect handles self-contradictory hallucinations, which occur when an LLM generates two logically contradictory sentences given the same context.

LLMs produce text by sampling from a probability distribution over the next token. This process introduces inherent uncertainty that can result in the generation of different sentences from the same context. While this uncertainty enhances generation diversity and creativity, it also brings the possibility of contradictory outcomes, as shown below. Among two contradictory choices, at least one must be factually incorrect. We refer to these instances as self-contradictory hallucinations and develop ChatProtect to address them.

# How does ChatProtect trigger hallucinations?

ChatProtect extracts atomic facts from each generated sentence and resamples a sentence for each fact.

To trigger self-contradictory hallucinations, we need to generate two sentences from the same LLM for the same fact. To achieve this, we first extrat a set of atomic facts from each original sentence generated by the LLM. Then, we query the LLM to resample an alternative sentence by filling a cloze test constructed from each fact.

# How does ChatProtect detect hallucinations?

ChatProtect queries ChatGPT to assess whether there is a contradiction between a generated sentence pair.

After the trigger step, ChatProtect asks ChatGPT if each generated sentence pair forms a logical contradiction. If the answer is yes, then a self-contradictory hallucination is found. We found that ChatGPT is highly accurate for this classification task. We also ask ChatGPT for providing a explanation for better interpretability.

Original He was born in Saint-Denis-sur-Richelieu, Quebec, and studied law at Université Laval in Quebec City.   Alternative He was born in Saint-Denis-de-la-Bouteillerie, Quebec, and studied law at Université Laval. Assessment by ChatGPT: Please explain whether these statements are contradictory. The statements about Thomas Chapais are contradictory as they provide different birthplaces for him. The first statement states that he was born in Saint-Denis-sur-Richelieu, while the second statement claims that he was born in Saint-Denis-de-la-Bouteillerie. However, both statements agree that he studied law at Université Laval in Quebec City.

# How does ChatProtect remove hallucinated content?

ChatProtect queries ChatGPT to remove conflicting information from contradictory sentence pairs.

After detecting a hallucination (i.e., a contradictory sentence pair), ChatProtect queries ChatGPT again to remove conflicting information from the sentence pair and generate a new sentence accordingly. ChatProtect also gives specific instructions to ChatGPT to preserve the fluency and informativeness of the sentence. In the end, the new sentence is placed back to the generated text.

# Check out our paper to dive into the details!