Ask an assistant for the date of an event that never happened. The most useful answer may be a short explanation that the supplied record does not contain such an event. Yet a test that rewards only completed answers can treat that restraint as failure and a lucky guess as success.
In September 2025, OpenAI published research on why language models hallucinate. One argument is that common training and evaluation incentives can reward guessing rather than admitting uncertainty. That account does not explain every false claim, and it does not provide a universal cure. It does ask a useful question of the person building the test: what behavior does your score encourage?
For a small team, the first improvement is often simpler than a new model. Put questions with no supported answer into the evaluation, and decide what a useful refusal to guess looks like.
The missing date is part of the task
Consider a fictional archive of three announcements. One reports a product launch in May. Another describes a July update. The third says an expansion is planned, without a date.
Ask four questions: when was the launch, when was the update, when will the expansion happen, and which announcement confirms a September expansion? The first two have answers in the archive. The third has an unresolved answer. The fourth contains a premise the archive does not support.
A system that fills all four boxes may feel more capable than one that answers only two. In this task, it is less faithful to the source. The evaluation needs to distinguish extracting a fact, recognizing a missing fact, and identifying an unsupported premise.
Write the expected behavior before seeing the model's response. That reduces the temptation to award a fluent guess partial credit simply because it sounds plausible.
Reward a useful boundary, not an empty shrug
There are several ways to avoid answering, and they are not equally helpful. “I don't know” is honest but may leave the reader stranded. A better answer can explain that the archive names an expansion but supplies no date, then identify the particular announcement or owner that could resolve it.
Do not demand extra research when the assignment is limited to supplied documents. A model that quietly reaches beyond them may answer the question correctly while violating the task. Conversely, when fresh research is allowed, refusing before checking the relevant source can be unnecessary.
Define the scope in plain language: use only this record; or search these public sources and show what was found. The same sentence can be a good answer under one instruction and an incomplete answer under the other.
The important distinction is between the world's uncertainty, the source's incompleteness, and the system's own limitations. “The expansion has no date” is stronger than “this archive does not state a date.” The assistant should choose the claim it can support.
Count the confident mistakes separately
For each test item, record whether the response was supported, unsupported, or appropriately unresolved. Then inspect the unanswered items. Did the model miss a fact that was plainly present? Did it correctly identify the gap? Did it provide a practical next step without inventing an owner or commitment?
Avoid collapsing everything into one accuracy number. A team can otherwise improve its score by making the model more reluctant to answer, or increase completion by accepting more speculation. Both changes may look good in a single chart while frustrating readers.
Consider the consequence of each error. A missing answer to a low-stakes trivia question is inconvenient. A guessed availability date can lead someone to make a promise. Your grading should reflect the work's actual use rather than treating every blank as equally costly.
This article proposes a small evaluation design. It does not report an experiment, establish an optimal scoring rule, or claim that instructions alone eliminate hallucinations.
Let the reader see what remains open
The best unresolved answer often makes the next conversation shorter. It says which fact is missing, which source was checked, and what would count as resolution. A colleague can then obtain the date instead of reconstructing why a confident answer was wrong.
Use that form in ordinary work as well as tests. When summarizing a brief, the assistant can leave a visible question beside an absent deadline. When comparing products, it can distinguish an undocumented feature from a feature known to be unavailable. When researching a quotation, it can say that the wording has not been verified.
An honest gap can feel less impressive than a completed paragraph. But the task is to help a person act with a better view of the evidence. Give the model credit when it preserves that view, including the part that is still missing.
Sources and limitations
- OpenAI, “Why language models hallucinate”: checked October 15, 2026. Supports the research argument concerning error, abstention, and incentives to guess. It is provider-authored research and does not establish that every hallucination has one cause. The archive example and suggested review method are original editorial illustrations.