How to validate what AI gives you

This piece first appeared in German. The English version is a rewrite rather than a line-by-line translation, and the German original stays online in the archive: KI-Outputs Validieren.

A student researching tuition fees at foreign universities asks ChatGPT and gets figures from 2022. Two other models return consistent 2025 numbers. The discrepancy catches the error before it reaches the seminar paper, at a cost of about ninety seconds.

That is the whole method in one example. AI systems produce fluent, structured, convincing answers, and the problem is plausibility rather than randomness: large language models are optimized to sound coherent, not to be true. Every output is a structured hypothesis that has to be checked.

The framework below splits that check into six areas, each with a prompt you can copy directly, and a combined prompt at the end for work that carries consequences.

Classify before verifying

Different outputs fail in different ways, and applying the wrong method wastes time or misses the actual risk.

Run the classification prompt before anything else:

Before you answer my question: tell me what type of statement is involved (factual, statistical, legal, citation, strategic or code). In what way is each output type particularly prone to error?

That forces the model to name its own failure modes before committing to an answer, which makes overconfident responses easier to spot.

Compare models, not only sources

The most reliable method without any technical setup is a multi-model comparison. Put the same question to ChatGPT, Claude, Gemini and one independent source, separately.

Agreement raises confidence and confirms nothing, while any divergence is a mandatory signal to check a primary source. For time-sensitive or statistical questions, a search-grounded tool that cites live web sources earns its place here.

Answer the following question. Then rate your own confidence from 1 to 10 and name the most likely reason your answer could be wrong: [your question].

When the same question produces different confidence ratings or different most-likely-errors across models, you have found your verification target. Divergence in the self-assessment is worth as much as divergence in the answer.

Make the model attack its own answer

Do not ask whether the answer is right. Ask where it is weak. In practice these prompts produce sharper critical analysis than a follow-up confirmation question.

Red-team your previous answer. What are the three biggest weaknesses? Where is the claim most open to attack? What would a skeptic say?

Generate four questions that would verify the most important factual claims in your answer above. Then answer each one independently, as though you had not yet given the original answer.

The word independently is what carries that second prompt. Answered in isolation, verification questions are not biased by the model's earlier output. The technique comes out of the chain-of-verification work at Meta AI, where it reduced hallucinations across list questions, closed-book question answering and longform text.

Review your answer above. List every claim that is presented as fact but may be outdated, or stated with confidence but hard to verify, or an assumption you made without being asked. Mark each one explicitly.

Use at least one of those before trusting any output you cannot check independently.

Authenticate every citation directly

AI systems regularly invent plausible-looking references, and perfect formatting is no proof of existence.

For any academic or formal citation, look the digital object identifier up at doi.org, and treat a DOI that fails to resolve as an invented citation. Check that the article appears on the journal's own site and not only in the summary. Make sure the abstract actually supports the specific claim attributed to it. And check that author affiliations are consistent with the publication.

The warning signs are vague journal names, DOIs that resolve to a different paper, and publication years that do not fit the subject.

List every factual claim in your answer that depends on a specific source, study or statistic. For each one, tell me the exact source name, whether you are certain it exists, and whether the claim would change if the source turned out to be wrong or invented.

That surfaces the citations the model is least sure about first, instead of starting in the wrong place.

Reduce hallucination at the prompt

How you ask affects how often the answer is wrong, and these techniques apply before any answer arrives.

Adding "according to [source]" to a factual question anchors the model in a particular body of knowledge. Researchers found that such prompts make answers quote more directly from reliable text and often improve task performance as well. A line such as "if you do not know, say so explicitly" gives permission for uncertainty, which reduces confident invention. For important claims, the deep research modes now available in the major assistants are worth the wait, because they search and cite sources before they answer. And start with subjects you already know, because errors are only visible to somebody who knows the field.

I need accurate information on [topic]. Base your answer on established sources as far as possible. Where any part of your answer is uncertain, speculative or possibly outdated, mark that explicitly. Do not fill gaps with plausible-sounding information; say that you do not know.

Check currency separately from correctness

A statement can be factually correct for an earlier date and dangerously wrong today. Temporal validation is a different operation from fact-checking.

Check the publication date of every source the model cites or summarizes. For statistics and institutional data, go directly to the most recent release from the statistical office or the international body concerned. Treat any statistic without an explicit year as unverified.

For every statistic, figure and data point in your answer, tell me which year it comes from, whether it may have changed materially since, and where I can find the most current version.

One combined prompt for work that matters

For anything with real consequences, seminar papers, legal questions, theses, job applications, run the steps together after a first answer:

I want to validate the answer you just gave. Please do the following in order. First, classify each main claim as factual, statistical, interpretive or speculative. Second, list every claim that depends on a specific source, and tell me how certain you are that it exists and is represented correctly. Third, generate three verification questions for the claims you are least sure about, and answer each independently. Fourth, red-team your own answer: what are the two strongest objections to it? Fifth, mark anything that may be outdated and tell me where to find the current version. Sixth, give an overall reliability rating from 1 to 10 and explain what would lower it.

That takes about two minutes, and it catches nothing like everything, because no prompt can. What it does is expose the most likely failure points systematically, before they reach your work.

The quality of AI-supported work depends less on the model than on the care of the checking process around it. These prompts make that care portable.

This piece is part of a workshop series.

Sources

Grouped by the section they support.

Opening

• Kalai, Nachum, Vempala and Zhang (2025), Why Language Models Hallucinate, arXiv:2509.04664. Supports the point that training and evaluation reward plausible guessing over admitting uncertainty.
• The tuition-fee example is the author's own.

Classify before verifying

• No external source. The classification table is the author's own framework.

Compare models, not only sources

• No external source. The method is the author's own practice.

Make the model attack its own answer

• Dhuliawala, Komeili, Xu, Raileanu, Li, Celikyilmaz and Weston (2023), Chain-of-Verification Reduces Hallucination in Large Language Models, arXiv:2309.11495. Supports verification questions answered independently and the reduction of hallucinations across tasks.

Authenticate every citation directly

• DOI Foundation, Resolve a DOI, doi.org. Supports resolving a DOI to check a citation.
• The warning signs are the author's own checklist.

Reduce hallucination at the prompt

• Weller, Marone, Weir, Lawrie, Khashabi and Van Durme (2024), "According to . . .": Prompting Language Models Improves Quoting from Pre-Training Data, EACL 2024, 2288–2301, doi:10.18653/v1/2024.eacl-long.140. Supports the effect of according-to prompts on grounding and task performance.
• The other techniques are the author's own practice.

Check currency separately from correctness

• No external source. The method is the author's own.

One combined prompt for work that matters

• No external source. The combined prompt draws on the techniques above and is the author's own.