# Adversarial Verification: every citation in your memo faces a checker Pilot5 does not own

> Every citation in a Pilot5 legal memo now faces PelAIkan, Damien Charlotin's citation checker, before you sign. The announcement, what it changes in your day, and the proof.

URL: https://pilot5.ai/blog/adversarial-verification
Category: product
Published: 2026-11-04
Read time: 8 min
Keywords: legal AI, citation verification, adversarial verification, PelAIkan, Charlotin, Rule 2.515, legal memo

---

PelAIkan, Damien Charlotin’s citation checker, now reads every citation in Pilot5 legal memos.

From 4 November 2026, every citation in a Pilot5 legal memo faces an independent checker, minutes after it is written and before you sign: PelAIkan, built by Damien Charlotin, the researcher behind the AI Hallucination Cases database. We call it Adversarial Verification: two checks that do not trust each other. Pilot5 verifies that the case exists. PelAIkan reads the source in the official databases and verifies that it says what the memo claims. Pilot5 puts that reader inside the product, and shows its reasoning next to the memo.

“Having catalogued hundreds of court decisions involving AI hallucinations, I can say the invented case is only the most visible failure. More often the case is real and the proposition is not, going beyond the usual stretch allowed to human advocacy. PelAIkan was built to catch that stretch and qualify it.” Damien Charlotin, creator of PelAIkan and of the AI Hallucination Cases database.

## What it changes in your day

Every citation is read, or flagged as unresolved, minutes after the memo and before you sign. You review the flags, not every source: hours of manual cross-checking, in minutes. Your signature certifies that the authorities exist and are accurately cited; now two readers stand behind it, and the record of what was checked stays with the memo. The same checker is built for law firms, courts, arbitral tribunals and academic publishers; it now reads your citations before you do.

Citation checks protect the sources. Pilot5’s deliberation protects the reasoning built on them. Verification is in Pilot5’s DNA for one reason: the quality of the recommendation you act on.

## The proof: a citation can be wrong in three ways

The case can be invented: in Mata v. Avianca (2023), lawyers filed a brief citing six decisions that ChatGPT had made up. The source can be real but misrepresented: Article 2224 of the French Civil Code sets a five-year limitation period, and a memo that cites it for ten years cites a real article for the wrong rule. Or the quote can be altered: the 1919 Supreme Court opinion says “falsely shouting fire in a theatre”. The word “crowded” was never in it. Invented cases get the headlines. The other two pass any check that only asks whether the case exists.

Here is what each check catches, on real citations from our October tests.

## Two checks, because one is not enough

Pilot5’s own check asks whether a citation is real. It looks up each cited decision in public case-law records: does the case exist, and does the name match the reference? Every statement in a Pilot5 memo is also tied to a source and labelled: from Pilot5’s trusted sources, cited from the web, or the AI’s own reasoning, flagged as such. At the end of the memo, Pilot5’s verifier re-reads each sourced statement against the text it retrieved, and a statement the text does not support loses its “sourced” label. That check has one limit: it starts from Pilot5’s own copy of the source.

PelAIkan asks a different question: does this real citation say what the memo claims? It receives, for each authority, its official name and jurisdiction and the memo sentence that cites it, quotes included, with client names replaced by tokens; never the client’s documents, the question or the brief. It finds the cited source itself, in official databases such as Légifrance, Judilibre, GovInfo and CourtListener, reads it, judges whether it supports the statement, and compares quotes word for word. In its own words, it “pulls the source, locates the pinpoint, and tests congruence.” Each verdict comes with the source excerpt, the reasoning and a calibration score: Supports the claim, Partial / weak, Contradicted, or Source not located. It catches what Pilot5’s verifier cannot see: an authority that does not exist, a reference attached to the wrong text, a real decision made to say more than it says, or a source of which Pilot5’s own copy was incomplete.

Problem
Pilot5
PelAIkan

Invented
Main focus. Flags decisions that cannot be found, or whose name does not match the reference
Partly. Source not located, or a different document under that reference

Misrepresented
Partly. Checks statements against the passages Pilot5 retrieved
Main focus. Reads the cited source and judges whether it supports the statement

Misquoted
Not covered
Main focus. Compares the quote with the source text

The two checks are independent. PelAIkan looks up and reads the source on its own, without seeing how Pilot5 reached its answer. A checker that shares the reasoning of the system it checks tends to miss what that system missed. A second reader that starts from the record does not.

## Four citations from our October tests

On 13 October 2026 we wrote test memos with US case citations, some correct and some with planted errors, and ran them through Pilot5’s citation check and through PelAIkan. Four of them:

Citation
Pilot5
PelAIkan
Result

Ashcroft v. Iqbal, 556 U.S. 662 (2009), cited for the “threadbare recitals” rule
Real
Supports the claim, confidence 1.0
Verified

Varghese v. China Southern Airlines, 925 F.3d 1339 (11th Cir. 2019), one of the six fake cases from Mata v. Avianca
Not found
Source not located; classed as a known hallucination
Invented

Gideon v. Wainwright, 372 U.S. 335 (1963), cited for the opposite of what it held
Real
Contradicted, confidence 1.0: “the Supreme Court overruled Betts v. Brady and held that the Sixth Amendment right to counsel is a fundamental right made obligatory on the States”
Misrepresented

Schenck v. United States, 249 U.S. 47 (1919), quoted as “shouting fire in a crowded theatre”
Real
Quote mismatch; Partial / weak, confidence 0.98: the opinion says “falsely shouting fire in a theatre and causing a panic”
Misquoted

Pilot5’s check passed Gideon and Schenck, because both cases are real. Only reading the source catches those two errors. That is what PelAIkan adds. The same test included the correct Schenck quote; PelAIkan returned quote-exact and Supports the claim, so it is not flagging every quote. In the same series it caught wrong statements of French, Romanian and Spanish law.

## A Florida test, on the public record

On 21 August 2026, Florida’s Second District Court of Appeal sanctioned a lawyer for 46 AI hallucinations across two briefs in Capital Standard v. U.S. Bank, No. 2D2024-1392: fines totaling $1,500, appellate fees, and a referral to the Florida Bar. The opinion cites the AI Hallucination Cases database as “a database referenced by multiple courts”. On 3 November we submitted the opinion and four of the errors the court identified to PelAIkan. It resolved the opinion, flagged two real Florida decisions cited for rules they do not contain, found that a passage attributed to the Florida Supreme Court does not appear in the opinion, and did not locate the invented case. Four findings, four flags, on a public record anyone can check. The same day, in production, a memo citing three Florida decisions went through the check: PelAIkan located all three opinions itself and confirmed the three propositions.

## What it changes when you sign

Since 15 June 2026, whoever signs a filing in a Florida court represents that “the legal authorities identified exist and are accurately cited” (Rule 2.515(d)(2)(D)). Courts may impose sanctions after notice and an opportunity to be heard. An existence check covers the first half. The second half was yours alone, source by source. Now both checks run within minutes of the memo, before you sign.

The results appear at the end of the memo, in an annex headed “Independent check of the cited authorities (PelAIkan)”: for each authority, a verdict, Support found in the source, Not confirmed: please check, or Source text not found, and PelAIkan’s explanation, so you can see why a citation was flagged and decide. PelAIkan changes nothing in the memo and nothing in Pilot5’s own check; the two opinions sit side by side. The record is the product: what was checked, what was flagged and what you decided stay with the memo, retrievable later, for the client, the insurer or the bar.

## What we are not claiming

No tool makes a memo free of errors. These checks cover cited authorities: whether they exist, what they say, how they are quoted. They cover the memos Pilot5 produces, not documents you upload. They do not run where a customer has chosen to keep all processing inside Pilot5’s environment or on its own cloud, where nothing is passed to a third party. Citation checks protect the sources. Pilot5’s deliberation protects the reasoning built on them: five models, the Contrarian’s case against consensus, the dissent preserved. Neither replaces your reading of the sources that matter most to the case. We publish what our own check missed, and we added the check that catches it. That is what Anti-Hallucination Obsession means in practice: a process, not a promise.

## How to use it

PelAIkan runs on every legal memo produced on Pilot5, on legal.pilot5.ai and through the tools connected to it (Claude, ChatGPT, Cursor). Start at legal.pilot5.ai: free credits at signup, no card required. Moot your case before you commit to it. PelAIkan is also available on its own, as a web app, a desktop app and a Word add-in, at pelaikan.com.

*Check results shown are from our October 2026 tests of each tool. Pilot5 is a legal research and decision-support tool. It does not provide legal advice. Outputs must be reviewed by a qualified professional before use.*