Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents

Generative AI & LLMs
Published: arXiv: 2604.03173v1
Authors

Delip Rao Eric Wong Chris Callison-Burch

Abstract

Large language models and deep research agents supply citation URLs to support their claims, yet the reliability of these citations has not been systematically measured. We address six research questions about citation URL validity using 10 models and agents on DRBench (53,090 URLs) and 3 models on ExpertQA (168,021 URLs across 32 academic fields). We find that 3--13\% of citation URLs are hallucinated -- they have no record in the Wayback Machine and likely never existed -- while 5--18\% are non-resolving overall. Deep research agents generate substantially more citations per query than search-augmented LLMs but hallucinate URLs at higher rates. Domain effects are pronounced: non-resolving rates range from 5.4\% (Business) to 11.4\% (Theology), with per-model effects even larger. Decomposing failures reveals that some models fabricate every non-resolving URL, while others show substantial link-rot fractions indicating genuine retrieval. As a solution, we release urlhealth, an open-source tool for URL liveness checking and stale-vs-hallucinated classification using the Wayback Machine. In agentic self-correction experiments, models equipped with urlhealth reduce non-resolving citation URLs by $6\textrm{--}79\times$ to under 1\%, though effectiveness depends on the model's tool-use competence. The tool and all data are publicly available. Our characterization findings, failure taxonomy, and open-source tooling establish that citation URL validity is both measurable at scale and correctable in practice.

Paper Summary

Problem
Large language models and deep research agents are being used to generate long-form reports with inline citations, but these citations are often unreliable. In fact, some of these models fabricate entire citations, which can lead to real-world harm, such as legal sanctions, academic retractions, and medical misinformation. The authors of this paper aim to systematically measure the reliability of citation URLs generated by these models and agents.
Key Innovation
The authors release an open-source tool called urlhealth, which uses the Wayback Machine to check the liveness of URLs and classify them as either stale (once real but now non-resolving) or hallucinated (never existed). This tool can be used to detect and correct broken and fabricated citations. The authors also conduct a large-scale study to measure the prevalence of citation URL failures across commercial language models and deep research agents, and to investigate how these failures vary across different domains and models.
Practical Impact
The findings of this study have important implications for the use of large language models and deep research agents in various applications, such as academic research, legal proceedings, and medical decision-making. By using the urlhealth tool, these models and agents can reduce the number of non-resolving citation URLs, which can help to prevent the spread of misinformation and ensure the accuracy of citations. The study also highlights the need for more robust and reliable citation practices in these models and agents.
Analogy / Intuitive Explanation
Imagine you're writing a research paper and you need to cite a source to support your argument. You ask a language model to generate the citation for you, but it makes up a fake URL. This is like a game of telephone, where the information gets distorted and becomes unreliable. The authors of this paper are trying to identify how often this happens and how to prevent it from happening in the future. By using the Wayback Machine to check the liveness of URLs, they can distinguish between stale URLs (which were once real but are now broken) and hallucinated URLs (which were never real in the first place). This helps to ensure that citations are accurate and trustworthy.
Paper Information
Categories:
cs.CL
Published Date:

arXiv ID:

2604.03173v1

Quick Actions