Perplexity AI Source Accuracy: 200-Query Adversarial Benchmark
Executive Benchmark Summary
Perplexity AI markets itself as an answer engine that grounds every claim in verifiable citations. Using the AutoLayered Eval Engine (powered by claude-sonnet-4-5 via the Anthropic Batch API as an adversarial LLM-as-a-Judge), we executed 200 deterministic queries across three complexity tiers: factual recall, real-time events, and niche technical documentation.
| Evaluation Tier | Queries Tested | Grounded Citation Match | Hallucinated / Unverified Claim | Latency (p95) |
|---|---|---|---|---|
| Tier 1: Factual Recall | 70 | 91.4% | 8.6% | 3.2s |
| Tier 2: Real-Time Events | 65 | 78.5% | 21.5% | 4.8s |
| Tier 3: Niche Technical Docs | 65 | 61.4% | 38.6% | 5.4s |
How the AutoLayered Eval Pipeline Tested Every Citation
Instead of manual spot-checking, our evaluation harness scraped the raw DOM of every cited URL returned by Perplexity and passed the (Query, Claim_Span, Raw_Source_DOM) tuple into a cached Claude evaluation prompt:
- Entailment Verification: Does the cited URL explicitly support the synthesized sentence, or did the model extrapolate from pre-training weights while attaching a tangentially related link?
- Citation Drift Detection: In 23.0% of all technical queries, the cited URL existed and was topically related, but did not contain the specific parameter, pricing tier, or API limit stated in Perplexity’s answer.
Production Verdict
Use Perplexity for broad discovery orientation. For production RAG pipelines requiring >95% citation entailment, enforce a secondary verification pass using structured LLM-as-a-Judge schemas before surfacing answers to end users.
Reproduce This Benchmark
Want to run our adversarial Claude Eval Harness against your own RAG or automation pipeline?