FactGPT — Automated Fact-Checking for Language Models
Published:
FactGPT is an automated fact-checker for the outputs of large language models — built to attack the trust-and-hallucination problem head-on. Given a statement, it retrieves the most relevant supporting evidence and returns a verdict on how well that statement holds up, so a reader can tell at a glance whether a model’s claim can be trusted.
How it works
FactGPT is a retrieval-augmented verification pipeline:
- Embed the claim the user wants to check.
- Retrieve the most semantically similar passages from a Wikipedia knowledge base (Cohere embeddings) stored in a Pinecone vector database.
- Score how well the retrieved evidence supports the claim and surface a truthfulness assessment.
In Ankit’s words, “FactGPT acts like a detective that quickly checks a huge library of facts (Wikipedia in this case) to tell you how true a statement is and whether it can be trusted.”
Recognition
FactGPT won the top prize at the NYC AI GPT Hackathon (organized by the AGI House team, New York City, April 2023) — a six-hour build against roughly 100 participants, with sponsors including OpenAI and Anthropic. The team brought together researchers from Carnegie Mellon University, NYU, Google DeepMind, and others; teammates included Misha Laskin (then at Google DeepMind).
This project sits squarely in a theme that runs through much of Ankit’s work: authenticity, verification, and trustworthy AI.