Are you working on RAG?
Consider participating in RAGTIME (RAG TREC Instrument for Multilingual Evaluation) track!
RAGTIME features:
· Multilingual Corpus!
· Rich, personalized report request (our long query)
· Required citation for every fact!
· Zero tolerance for hallucination!
Our guidelines are available. Register here. Follow us on Twitter: @trec_ragtime
Timeline:
· Dry run Topic Release: May 27, 2025
· Main Topic Release: June 30, 2025
· Dry Run Submission: July 8,2025
· Dry Run Results: July 28, 2025
· Main Submission: August 14, 2025
Our report generation task uses a submission format similar to other RAG-related TREC tracks: RAG, iKAT, DRAGUN, and BioGEN. We encourage cross-participation by quickly adapting your RAG pipelines. Please sign up and join our mailing list!
-- The RAGTIME Organizers
Dawn Lawrie
Sean MacAvaney
James Mayfield
Luca Soldaini
Eugene Yang
Andrew Yates