Information Retrieval | Research Scientist at HLTCOE @JohnsHopkins | CS PhD from @Georgetown | from Taiwan
Washington, DC
Joined July 2018
- Tweets471
- Following426
- Followers650
- Likes1.2K
Eugene Yang retweeted
RAG Track submissions are all in.
And now it's time for RAGTIME!
2 more days for RAGTIME submissions to close.
Submit your system to compare on two tracks 🛤️
Eugene Yang retweeted
Given a request, AI systems generate long-form reports. Report evaluation relies on nuggets: essential & granular facts. Prior work on nugget generation treats them as short statements. Our #ICTIR2026 oral proposes DoGMaTiQ, an automated pipeline for Q&A nuggets.
1/6🧵
Alright! #RAGTIME submission is due in about a month!
Come join us and evaluate your deep research system
I would say this is one of the most expensive and involved annotation processes that we are developing/paying.
So your systems will be well evaluated!
🎶 It's #RAGTIME! 🎶
Topics for TREC RAGTIME 2026 are out — your deep research / report generation system now has a benchmark to prove itself on. Multilingual news in, cited English reports out.
Runs due Aug 12. One month. Let's go 👇
🔗 trec-ragtime.github.io/
📢 Excited to announce the program & keynote for RAG4Reports @ ACL 2026!
RAG for Report Generation — exploring grounded, multilingual, long-form RAG.
🗓 July 4, 2026
📍 San Diego, CA (Room Harbor A&B)
🔗 rag4reports.github.io/
🧵👇
We ran TWO shared tasks this year:
1️⃣ Automatic Report Evaluation
2️⃣ Multilingual Report Generation
…over a 4M-doc corpus spanning English, Chinese, Russian & Arabic. 🌍
Huge thanks to our authors and shared task participants
See the full program & paper list here 👇
rag4reports.github.io/
Come join us in San Diego! 🌊
LLM-as-a-judge is the default for evaluating open-ended generation — but it's riddled with silent failure modes: position bias, verbosity bias, sycophancy, run-to-run inconsistency.
Autorubric consolidates best practices into one open-source library. 🔧
Eugene Yang retweeted
News Update: a lot of smart hard working people put up with me for many years. Thanks :)
Happy to share that Ben Van Durme @ben_vandurme has been promoted to Full Professor, effective immediately!
Congratulations on this well-deserved recognition!
Eugene Yang retweeted
How do models choose which sources to cite?
Often, they choose English, even when other languages are more relevant to your query.
Our work on language preference in citations is a Spotlight at #ICML2026! 🎉
📑 arxiv.org/pdf/2509.13930
See you in Seoul! (k-bbq recs welcome 😗)
1/ 🌍 Do #LLMs really treat all languages equally when citing evidence? 📑
In our new work, we uncover linguistic nepotism: models often trade off citation quality for language preference 👇
Eugene Yang retweeted
Registration deadlines have been extended for several #NTCIR19 tasks:
AgenticInstruction: July 10, 2026
SciClaimEval: July 10, 2026
ToT: July 1, 2026
DAGRI: July 15, 2026
Check the task details and register here:
research.nii.ac.jp/ntcir/ntc…
We look forward to your participation!
RAGTIME is coming back for the 2nd edition!
Come join us to evaluate your search agents!
We once again feature multilingual and fully human evaluation
This is the ultimate way to know whether your systems are doing better
🚨 Every major AI lab is racing to build better "deep research" agents — systems that search, synthesize, and report across the web.
But how do we actually *benchmark* them?
Introducing 🧵 TREC RAGTIME — the shared task for rigorous RAG evaluation.
trec-ragtime.github.io/
Search endpoint is expensive to host, so please make good use of it!
xitter.cf/trec_ragtime/status/20…
🔍 We provide a free state-of-the-art dense retrieval endpoint.
Plug in your agent, skip the retrieval infra, and focus on what you actually care about: reasoning, synthesis, and report generation.
Search service is live now.
trec-ragtime.github.io/searc…
And try out the new RoutIR client!
We now support gRPC so it's actually faster!
github.com/hltcoe/routir/rel…
Eugene Yang retweeted
Want to join Google DeepMind as a research scientist working on AI for health and medicine?
Eugene Yang retweeted
Using Chinese AI models for coding be like:
NEW: House committees are probing Airbnb and Cursor parent Anysphere over their use of Chinese AI models @semafor
