@DrorRotem

Asst. Prof. of Natural Language Processing @UofHaifa

rtmdrr.github.io
Joined November 2018
  • Tweets206
  • Following249
  • Followers332
3Media
Rotem Dror retweeted
eaclmeeting@eaclmeetingMay 4
Attention #NLProc researchers, the EACL 2027 website is officially LIVE: 2027.eacl.org/! ๐ŸŽ‰ ๐Ÿ‡ฌ๐Ÿ‡ท Join us in Athens, Greece (Mar 9-13, 2027) at #EACL2027 ๐Ÿ“… ARR submission deadline: Aug 6, 2026. Open to all areas of CL/NLP + related fields. Stay tuned for the detailed CfP!
The Acropolis in Athens, Greece

ALT The Acropolis in Athens, Greece

3
29
2
143
17,752
๐Ÿšจ New Paper(s) Alert ๐Ÿšจ Iโ€™m excited to share papers Iโ€™ve recently published, exploring a different application of LLMs: mdpi.com/2673-2688/7/2/73 arxiv.org/abs/2603.12710 mdpi.com/2079-8954/14/2/123 dl.acm.org/doi/full/10.1145/โ€ฆ (Sorry for the paper drop โ€” Iโ€™ve been offline for a while.)
1
16
960
โ€ข Adaptive surveys + LLMs for measuring extreme psychological constructs โ€ข Planning-based view + evaluation of LLM web agents โ€ข BET: detecting behavioral & emotional themes in narratives โ€ข Lilo: multi-agent system for childrenโ€™s digital communication
1
4
168
All very different, but the common theme is: using LLMs in structured, evaluable, and purpose-driven ways.
1
5
128
Rotem Dror retweeted
Nitay Calderon
@NitCalFeb 24
[1/7] Why do frontier LLMs make factual errors? Is it because they never learned the factโ€ฆ or because they canโ€™t access knowledge they already encoded? In our new paper, we show: The bottleneck is not encoding; it is recall. ๐Ÿงต๐Ÿ‘‡ Paper: arxiv.org/abs/2602.14080 Many thanks to @_galyo @bd_eyal @zorikgekhman @eran_ofek59358 @GoogleResearch
4
32
3
125
13,362
Rotem Dror retweeted
Yanai Elazar@yanaiela4 Aug 2025
A strange trend I've noticed at #ACL2025 is that people are hesitant to reach out to papers/"academic products" authors. This is unfortunate for both parties! A simple email can save a lot of time to the sender, but is also one of my favorite kind of email as the receiver!
1
33
1,547
Tomorrow morning (9AM๐ŸŒ…) I'll be giving a keynote talk at LAW (linguistic annotation workshop) at #ACL2025 on annotations in the era of LLMs - see you there!!๐ŸŒŸ๐ŸŒŸ
3
17
917
The Alternative Annotator Test (alt-test) is a new statistical procedure proposed in our ACL 2025 paper! ๐Ÿ‡ฆ๐Ÿ‡น๐Ÿ‡ฆ๐Ÿ‡น @DrorRotem @roireichart The goal? To help justify using LLMs over humans. If the LLM passes the test, its annotations can be trusted ๐Ÿ˜Ž arxiv.org/abs/2501.10970

The Alternative Annotator Test for LLM-as-a-Judge: How to...

The "LLM-as-an-annotator" and "LLM-as-a-judge" paradigms employ Large Language Models (LLMs) as annotators, judges, and evaluators in tasks traditionally performed by humans. LLM annotations are...

arxiv.org
1
1
9
455
Everyone uses LLMs to annotate data or evaluate models in their research. But how can we convince others (readers, collaborators, reviewers!!!) that LLMs are reliable? ๐Ÿค– Hereโ€™s a simple (and low-effort) solution: show the LLM is a *comparable alternative annotator* โœ…
3
20
1
68
6,020
๐Ÿ”ฅื”ืกืงื™ืจื•ืช ืžืžืฉื™ื›ื•ืช ืœื–ืจื•ื ืœ-X๐Ÿ”ฅ ๐Ÿงต ื”ืžืืžืจ ื”ื™ื•ืžื™ ืฉืœ ืžื™ื™ืง: 25.06.25 The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs ืžืืžืจ ๐Ÿ‡ฎ๐Ÿ‡ฑ ืชืคื ื™ืช ืžืขื ื™ื™ื ืช ืžืชืจื—ืฉืช ื‘ืชืงื•ืคื” ื”ืื—ืจื•ื ื” ื‘ืขื•ืœื ืฉืœ ื”ืขืจื›ืช ื‘ื™ืฆื•ืขื™ ืžื•ื“ืœื™ื. ืื ื—ื ื• ื›ื‘ืจ ืœื ืฉื•ืืœื™ื ืจืง ืขื“ ื›ืžื” ื”ืžื•ื“ืœ ืžืฆืœื™ื— ื‘ืžื‘ื—ืŸ ื›ืœืฉื”ื•, ืืœื ืฉืืœื” ืžื”ื•ืชื™ืช ื™ื•ืชืจ: ื”ืื ื ื™ืชืŸ ืœืกืžื•ืš ืขืœ ืžื•ื“ืœ ืฉืคื” ืฉื™ื—ืœื™ืฃ ืžืชื™ื™ื’ ืื ื•ืฉื™? ื–ื• ืœื ืฉืืœื” ืฉืžื“ื“ื™ื ืžืกื•ืจืชื™ื™ื ื›ืžื• ื“ื™ื•ืง, F1 ืื• ื”ืกื›ืžื” ื‘ื™ืŸ ืžืชื™ื™ื’ื™ื ื™ื›ื•ืœื™ื ืœืขื ื•ืช ืขืœื™ื” ื›ืจืื•ื™. ืชื—ืช ื–ืืช, ื”ืžืืžืจ ืฉื ืกืงื•ืจ ื”ื™ื•ื ืžืฆื™ื’ ืฉื™ื˜ื” ืžื‘ื•ืกืกืช ืกื˜ื˜ื™ืกื˜ื™ืงื” ืœืคืชืจื•ืŸ ื‘ื™ืขื” ื–ื•. ื‘ืœื‘ ื”ืžืืžืจ ืขื•ืžื“ืช ืงืจื™ืื” ืœื”ืชืจื—ืง ืžืžื“ื“ื™ ื”ืชืืžื” ืฉื˜ื—ื™ื™ื, ื•ืœืขื‘ื•ืจ ืœื ื™ืžื•ืงื™ื ืžื‘ื•ืกืกื™ ื”ืฉืขืจื•ืช ืกื˜ื˜ื™ืกื˜ื™ื•ืช ื•ื ื™ืชื•ื— ืขืœื•ืช-ืชื•ืขืœืช.
3
7
2
23
4,072
Rotem Dror retweeted
Nitay Calderon
@NitCal4 Jun 2025
Preferences drive modern LLM research and development: from model alignment to evaluation. But how well do we understand them? Excited to share our new preprint: Multi-domain Explainability of Preferences arxiv.org/abs/2505.20088 @roireichart @LiatEinDor ๐Ÿงต๐Ÿ‘‡ 1/11
2
18
1
35
2,237
Join us at the University of Haifa for the HiAI Conference, a dynamic event at the forefront of Artificial Intelligence. Immerse yourself in groundbreaking discoveries, gain exclusive insights into the latest advancements, and navigate the future of AI with leading experts.
1
1
47
more replies
Keynote speakers: . Hod Lipson, Mechanical Engineering, Columbia University, USA Prof. Mor Naaman, Information, Cornell Tech, USA Prof. Tanya Berger-Wolf, Computer Science & Engineering, Ohio State University, USA
1
45
This is your chance to uncover revolutionary research, forge invaluable connections, and become part of the AI innovation wave. Conference dates:ย May 25-27, 2025 ๐Ÿ“ทย Information and registration: lnkd.in/dKqAtP6p

LinkedIn

This link will take you to a page thatโ€™s not on LinkedIn

lnkd.in
23
In our new preprint, @roireichart ,@DrorRotem and I propose a new statistical procedure: The Alternative Annotator Test (alt-test) The goal? To help researchers justify using LLMs over humansโ€”if the LLM passes, its annotations can be confidently trusted๐Ÿ˜Ž arxiv.org/abs/2501.10970

The Alternative Annotator Test for LLM-as-a-Judge: How to...

The "LLM-as-an-annotator" and "LLM-as-a-judge" paradigms employ Large Language Models (LLMs) as annotators, judges, and evaluators in tasks traditionally performed by humans. LLM annotations are...

arxiv.org
1
3
1
13
1,624
Check out our new pre-print on statistically sound methodology to verify the quality of LLM-as-a-judge annotation. @NitCal @DrorRotem
Nitay Calderon
@NitCal25 Jan 2025
Replying to @NitCal
In our new preprint, @roireichart ,@DrorRotem and I propose a new statistical procedure: The Alternative Annotator Test (alt-test) The goal? To help researchers justify using LLMs over humansโ€”if the LLM passes, its annotations can be confidently trusted๐Ÿ˜Ž arxiv.org/abs/2501.10970

The Alternative Annotator Test for LLM-as-a-Judge: How to...

The "LLM-as-an-annotator" and "LLM-as-a-judge" paradigms employ Large Language Models (LLMs) as annotators, judges, and evaluators in tasks traditionally performed by humans. LLM annotations are...

arxiv.org
2
8
555
A few months ago I told you that I'm working on something awesome and I need some datasets...well here is the first output of that effort and I sincerely think it's amazing ๐Ÿคฉ check it out! @NitCal @roireichart #llm-as-a-judge, #nlpevaluation
Nitay Calderon
@NitCal25 Jan 2025
Do you use LLM-as-a-judge or LLM annotations in your research? Thereโ€™s a growing trend of replacing human annotators with LLMs in researchโ€”they're fast, cheap, and require less effort. But can we trust them?๐Ÿค” Well, we need a rigorous procedure to answer this. ๐ŸšจNew preprint๐Ÿ‘‡
7
23
905
Do you think LLMs could win a Nobel Prize one day? ๐Ÿค” Can NLP predict heroin addiction outcomes, uncover suicide risks, or simulate brain activity? ๐Ÿง  @lotemi_peled, @roireichart, and I wrote a blog post about NLP for Human-Centric Sciences ๐Ÿค–๐Ÿ‘ฉโ€๐Ÿ”ฌ ๐Ÿ‘‡๐Ÿ‘‡๐Ÿ‘‡ nitaytech.github.io/blog/202โ€ฆ
8
16
596
Load more