Seattle Area, WA
Joined June 2009
- Tweets249
- Following482
- Followers314
- Likes577
Markus Dreyer retweeted
Apple's Xcode now has direct integration with the Claude Agent SDK, giving developers the full functionality of Claude Code for building on Apple platforms, from iPhone to Mac to Apple Vision Pro.
Read more: anthropic.com/news/apple-xco…
📊 Our experiments with multiple LLMs show significant gaps in evidence-based reasoning. 🚧 NeoQA exposes limitations in multi-hop reasoning and shortcut reliance—crucial insights for building trustworthy AI. 🛡️
❓ What's unique? NeoQA includes answerable, unanswerable, and misleading evidence scenarios to truly challenge LLMs. 🔥 It reveals where models rely on shortcuts and struggle to detect mismatches between questions and evidence.
🔎 Why NeoQA? Traditional RAG datasets grow stale as LLMs internalize real-world events during pretraining. 🕰️ NeoQA solves this with fictional news timelines, preventing models from shortcut reasoning with parametric knowledge. 🔐
#EvidenceBasedAI
🌟Paper accepted! NeoQA: A benchmark for evidence-based question answering with generated news events. Unlike traditional datasets, NeoQA constructs fictional timelines, ensuring LLMs must reason exclusively over retrieved evidence. 🔍
📃 arxiv.org/abs/2505.05949
#ACL2025 #NLProc
ALT NeoQA: Evidence-based Question Answering with Generated News Events
Check out our #ACL2024 paper on multimodal summarization! w/ @vaidehi_patil_ @mohitban47 @leonardoribeiro
🚨In the age of information overload, summarizing key info from diverse modalities such as text and images is crucial. But how to do this effectively? Our #ACL2024 paper introduces a self-refinement approach for improving multimodal LLM summaries and REFINESumm, a new dataset 🧵
Don't miss the CCSum presentation at #NAACL2024. We present a large-scale dataset (1.3M) for abstractive news summarization. More factual and informative than summaries in CNN/DM, XSum & Multi-News! Joint work w/ Xiang Jiang
💻 github.com/amazon-science/cc…
📃 aclanthology.org/2024.naacl-…
Markus Dreyer retweeted
Happy new year! Just a reminder to everyone who works on language modeling that COLM is a new conference that I am co-organizing with several esteemed colleagues from academia and industry. Abstract deadline is on March 22 followed by the paper deadline on March 29.
colmweb.org/cfp.html
Markus Dreyer retweeted
We are excited to announce the #NeurIPS2023 paper awards:
blog.neurips.cc/2023/12/12/a…
Markus Dreyer retweeted
I'm thrilled to announce our company, @essential_ai . We believe that breakthroughs in AI will unlock the most profound tools for thought, advancing humanity's collective knowledge and capability.
essential.ai
Exciting work w/ @AdithyaPratapa and @smallcadenza! ✨We generate background summaries for events based on news timelines. arxiv.org/abs/2310.16197 Outstanding paper award at #EMNLP2023 summarization track.🤩Check out our dataset for this task: github.com/amazon-science/ba… #NLProc
Delighted to share that our @emnlpmeeting work on background summarization received an outstanding paper award in the summarization track 🏆
w/ Kevin Small and @markusdr
Paper: arxiv.org/abs/2310.16197
Github: github.com/amazon-science/ba…
Highlights below, (1/4) #EMNLP2023
#EMNLP2023 is coming up. Don't miss our paper on controllable text generation! arxiv.org/abs/2310.10623
We control text readability using RL/PPO with Gaussian rewards, look-ahead decoding, and prompt instructions.
#NLProc @mohitban47 @leonardoribeiro
🚨How to generate summaries w/ fine-grained readability-level control?
In our new #EMNLP2023 main paper, we develop 3 readability-control methods via instruction-prompting, RL, & lookahead decoding.
arxiv.org/abs/2310.10623
@mohitban47 @markusdr @AmazonScience @uncnlp
🧵👇
Markus Dreyer retweeted
Looking for something to read? The proceedings of #EACL2023 can be found on the @aclanthology !
aclanthology.org/events/eacl…
#NLProc #eacl #ACLAnthology
In our paper, we propose metrics that combine abstractiveness and factuality. The metrics adjust for different levels of abstractiveness when comparing the factuality of different text generation systems. arxiv.org/abs/2108.02859
The tradeoff between abstractiveness and factuality is similar to the tradeoff between precision and recall. High precision can be trivially achieved with low recall, just as high factuality can be achieved with low abstractiveness. #NLProc
Slides and presentation video at the #EACL2023 website: underline.io/events/383/sess…
We release two datasets with carefully selected human factuality judgements:
(1) ConstraintsFact has 10.2k factuality judgements,
(2) ModelsFact has 4.2k factuality judgements.
Check out the data and code on:
github.com/amazon-science/ab…