I am at https://xitter.cf/t.co/1LW9HK5FY0 Asst professor @UW_iSchool; @allen_ai @SemanticScholar #nlproc #healthinformatics #scinlp #bionlp #openaccess she/her
Seattle, WA
Joined September 2008
- Tweets365
- Following1.1K
- Followers1.6K
- Likes1.7K
Lucy Lu Wang retweeted
Agentic Abstention has been accepted to #NeurIPS2026.
We introduce a consolidated benchmark for evaluating whether agents know when to stop rather than continue acting when a task is underspecified or cannot be completed in the current environment.
It was amazing to co-lead this project with @bingbingwen1, with the guidance and support of @lucyluwang 🌟
Lucy Lu Wang retweeted
Human evaluation is often treated as the gold standard for long-form generation, but how much can we trust it if papers don’t report how it was done?
We study this question in our large-scale survey “Illusions of the Gold Standard” to be presented at @aclmeeting! 🧵1/n
Lucy Lu Wang retweeted
🔎 Deep research agents like Asta ScholarQA and OpenAI Deep Research are transforming how we perform literature review.
But how do we know if the way we evaluate them is actually meaningful?
Announcing our new paper: “Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks” 🧵
try out our new prototype system! you can ask questions about a paper and the system will answer with both text and figures from the paper. your data will go towards understanding how to better serve diverse visual needs!
Ever want to ask questions about a paper, including its figures & tables? 📊📈 Want smoother interactions w/papers on desktop & mobile?
Try Paper+Figure QA, a new tool from @allen_ai that answers with the original figures, tables, and excerpts from papers: paperfigureqa.allen.ai
ALT A screenshot of a user asking about examples of how a system works in a paper, and the system responding with details about the system along with a relevant figure and caption from the paper.
Lucy Lu Wang retweeted
We introduce MMMG: a Comprehensive and Reliable Evaluation Suite for Multitask Multimodal Generation
✅ Reliable: 94.3% agreement with human judgment
✅ Comprehensive: 4 modality combination × 49 tasks × 937 instructions
🔍Results and Takeaways:
> GPT-Image-1 from @OpenAI leads image generation at 78.3% accuracy—13.7% ahead of the next-best model. The top open-source model, BAGEL from #ByteDance , achieves 45.5% accuracy.
> Audio generation is still challenging: Top open-sourced models achieve only 48.7% accuracy in sound (Make-An-Audio 2 from #ByteDance) and 41.9% in music (MusicGen from @AIatMeta).
📜 Paper: arxiv.org/abs/2505.17613v1
🛠️ Code and Evaluation Suite: github.com/yaojh18/MMMG
🥇Leaderboard: yaojh18.github.io/mmmg-leade…
🧵1/N
Lucy Lu Wang retweeted
[Please RT]
I’m recruiting PhD students to work with me at @UW!
I’m looking for students passionate about developing new *social media algorithms*, both broadly and within the scope of this NSF grant: tinyurl.com/395yfphd
More info: faculty.washington.edu/msave…
@UW / @UW_iSchool
Lucy Lu Wang retweeted
📢 📅 After a long process of soliciting & vetting bids, I'm excited that we've finally been able to reveal the location for #EMNLP2025 -- it'll be at the International Expo Centre, Suzhou, China from 5-9 November 2025. Looking forward to seeing you there!
@emnlpmeeting #NLProc
Lucy Lu Wang retweeted
I'm recruiting a PhD student to join my group @uw_ischool in 2025-26. If you like the mountains and interdisciplinary research that blends data and culture, this could be a good fit!
PhD apps due Dec 2: ischool.uw.edu/programs/phd/…
More info about my group: melaniewalsh.org/mentorship
Lucy Lu Wang retweeted
📉 Open access papers previously had higher accessibility compliance than closed access papers, but since 2019, we observe a sharp decline in compliance among OA papers (from the same publishers), driving much of the overall drop in PDF accessibility.
ALT A line plot displays the mean normalized total compliance of scholarly PDFs across publishing models (open versus closed-access) from 2014 to 2023, with error bars showing standard deviation. The x-axis denotes the year, while the y-axis shows the proportion of PDFs. Open access papers show a significant decline in compliance after 2019, stabilizing or slightly increasing thereafter. Conversely, compliance for closed-access papers has been gradually improving since 2014.
In 2019 when we first did this analysis, PDF accessibility trends were mostly improving, slowly. These 2024 results surprised me, and reflect major shifts in #OA publishing since Plan S and exacerbated by Covid.
Has OA mostly been a win? Sure. But not evenly for everyone…
📢 Crisis alert in academic publishing!
Less than 3.2% of scholarly PDFs meet #accessibility standards for blind and low-vision readers, and compliance has dramatically declined since 2019, especially for #OpenAccess papers!
What’s going on?👇
Joint w/ @lucyluwang @uw_ischool
ALT Our research paper titled 'Uncovering the New Accessibility Crisis in Scholarly PDFs: Publishing Model and Platform Changes Contribute to Declining Scholarly Document Accessibility in the Last Decade.'
ALT A table showing the percent of papers in our dataset of 19997 PDFs that satisfy each criterion, along with Adobe-6 Compliance. The table has two columns. The left column lists various criteria: Default language, Tagged PDF, Tab order, Appropriate Nesting, Alt-text, Table headers, and Adobe-6 Compliance. The right column shows the corresponding percentage of papers that satisfy each criterion: 17.3% for Default language, 12.6% for Tagged PDF, 6.8% for Tab order, 15.9% for Appropriate Nesting, 8.5% for Alt-text, 13.4% for Table headers, and 3.2% for Adobe-6 Compliance.
Lucy Lu Wang retweeted
🚀Varying Shades of Wrong: When no correct answers exist, can alignment still unlock better outcome?
Introducing wrong-over-wrong alignment, where models learn to prefer "less-wrong" over "more-wrong". Surprisingly, aligning with wrong answers only can lead to correct solutions!
Lucy Lu Wang retweeted
Hi friends, colleagues, followers.
I am on the faculty job market! I am a PhD student @BerkeleyISchool + @berkeley_ai. I work on NLP, and I believe all language, whether AI- or human-generated, is ✨social and cultural data✨. My work includes: 🧵
today i left a bunch of comments for a collaborator on a grant like “what did you mean here?” and “you should expand upon this” only to realize later that i wrote those sections 😭
Lucy Lu Wang retweeted
🚨Curious how LLMs deal with uncertainty? In our new #EMNLP2024 Findings paper, we dive deep into their ability to abstain from answering when given insufficient or incorrect context in science questions 💡arxiv.org/pdf/2404.12452
Joint work w/ @billghowe @lucyluwang @uw_ischool
Lucy Lu Wang retweeted
@allen_ai @SemanticScholar is hiring #nlproc #hci #ml #ai researchers for the following positions with target start dates in 2025, apply by *Nov 1* for the 1st rolling deadline.
- Research intern
- Young investigator (Postdoc)
- Research scientist
Apply: job-boards.greenhouse.io/the…
Come be my colleague! We're hiring TWO tenure-track Assistant Professors at @UW_iSchool in AI, Data Science, and HCI 📊💻👩💻🌄
Link to apply: apply.interfolio.com/150031
Feel free to reach out with any questions!
Lucy Lu Wang retweeted
Excited to share that our paper on plain language summarization evaluation has been accepted to the #EMNLP2024 main conference! I’ll be in Miami and will have several PhD openings for Fall 2025. Feel free to reach out if you’d like to chat!
(1/n) Announcing 🍎APPLS, a testbed to evaluate metrics for plain language summarization!
arxiv.org/abs/2305.14341
Joint work with @tal_august, @GondyLeroyUA, Trevor Cohen @UW BIME, and @lucyluwang #NLProc #SDProc
Lucy Lu Wang retweeted
Sexual harassment is a horrible impediment to academic research, shutting out talented researchers and slowing scientific progress.
What can we do? I believe we're not helpless; we can improve our communities through practical actions.
Take a look: github.com/maria-antoniak/fi…
Lucy Lu Wang retweeted
1/ 🎉 Excited to share our #ACL2024 Findings paper on using LLMs to assist with literature review! 📝
"CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Support"
Please check out our virtual poster session today at 8:15 p.m. PT!
very excited about this new paper in which we survey work on LLM abstention! please check it out!
this was in part motivated by another recent work in which we analyze the limitations of LLMs in abstaining for context-dependent science QA questions: arxiv.org/abs/2404.12452
🤔💭To answer or not to answer? We survey research on when language models should abstain in our new paper, "The Art of Refusal." . Thread below! 🧵⬇️ arxiv.org/abs/2407.18418
Joint w/ @jihan_yao @shangbinfeng Chenjun Xu @tsvetshop @billghowe @lucyluwang
@uw_ischool @uwcse #nlproc
