@blnewm

NLP research - PhD student @uwcse

blnewman.com
Joined June 2017
  • Tweets19
  • Following35
  • Followers97
10Media
🤖➡️📉 Post-training made LLMs better at chat and reasoning—but worse at distributional alignment, diversity, and sometimes even steering(!) We measure this with our new resource (Spectrum Suite) and introduce Spectrum Tuning (method) to bring them back into our models! 🌈 1/🧵
5
49
3
194
69,675
✨EMNLP Paper! ✨ Have you ever constructed a table to organize your lit review process? Can we use LMs to generate these automatically? We are excited to present ArxivDIGESTables 🍽️ a study of collecting, generating, and evaluating 🎓 scientific literature review tables 📃!
A screenshot of the first page of the paper discussed in the thread. Figure 1 contains a set of three cartoon papers with related text highlighted in three different colors. To its left, there's an arrow pointing to a cartoon table with a column corresponding to each color and a row corresponding to each table.

ALT A screenshot of the first page of the paper discussed in the thread. Figure 1 contains a set of three cartoon papers with related text highlighted in three different colors. To its left, there's an arrow pointing to a cartoon table with a column corresponding to each color and a row corresponding to each table.

1
16
3
77
17,790
more replies
We also find that providing more table context (captions, in-text references) to models leads to higher recall when generating columns but does not help when generating values.
Two plots of recall versus threshold for determining a schema match: one for GPT-3.5 Turbo and another for Mixtral 8x22B. There are five lines in each plot. Each line travels from the top left to bottom right of the plot with y-intercepts that are generally in increasing order by the following types of context: generated caption, baseline, gold caption, in-context examples, caption + in-text references.

ALT Two plots of recall versus threshold for determining a schema match: one for GPT-3.5 Turbo and another for Mixtral 8x22B. There are five lines in each plot. Each line travels from the top left to bottom right of the plot with y-intercepts that are generally in increasing order by the following types of context: generated caption, baseline, gold caption, in-context examples, caption + in-text references.

1
1
1
294
This is work with @yoonjoo_le2, @arnaik19, @Siangliulue, @rayrayfok, @imjuhokim, @dsweld, @josephcc, and @kylelostat conducted at @SemanticScholar @allen_ai @uwcse @kaist Code & Data: github.com/bnewm0609/arxivDI… Paper: aclanthology.org/2024.emnlp-…

GitHub - bnewm0609/arxivDIGESTables

Contribute to bnewm0609/arxivDIGESTables development by creating an account on GitHub.

github.com
2
4
463
I've started thinking about what I want to do after my postdoc at AI2 💫 If you know of an academic department or industry team where I might be a good fit, please reach out! 👩‍💻 NLP, cultural analytics, healthcare, narratives, online communities 🌎 maria-antoniak.github.io/
51
6
195
38,795
I'm on the faculty market! My goal is to build language systems that we understand deeply through discovery and by design, so we can precisely control them and treat their failures. Let's tackle this grand challenge of science and engineering together. nlp.stanford.edu/~johnhew/
6
72
3
404
97,037
If you missed our demo at #emnlp2023 this week, you can now play with PaperMage yourself and explore how you can easily access PDF documents using Python for your future projects: (1/5)
EMNLP 2026@emnlpmeeting10 Dec 2023
EMNLP 2023 Best Paper Demo PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scientific Documents (Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang et al.) aclanthology.org/2023.emnlp-… #EMNLP2023 #NLProc

PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scienti...

Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang, Russell Authur, Erin Bransom, Stefan Candra, Yoganand Chandrasekhar, Regan Huff, Bailey Kuehl, Amanpreet Singh, Chris Wilhelm, Angele Zamarron,...

aclanthology.org
1
9
1
28
6,489
Ben Newman retweeted
Kyle Lo
@kylelostat10 Dec 2023
so happy to have our work recognized at #emnlp2023 🥳 big thanks to @shannonzshen @blnewm @josephcc @soldni and collaborators at @SemanticScholar @allen_ai @UCBerkeley @MIT two of my favorite aspects of this work:
EMNLP 2026@emnlpmeeting10 Dec 2023
EMNLP 2023 Best Paper Demo PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scientific Documents (Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang et al.) aclanthology.org/2023.emnlp-… #EMNLP2023 #NLProc

PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scienti...

Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang, Russell Authur, Erin Bransom, Stefan Candra, Yoganand Chandrasekhar, Regan Huff, Bailey Kuehl, Amanpreet Singh, Chris Wilhelm, Angele Zamarron,...

aclanthology.org
7
15
1
104
20,324
EMNLP 2023 Best Paper Demo PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scientific Documents (Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang et al.) aclanthology.org/2023.emnlp-… #EMNLP2023 #NLProc

PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scienti...

Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang, Russell Authur, Erin Bransom, Stefan Candra, Yoganand Chandrasekhar, Regan Huff, Bailey Kuehl, Amanpreet Singh, Chris Wilhelm, Angele Zamarron,...

aclanthology.org
1
31
4
177
47,114
Hello! We will be presenting our poster on 📃 scientific decontextualization🎓 at #EMNLP2023 TODAY (12/9) at 11am in the east foyer! This is work with @soldni, @rayrayfok @armancohan, and @kylelostat, conducted at @SemanticScholar/@allen_ai
A screenshot of the first page of the paper.

ALT A screenshot of the first page of the paper.

1
5
2
36
8,088
more replies
We propose a QA-based decontextualization framework with three stages: 1️⃣ Question Generation: identify clarifying questions 2️⃣ Question Answering: find evidence from the paper and cited papers. 3️⃣ Rewriting: Incorporate the answers in the snippets, which are now decontextualized
Illustration of the three stages of our pipeline.

ALT Illustration of the three stages of our pipeline.

1
2
139
We find our framework helps annotators when collecting data. ✅ And it leads to LLM pipelines based that improve over end-to-end baselines. ✅ Findings: open models underperform closed ones, and that QG and QA are the most challenging. Link: arxiv.org/abs/2305.14772
Table showing relative performance of open and closed models at the rewriting component of our pipeline.

ALT Table showing relative performance of open and closed models at the rewriting component of our pipeline.

1
121
multimodal PDF processing is painful but doesn’t have to! come to our demo at #EMNLP2023 of Papermage, a library for fast manipulation of PDFs (Friday 9/12 @ 9am) we have used it for LLM data cleanup, paper QA, HCI prototypes github.com/allenai/papermage aclanthology.org/2023.emnlp-…
A picture of kyle in front of our poster

ALT A picture of kyle in front of our poster

A screenshot of the first page of papermage demo paper

ALT A screenshot of the first page of papermage demo paper

7
45
4
304
33,310
Load more