Open-Source Interpretability for Generative Language Models 🔎 🐛
Europe
Joined November 2022
- Tweets57
- Following607
- Followers336
- Likes94
Inseq retweeted
Model Internals-based RAG Evaluation (MIRAGE) 🌴 is accepted to #EMNLP2024 Main!
➡️ To celebrate, here's our new MIRAGE demo combining @InseqLib and Transformers-specific LRP: huggingface.co/spaces/gsarti….
Reach out if you want to catch up in Miami! 🤗🏖️
[1/8] Struggling with verifying the trustworthiness of RAG outputs? Check our latest work where we utilize *model internals* as a powerful and faithful tool for attributing answers to retrieved docs! (w/ @gsarti_ @AriannaBisazza @raquel_dmg)
📄: arxiv.org/abs/2406.13663
#NLProc
Inseq retweeted
Very hyped for the new beautiful viz that just landed in the @InseqLib main branch! 🔥 This will empower users to explore attribution tensors more flexibly and intuitively.
h/t to @_ddjohnson for his awesome work on the treescope toolkit powering this release!
Thanks to the new treescope integration, @InseqLib now supports interactive visualizations for multidimensional attributions (show_granular), token highlights (show_tokens) and improved viz for attribute_context CLI! 🚀 Install main, will appear in v0.7
xitter.cf/_ddjohnson/status/1821…
ALT Granular visualization of an attention weight matrix for Gemma 2B on a summarization task. The slider on the bottom allows to visualize scores for individual heads without aggregation.
ALT Generated text with probability highlights (in green). For every generated token, source and target attribution scores can be visualized by opening the collapsed section (scores for token "colore" shown in figure)
ALT Output visualization of the attribute_context command for a chain-of-thought reasoning task with output-side context. The token "pairs" in the generated sentence "20 pairs of legs." is found to be context-sensitive and its prediction over the non-contextual alternative "horses" is motivated mainly by the presence of "pairs" in the output context.
Inseq retweeted
🎓 We're thrilled to host Gabriele Sarti (@gsarti_) in our PhD seminar series tomorrow, July 16th, from 12:00-13:00 in Oe67 BU 101! Join us for his talk on interpreting context usage in generative language models, featuring the Inseq toolkit and PECoRe framework. 🕛Don't miss it!
Inseq retweeted
Today, we had the first seminar of our #XAI course!
@gsarti_ presented the @InseqLib to interpret LMs and the PECORE framework to identify & attribute context dependence in LMs! 🚀🌟
Thank you, it was so interesting! 🤗
Great start to our series!
gsarti.com/talk/polito-inseq…
Inseq retweeted
[1/4] Introducing “A Primer on the Inner Workings of Transformer-based Language Models”, a comprehensive survey on interpretability methods and the findings into the functioning of language models they have led to.
ArXiv: arxiv.org/pdf/2405.00208
Inseq retweeted
🐑 PECoRe repository is now public (github.com/gsarti/pecore) and all model/datasets are available on @huggingface (huggingface.co/collections/g…)!
🐑Interested in using PECoRe on your models? Have a look at the @InseqLib implementation (`inseq attribute-context`)!
[1/8] Our new work (w/ @AriannaBisazza @gchrupala @MalvinaNissim) is finally out! 🎉
We introduce PECoRe, an interpretability framework using model internals to identify & attribute context dependence in language models.
📄Paper: arxiv.org/abs/2310.01188
#NLProc #neuralempty
Inseq retweeted
Excited to present my recent work on @InseqLib and the #ICLR2024 PECoRe interpretability framework for the @SheffieldNLP group this afternoon! Many thanks @casszzx for inviting me! 🤗
🐛 Inseq: github.com/inseq-team/inseq
🐑🐑 PECoRe: openreview.net/forum?id=XTHf…
Inseq retweeted
Wow!! This is great news🥳
Thanks @InseqLib! I’m excited to try it out!
Value Zeroing, a faithful approach for analyzing context mixing in Transformers, is now available on @InseqLib main branch for all @huggingface text generation models! 🔀
🔍Paper introducing VZ: aclanthology.org/2023.eacl-m…
🐛VZ in Inseq: tinyurl.com/inseq-vz
Inseq retweeted
Very proud of this release and all the amazing new contributors working towards democratizing interpretability research tools for the NLP community! 🚀
@InseqLib v0.5 is finally out! 🐛 New tutorial, distributed and 4-bit quantized models, easier & better contrastive attribution, and more! 🎉 Thanks to @daniel_sc4 @peppeatta and all other contributors!
Find out more in the release notes 👀 github.com/inseq-team/inseq/…
Inseq retweeted
Join us at the @BlackboxNLP poster session today at 11:00 (in-person at #EMNLP2023) for our PECoRe presentation and to grab some @InseqLib stickers! 🐛 #nlproc
Inseq retweeted
[1/8] Our new work (w/ @AriannaBisazza @gchrupala @MalvinaNissim) is finally out! 🎉
We introduce PECoRe, an interpretability framework using model internals to identify & attribute context dependence in language models.
📄Paper: arxiv.org/abs/2310.01188
#NLProc #neuralempty
Inseq retweeted
Excited to see the tools that leverage features of Petals (e.g. backprop through the distributed model) in creative ways. Thanks to @InseqLib devs, we now have an easy way to interpret predictions of LLMs without actually running them on your PC — give it a try!
Want to analyze LLMs with >50B params, but they're so huge that even quantization fails? 🤯
Fear not, attribution for petals 🌸 distributed LLMs has just landed in the @InseqLib main branch! 🚀 Tutorial: tinyurl.com/inseq-petals
cc @m_ryabinin @sasha_borzunov @BigscienceW
Inseq retweeted
Working on interpretability? Trying to gain a better insight into *how* a model came to a prediction?
Consider submitting to BlackboxNLP!
Submission deadline: Sep 1
Workshop date: Dec 7 @ EMNLP
We also accept work-in-progress in a non-archival track!
blackboxnlp.github.io/
ALT Dall-e generated image of "An oil painting of a black box model that is being explained"
Inseq retweeted
@InseqLib 🐛 is a new toolkit built on top of the @huggingface ecosystem to enable easy and reproducible interpretability analyses of generative LMs!
@nfelnlp and I will present Inseq today at #ACL2023NLP Demo Session 5 (16:15-17:30). Come say hi and get a sticker! 😄
Inseq retweeted
Heading to Toronto to attend ACL with the @GroNlp squad! 🍁
Will present:
- Inseq, interpretability tool for generative LMs (arxiv.org/abs/2302.13942)
- RAMP prompting for ACT (arxiv.org/abs/2305.17131)
First ACL, or want to talk about LM interpretability? DMs open! 🤗
Inseq retweeted
Glad to be part of the @eScienceCenter Fellows cohort of 2023! 🎉
This generous fellowship will contribute to the development and promotion of @InseqLib to make interpretability research on generative LMs more simple and reproducible. #nlproc
Inseq retweeted
While RAMP relies on attribute-marked examples for retrieval, I'm especially excited about the potential of feature attribution (w/ @InseqLib) as an unsupervised attribute extractor!
Find me at ACL to talk about attribute-controlled MT and interpretability for NLG! 🍁
Inseq retweeted
There is a new library to aid in LLM interpretability.
arxiv.org/abs/2302.13942
Inseq retweeted
The nicest part is how @InseqLib a super easy extraction of layer gradients obtained from a contrastive attribution target from a quantized LM using just a few lines of code 🤩 looking forward to new amazing NLP interpretability research powered by Inseq 🐛! 4/4
Inseq retweeted
I'm looking forward to exploring new ways that Inseq can be used for studying social biases in MT models!
New @huggingface Space by @oskarvanderwal using Inseq to highlight gender biases in MT models 🔍 huggingface.co/spaces/oskarv…