Roy Mayan (@Roym4498) profile picture

NLP and Computational Linguistics πŸ’¬

Joined March 2024
Photo by Roy Mayan (@Roym4498)
🧠 To reason over text and track entities, we find that language models use three types of 'pointers'! They were thought to rely only on a positional oneβ€”but when many entities appear, that system breaks down. Our new paper shows what these pointers are and how they interact πŸ‘‡
2
15
3
76
16,793
Can we precisely erase conceptual knowledge from LLM parameters? Most methods are shallow, coarse, or overreach, adversely affecting related or general knowledge. We introduceπŸͺππˆπ’𝐂𝐄𝐒 β€” a general framework for Precise In-parameter Concept EraSure. 🧡 1/
2
10
4
72
8,209
Roy Mayan retweeted
How can we interpret LLM features at scale? πŸ€” Current pipelines use activating inputs, which is costly and ignores how features causally affect model outputs! We propose efficient output-centric methods that better predict how steering a feature will affect model outputs. New preprint led by my student @GurYoav with dream team @Roym4498, Chen Agassy, and Atticus Geiger 🧡1/
6
27
1
114
7,425
These bracelets are awesome! Thank you Wiz 😎 @wiz_io #SecDevLove
2
72