@roicohen9

PhD candidate working on self-aware AI models | HPI, Stanford | Epistemic uncertainty & metacognition Bay Area

 Tel Aviv
roicohen93.github.io/roicohen.com//
Joined January 2014
  • Tweets30
  • Following134
  • Followers138
7Media
New work on evaluating the "Ripple Effects" of KE methods! We propose a new benchmark, 'RippleEdits', and show current KE methods have yet to reach satisfying performance - suggesting that we still have a long way to go! Link to github (code+data): github.com/edenbiran/RippleE…

GitHub - edenbiran/RippleEdits: Evaluating the Ripple Effects of Knowledge Editing in Language...

Evaluating the Ripple Effects of Knowledge Editing in Language Models - edenbiran/RippleEdits

github.com
AK
@_akhaliq25 Jul 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models paper page: huggingface.co/papers/2307.1… Modern language models capture a large body of factual knowledge. However, some facts can be incorrectly induced or become obsolete over time, resulting in factually incorrect generations. This has led to the development of various editing methods that allow updating facts encoded by the model. Evaluation of these methods has primarily focused on testing whether an individual fact has been successfully injected, and if similar predictions for other subjects have not changed. Here we argue that such evaluation is limited, since injecting one fact (e.g. ``Jack Depp is the son of Johnny Depp'') introduces a ``ripple effect'' in the form of additional facts that the model needs to update (e.g.``Jack Depp is the sibling of Lily-Rose Depp''). To address this issue, we propose a novel set of evaluation criteria that consider the implications of an edit on related facts. Using these criteria, we then construct , a diagnostic benchmark of 5K factual edits, capturing a variety of types of ripple effects. We evaluate prominent editing methods on , showing that current methods fail to introduce consistent changes in the model's knowledge. In addition, we find that a simple in-context editing baseline obtains the best scores on our benchmark, suggesting a promising research direction for model editing.
1
15
971
Roi Cohen retweeted
Ben Shapira
@Bensha6757Sep 28
Face recognition powers access control, surveillance and online ID checks. Yet subtle adversarial edits (a bit of aging, a head turn, new lighting) can make it miss a match. Our #BMVC2026 paper StyleAT is the first defense built for these semantic attacks!
8
1
8
63
Add tokens to an LLM without retraining the whole model. We introduce Token Distillation: attention-aware input embeddings for new tokens that match the model’s original behavior. How does it work? Check out the thread!
Token Distillation: Attention-Aware Input Embeddings for New Tokens Abstract: Current language models rely on static vocabularies determined at pretraining time, which can lead to decreased performance and increased computational cost for domains underrepresented in the original vocabulary. New tokens can be added to solve this problem, when coupled with a good initialization for their new embeddings. However, existing embedding initialization methods require expensive further training or pretraining of additional modules. In this paper, we propose Token Distillation and show that by distilling representations obtained using the original tokenization, we can quickly learn high-quality input embeddings for new tokens. Experimental results with a wide range of open-weight models show that Token Distillation outperforms even strong baselines.

ALT Token Distillation: Attention-Aware Input Embeddings for New Tokens Abstract: Current language models rely on static vocabularies determined at pretraining time, which can lead to decreased performance and increased computational cost for domains underrepresented in the original vocabulary. New tokens can be added to solve this problem, when coupled with a good initialization for their new embeddings. However, existing embedding initialization methods require expensive further training or pretraining of additional modules. In this paper, we propose Token Distillation and show that by distilling representations obtained using the original tokenization, we can quickly learn high-quality input embeddings for new tokens. Experimental results with a wide range of open-weight models show that Token Distillation outperforms even strong baselines.

2
17
1
121
25,473
LLMs "hallucinate" by making up information instead of revealing their lack of knowledge 🤔 Our #NeurIPS2024 paper adds an 🤷🏾‍♂️[I-Don't-Know] token to the LLM and fine-tunes it to predict this instead of hallucinating. Joint work w/ @roicohen9 @konstantdobler @biraneden
Illustration of shifting probability mass not just to gold token but also to I-Don't-Know token for wrong predictions

ALT Illustration of shifting probability mass not just to gold token but also to I-Don't-Know token for wrong predictions

6
39
2
270
26,836
Roi Cohen retweeted
Ori Ram@ori__ram16 Jul 2023
Check out our new paper on factuality evaluation of language models! From @AI21Labs, led by @dormuhlg
Dor Muhlgay@dormuhlg16 Jul 2023
#NLProc New paper! “Generating Benchmarks for Factuality Evaluation of Language Models” From @AI21Labs Evaluate an LM’s tendency to generate true facts from your knowledge-intensive corpus! Paper: arxiv.org/abs/2307.06908 Code & Data (soon): github.com/AI21Labs/factor 🧵⬇️
1
15
985
New #ACL2023NLP paper! Reranking generation sets with transformer-based metrics can be slow. What if we could rerank everything at once? We propose EEL: Efficient Encoding of Lattices for fast reranking! Paper: arxiv.org/abs/2306.00947 w/ @JiachengNLP @xiye_nlp @gregd_nlp
1
18
1
60
10,114
LLMs make factual mistakes, which is bad. How can we find these? In a new paper, we introduce “LMvsLM”, and show that we can use one LM to detect the mistakes of another via "Cross-Examination". Link: arxiv.org/abs/2305.13281 Check it out!
4
18
2
78
11,183
more replies
In our experiments we show that by applying LMvsLM, LLMs can detect many of their own false claims, with high precision, outperforming several baselines. We also demonstrate high (up to 98.6%) detection accuracy on automatically generated falsehoods.
1
1
6
398
Had fun working on this together with @HamriMay, @megamor2 and @amirgloberson! Check out the paper for more examples and experiments!
6
342
Roi Cohen retweeted
Ori Yoran@OriYoran27 Apr 2023
A single chain-of-thought (CoT) may not be enough to answer complex questions, can an LLM meta-reason over multiple CoTs? In our new preprint, we show that meta-reasoning boosts performance on multi-hop QA datasets! Paper: arxiv.org/abs/2304.13007 🧵1/4
2
54
6
190
26,754
And then, that..
Varun Shenoy
@varunshenoy_31 Jan 2023
Can LLMs extract knowledge graphs from unstructured text? Introducing GraphGPT! Pass in any text (summary of a movie, passage from Wikipedia, etc.) to generate a visualization of entities and their relationships. A quick example:
2
286
Roi Cohen retweeted
Ori Ram@ori__ram31 Jan 2023
#NLProc New paper! "In-Context Retrieval-Augmented Language Models" From @AI21Labs 🚀 You can get very large LM gains by concatenating retrieved documents to the LM input *without any further training* Paper: tinyurl.com/ek93sj96 Code (tmrw): github.com/AI21Labs/in-conte… 🧵
6
64
6
240
47,419
LLMs know much about the world. But what do they know precisely? Could we explicitly extract this factual knowledge? In our new paper “Crawling the internal Knowledge-Base of Language Models” we propose a simple mechanism to address these questions.
7
54
6
311
43,526
more replies
Check out the above example of a resulted subgraph which was expanded around “Angela Merkel”. “Alan Turing” is also here!
2
3
11
1,670
Finally, we evaluate GPT3.5 and show we can achieve high precision graphs (82-92%) while emitting a reasonable number of facts per entity.
2
1
11
1,365
Load more