@ExplainableML

Institute for Explainable Machine Learning @HelmholtzMunich and Interpretable and Reliable Machine Learning group @TU_Muenchen

 Munich, Germany
eml-munich.de
Joined August 2021
  • Tweets400
  • Following108
  • Followers3.3K
157Media
Include

Only show posts containing:

Exclude

Hide posts containing:

Time range
-
Minimum likes
Explainable Machine Learning retweeted
Kirill Bykov@kirill_bykovOct 21
What does an LLM think of you? LLMs pick up the slightest implicit cues about users. Often, they refuse to say who they think the user is. In our new preprint, we show how to read and change these hidden beliefs. Turns out they even affect whether the model refuses New paper🧵
2
9
17
587
Explainable Machine Learning retweeted
Lukas Thede@lukas_thedeOct 2
📣 New preprint: Replay on Demand (RoD) Imagine you had 10 hours to refresh everything you learned in school. Would you divide them equally across every subject? Probably not. You’d spend your limited time on what you’ve forgotten most. 🧵 arxiv.org/abs/2609.40089
1
8
18
751
Explainable Machine Learning retweeted
Lukas Thede@lukas_thedeOct 1
📣 Introducing MedKIT, our new benchmark for knowledge integration in LLMs, accepted at #NeurIPS2026! 🧵 Why recalling updated facts ≠ integrated knowledge and why we need methods that can bridge this gap: 📄 arxiv.org/abs/2609.38543

MedKIT: Evaluating Knowledge Integration and Generalization in...

Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time, outdated knowledge can pose safety risks....

arxiv.org
2
10
25
1,143
Explainable Machine Learning retweeted
Luis
@lusxvrSep 25
FineVision was just accepted to NeurIPS! Big thanks to all my co-authors, excited to present it at the conference
Luis
@lusxvr4 Sep 2025
Today, we are releasing FineVision, a huge open-source dataset for training state-of-the-art Vision-Language Models: > 17.3M images > 24.3M samples > 88.9M turns > 9.5B answer tokens Here are my favourite findings:
2
6
29
1,767
On V*Bench and HR-Bench, VisLens matches or beats prior visual search baselines, running 8.5–9.9x faster than Thyme and up to 22.2x faster than training-free multi-pass search. Every crop traces back to decoded tokens, so you can see what the model looked at.
26
VisLens reads the model's own hidden states. A logit lens projects each image region through the LLM head to reveal what it already says; a light tuned-lens makes early layers readable. Match decoded tokens to the query, cluster, merge into crops, answer.
1
30
Finding a small object in a high-resolution image is still hard for MLLMs. Current fixes crop, zoom and re-query the model over and over: slow, and you cannot see why a region was picked. We wanted the answer in one pass, and we wanted to watch it happen.
1
10
[Paper]: arXiv: arxiv.org/abs/2608.30705

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs

Multimodal large language models (MLLMs) struggle with fine-grained Visual Search, the task of locating small or rare objects in high-resolution images. Existing remedies fall into two families:...

arxiv.org
1
5
3/ VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs Jingyi He, @Sanghwa30296060, @zeynepakata eXCV Workshop Poster: Tue 8 Sep, 13:15–14:45, Malmö Arena Palissad East
1
30
The adapter is where segmentation quality drops: alignment costs spatial detail. LLM layers claw it back. Correct tokens act as anchors, pulling wrong neighbors to the right label. Causal masking starves early tokens of anchors; bidirectional attention fixes that.
1
11
Three probes. Linear probing at every layer to track segmentation quality. Attention knockout to test whether tokens repair each other. Bidirectional attention among image tokens to measure what causal masking costs.
1
7
MLLMs are being pushed into pixel-level work like segmentation, but nobody knows where in the stack that spatial ability lives, or where it breaks. So we opened the pipeline: vision encoder, adapter, and every single LLM layer.
1
10
[Paper]: arXiv: arxiv.org/abs/2603.17228

From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs

Multimodal Large Language Models (MLLMs) are increasingly applied to pixel-level vision tasks, yet their intrinsic capacity for spatial understanding remains poorly understood. We investigate...

arxiv.org
1
8
2/ ECCV 2026: From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs Boyong Wu, @Sanghwa30296060, @zeynepakata Poster: Fri 11 Sep, 10:30–12:30, ExHall #40 eXCV workshop poster: Tue 8 Sep, 13:15–14:45, Malmö Arena Palissad East
1
40
One recipe, three tasks, zero training. V*Bench: 71.7 → 91.1 (+19.4). LongVideoBench: 52.7 → 60.5 (+7.8). Charades-STA mIoU: 10.4 → 46.8 (+36.4), beating TimeChat and VTimeLLM, both fine-tuned for temporal grounding. Uncertainty can steer perception in real time.
1
16
We propose a simple method called UG-Framework. We feed the model a crop, a frame, a time window, and read the output entropy. Low entropy means it found what it needs. Score every candidate, keep the most confident segments, then answer.
1
14
MLLMs still struggles with complex visual tasks like visual search, video temporal grounding and long video understanding. Instead of task-specific fine-tuning, we found the model's internal entropy as a useful signal that can boost all tasks in a training-free manner.
1
63
[Paper]: arxiv.org/abs/2510.00705 [Code]: github.com/ExplainableML/ug-…

Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs

Multimodal Large Language Models (MLLMs) often struggle with fine-grained perception, such as identifying small objects in high-resolution images or detecting key moments in long videos. Existing...

arxiv.org
1
2
78
1/ Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs @Sanghwa30296060, @RuiXiao99, Stephan Alaniz, @xyongqin, @zeynepakata Poster: Thu 10 Sep, 16:30–18:30, ExHall #52 MUCG workshop: Tue 8 Sep, oral at 11:45 Malmömässan A, poster 15:00 #394–423
1
1
2
196
We will have two poster presentations and one workshop poster at #ECCV2026 in Malmö🇸🇪! Please come and say hi if you are interested in topics related to multi-modal learning. Details in the thread👇
🤖 Made with AI
1
4
12
699
Load more