I like doing research (on adaptive models and ML for science). PhD student at @ELLISInst_Tue, @MPI_IS and @ETH_en. | Prev. @uni_tue (MSc), @ETF_Beograd (BSc)
Tübingen, Germany
Joined July 2024
- Tweets34
- Following785
- Followers155
- Likes62
Pinned Tweet
If a model that can adapt its compute to the task difficulty sounds cool, I hope you'll find our work interesting! // spoiler: it is looped ♾ arxiv.org/abs/2606.18206
code: github.com/nilskiKonjIzDunav…
checkpoints to play with huggingface.co/fixed-point-r…
Oh this is a beautiful one, enjoyed it!
Our group discovered that reasoning models produce fractals when asked to solve hard problems. We can use nonlinear dynamics to probe the thinking processes of recurrent depth models on Sudoku, mathematics, and even ARC-AGI (1/N)
arxiv.org/abs/2609.04963
I learned a lot from this paper, thanks to the authors! 😍Building on FPRM’s fixed-point approach, QRR introduces rotational symmetry to recognize convergence even while the hidden state keeps moving ---> it stops in fewer iterations!
Vera Milovanović retweeted
🚨 New Paper Alert, 🎤 Accepted for an oral presentation at the NeurIPS 2026 STODY workshop!
📣 Introducing: Normalizing Flow Maps (NFM): Combining few-step transport from Flow maps with tractable likelihoods from Normalizing Flows.
This is my first work with my first crop of new PhD students, @francescoshox and @tim_hitge, who genuinely knocked this project out of the park.
📄 Paper: arxiv.org/abs/2610.22711
📝 Blog: the-bose-lab.github.io/norma…
💻 Code: github.com/francescoshox/Nor…
🧵
1/8
Super cool usage of LLMs! ✨Scientists simply describe a discovery goal, and an LLM teaches molecular generators how to pursue it.
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
Depth Memory augments residual streams with compressed, queryable memory, outperforming PreLN, mHC, and AttnRes on perplexity at 400M and 1.5B parameters. Check out the paper! 😍
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔
Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output.
Linear Depth Memory (LDM) equips models with a matrix-valued memory.
Vera Milovanović retweeted
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔
Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output.
Linear Depth Memory (LDM) equips models with a matrix-valued memory.
Vera Milovanović retweeted
1/ New preprint: GRASS-MIL, a tool for discovering which tumor architectures matter.
From spatial single-cell data it maps the tissue neighborhoods tied to a clinical outcome, gives each an exact, signed contribution, and groups them into recurring niches. 🧵
Cool work!
Self-supervised transcriptomic FMs are pretrained on millions of gene-expression profiles, yet often fail to beat simple supervised baselines on clinical tasks.
Why?
Our new work suggests the pretraining objective may matter more than scale.
📄arxiv.org/abs/2610.08694
1/8
Vera Milovanović retweeted
Excited to be presenting this at #COLM2026 in SF:
Oct 8th Poster Session 5 (11am-1pm), Imperial Ballroom #8
Come chat about multi-agent interaction, societal impacts of AI, and more! :)
What do LLMs get up to when left to have free-form conversations with copies of themselves or other models?
And how can we even quantify what models are doing in free-form conversations?
We ran a number of free-form discussions between LLMs and observed the dynamics of the conversation in representation space.
There we can quantify a few quite interesting behaviors:
1) When we control for topics, the remaining representation space trajectories for models are quite predictable, so models 'end up' in a similar space across conversations when talking to a copy.
2) More interestingly, if talking to another model, we can observe that the dynamics circle the end points of both models, i.e. the models are being attracted by one another, taking on features and styles from the other.
3) This attraction is highly model-specific. For example Haiku is strongly attracting other models, while barely budging...
A few more details and plots (and an arxiv link below):
Nice contolled study!
🔥 Frontier labs are redesigning the residual stream to beat the curse of depth: Kimi K3 uses AttnRes, DeepSeek V4 uses mHC, ByteDance proposed HC, and there are more: LNS, KEEL, MoDA…
🙋 But each method was tested with its own training budget, model design and codebase, so the results can't be compared directly. Which one actually works best? More importantly, which one turns architectural depth into effective computational depth?
👇 We built DepthBench to find out.
📄 arxiv.org/abs/2609.32534
Vera Milovanović retweeted
Can neural networks learn to think by denoising?
Excited to share Thinking with Looped Flows!
We train recurrent reasoning with local denoising objectives. It is a new approach for reasoning that achieves SoTA on ARC-AGI-1 and 2 among looped models.
In collaboration with @chandavidlee, @FEijkelboom, @nmboffi, @ismaililkanc, @jw9730.
1/🧵
Vera Milovanović retweeted
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
Vera Milovanović retweeted
1/ If you have been reading about discrete diffusion, you might have asked yourself the following question:
What does a discrete diffusion model actually learn?
A denoiser? A concrete score? A bridge plug-in?
The literature treats them as interchangeable. They are not! 🧵
Vera Milovanović retweeted
A question on synthetic data generation: If we want a language model to solve k-step arithmetic problems (such as a+b*c-d=?), with operands from 1 to 100, which training distribution should we use?
A. Uniform distribution: Sample these k operands uniformly from 1 to 100
B. Power law: randomly shuffle 1-100 and impose an artificial power law. Sample these k operands according to this power law.
⚡Our ICML 2026 (spotlight) paper shows: Option B is better! Surprisingly, the same idea extends far beyond this simple example to many reasoning tasks that require implicit composition of multiple atomic skills, including multi-hop QAs and synthetic GSM problems.
📄Paper: arxiv.org/abs/2604.22951
📝Blog: zixuan-wang-dlt.github.io/po…
That's a crisp summary!
“Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers”
While Looped Transformers can spend more depth on harder problems, they still need a good way to know when to stop.
This paper makes the hidden state itself the stopping signal, basically it keeps looping until it converges to a fixed point.
With pre-norm, residual scaling, and damping, FPRM becomes stable at large depths, adapts compute to task difficulty, and beats similar 7M reasoning models on Sudoku, Maze, ARC-AGI-1, and state tracking.
Vera Milovanović retweeted
Why does Adam work better than SGD? Does it really? Many isolated explanations exist but finally we believe the picture is clearer: there is no single consistent source, except for the batch size trend, jointly shaped by data and architecture. Basically, all of it matters? Yes.
Judging optimizer gaps by looking only at language modeling with a fixed batch size is dangerous: one gets only 1/2 of the story. @orientino_ and @ruuustem_10 went beyond. Turns out that for every model, task, and data, there is always a setup where Adam > SGD. 🧵
Vera Milovanović retweeted
This TRM variant makes a transformer block a contractive map, so that looping becomes a fixed-point process. They leverage this by approximating gradients with Neumann series (Truncated BPTT). Very cool work!
🔗arxiv.org/abs/2606.18206
Vera Milovanović retweeted
Vera Milovanović retweeted
📊 Quick survey for researchers: How do you discover, use, and verify AI tools in your work?
5 min • Anonymous • All career stages welcome
Help us understand the gaps in AI tool accessibility for the research community.
🔗 forms.gle/24h27qy95cFEpZgE7
#AcademicTwitter #AItools
