@unregularized

Research Scientist at Google DeepMind, working on Gemini. (prev. PhD at ETH Zürich & MPI-IS Tübingen.) No second-hand opinions. They are absolutely my own ;)

 New York
sidakpal.com
Joined October 2022
  • Tweets51
  • Following110
  • Followers486
16Media
📢I'll be presenting two posters, at #ICML2024 HiLD workshop (Straus 2) today (assuming no further ✈️ delays): - Closed form of the Hessian spectrum for some neural networks openreview.net/forum?id=gW30… - Landscaping Linear Mode Connectivity openreview.net/forum?id=OSNM…
1
1
12
3,564
quite good actually. coding’s so smooth! xitter.cf/GoogleDeepMind/status/…
Google DeepMind
@GoogleDeepMind18 Nov 2025
This is Gemini 3: our most intelligent model that helps you learn, build and plan anything. It comes with state-of-the-art reasoning capabilities, world-leading multimodal understanding, and enables new agentic coding experiences. 🧵
237
sure the numbers are great, but screw that! it’s your phd, try wild and original ideas even if you fail once or twice at least. xitter.cf/gabriberton/status/194…
Gabriele Berton
@gabriberton29 Jul 2025
A few numbers from my PhD: 8 first-author top-conference (CVPR/ICCV/ECCV) papers 100% acceptance rate per paper 80% acceptance rate per submission 1 invited long talk at CVPR tutorial 5 top-conf demos (acceptance rate 100% vs ~30% average) ~2k GitHub stars
9
963
when you go beyond linear mode connectivity, interesting things happen 😮👇 xitter.cf/Theus__A/status/194300…
Alexander Theus@TheusResearch9 Jul 2025
1/ 🚨 New paper alert! 🚨 We explore a key question in deep learning: Can independently trained Transformers be linearly connected in weight space — without a loss barrier? Yes — if you uncover their rich symmetries. 📄 arXiv: arxiv.org/abs/2506.22712
1
6
651
Belated life update: 🎓 PhD — done 🔬 Joined Google in NYC 🗽as a Research Scientist ♊️ Gemini: now more than just my star sign :)
24
10
543
29,181
🚀 TOMORROW afternoon at ICLR: Learn about the directionality of optimization trajectories in neural nets and how it inspires a potential way to make LLM pretraining more efficient ♻️ (Poster# 585, hall 2b)
Sidak Pal Singh@unregularized12 Jul 2024
Ever wondered how the optimization trajectories are like when training neural nets & LLMs🤔? Do they contain a lot of twists 💃 and turns, or does the direction largely remain the same🛣️? We explore this in our work for LLMs (upto 12B params) + ResNets on ImageNet. Key findings👇
1
6
2,149
Don't miss out our spotlight ✨paper at ICLR 🇸🇬 about the loss landscape of Transformers and their special heterogeneous structure, done together with great collaborators! xitter.cf/wormaniec/status/19145…
Weronika Ormaniec@wormaniec22 Apr 2025
Ever wondered how the loss landscape of Transformers differs from that of other architectures? Or which Transformer components make its loss landscape unique? With @unregularized & @f_dangel, we explore this via the Hessian in our #ICLR2025 spotlight paper! Key insights👇 1/8
2
16
1,401
Sidak Pal Singh retweeted
Alice Bizeul@AliceBizeul12 Feb 2025
✨New Preprint ✨ Ever thought that reconstructing masked pixels for image representation learning seems sub-optimal? In our new preprint, we show how masking principal components—rather than raw pixel patches— improves Masked Image Modelling (MIM). Find out more below 🧵
17
60
6
526
49,094
Sidak Pal Singh retweeted
Yann N. Dauphin@ynd10 Dec 2024
Don’t miss our poster shedding more light on sharpness regularization at NeurIPS tomorrow neurips.cc/virtual/2024/post…
4
7
2,713
Reinventing things has a bad rep in today's age. But is it really that bad? Maybe it's something to be even cultivated, like selectively? The second post in this series of blogs is now out. Let's have a deeper look at this overused trope! wovencircuits.substack.com/p…
2
315
I’m exploring a new form of writing—threads of human curiosity woven through the circuits of AI, crafting reflections that are, in the end, fully machine-generated, yet in a way profoundly human. wovencircuits.substack.com/p…
221
Come, let's scale up the building one floor, And, layer up the neural networks once more. Soon our buildings will touch the sky, And, our computers will bear AGI. A quaint little hut in the mountains is out of fashion, Satisfaction has no gradients for backpropagation. ~Fitoor
5
316
At this paper count, recalling all the paper names would already be a big feat :) xitter.cf/peter_richtarik/status…
Peter Richtarik
@peter_richtarik4 Oct 2024
Source: papercopilot.com/paper-list/…
2
361
“Hypotheses are nets: only he who casts will catch.” - Novalis
4
340
At last some attempts to change the status quo: authors with three or more papers are obligated to review for ICLR xitter.cf/PreetumNakkiran/status…
Preetum Nakkiran@PreetumNakkiran7 Aug 2024
Review requirements! (And 10pg limit!)
1
6
614
Poster 1: Sharpness/Flatness are much talked about: better minima, Sharpness aware minimization, Edge-of-Stability, and so on. But what really is sharpness? What exactly does it quantify, besides the surface-level definition? How are the eigenvalues and eigenvectors really like?
2
283
Come to our posters today at 3:30 pm (Straus 2) to know more! :)
162
Poster 2: Linear Mode Connectivity (LMC) is yet another popular feature of neural loss landscapes. But how does LMC arise in the first place? How should the landscape be structured to allow LMC? Are barriers present just at the end, or do they start much early?
1
233
There goes away my Austrian flight to #ICML2024 🥲
1
356
Sidak Pal Singh retweeted
Yuhui Ding@yuhui_ding22 Jul 2024
Tuesday 1:30pm-3pm, Hall C 4-9 #515. Drop by our poster if you are interested in SSMs for graphs👇! Code: github.com/skeletondyh/GRED
3
11
1,869
Load more