Ask stupid question-Think impossible-Do step by step.
Joined January 2023
- Tweets1.1K
- Following908
- Followers15
- Likes3K
asfaan murthyulas retweeted
"second brain porn" goes on
asfaan murthyulas retweeted
This is f*king insane. OpenAI co-founder Andrej Karpathy:
"Prompting is going away.
Delete everything, keep Graph."
In 1 hour he shows how to build Graphs, and why it's the only thing left standing at the end
LLMs → Prompts → Agents → Graphs
skip the agent and you're the one finishing every job by hand
skip the graph and every finished job is forgotten, so the next run starts from zero again
Bookmark and watch it today
Then read the article below ↓
asfaan murthyulas retweeted
Looped Transformers should get better the more they loop. In practice, more loops often make them worse. We call this latent overthinking.
We found why, and how to fix it. Introducing InfiLoop 🧵
📄 arxiv.org/abs/2610.11570
asfaan murthyulas retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
asfaan murthyulas retweeted
“Make mistakes of ambition and not mistakes of sloth.”
— Niccolo Machiavelli
asfaan murthyulas retweeted
“One of the worst things that slows down progress is the concept of 'emergence.'”
This week on 632nm, @drmichaellevin (@TuftsUniversity) argues that the concept of emergence can hold science back by treating unexpected biological behavior as an explanation rather than something to investigate.
“Emergence just means surprise... You didn’t see it coming.”
If we call something “emergent,” we still need to explain why this behavior emerged instead of another. The goal should be to understand the underlying patterns well enough to predict and control them.
We discuss how this perspective could reshape biology, our understanding of intelligence, and AI.
Full episode on X and YouTube (see comment).
Timestamps:
00:00 - Intro
01:16 - Platonic Space
03:55 - Biology in the Absence of Evolution
21:11 - Extracting Knowledge from Platonic Space
37:04 - Development and Loss Functions
50:09 - Investigating Layers of Control
55:35 - Is Computer Science Misleading Our Understanding of Minds?
58:14 - Bioelectric Patterns and Drug Discovery
1:04:07 - How to Build a Predictive Developmental Model
1:13:20 - Levin’s Top Books, Thoughts on Collective Existence
1:18:00 - Questions that Keep Michael Up at Night
1:22:32 - How Does Levin Interact with AI?
asfaan murthyulas retweeted
Hot take: Most autoresearch today is just glorified overfitting.
Current autoresearch pipelines are essentially outer-loop searches for recipes that work in ONE specific setting. Try transferring a modded-nanogpt recipe to a different language or dataset. How much of that "optimal" recipe still holds up?
We need to move beyond static recipes toward inner-loop autoresearch: optimizers, architectures, hyperparameters, etc., continuously adapting DURING training based on the training trajectory itself.
Check out OPAL that enables this! Cool work with @SimulatedAnneal, Jia-Hao Liang, and @vincesitzmann.
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
asfaan murthyulas retweeted
“If you want to be successful, I would encourage you to grow a tolerance for failure.”
— Nvidia CEO, Jensen Huang
asfaan murthyulas retweeted
Researchers proved transformers can simulate any attention mechanism, without training a single weight.
Indian Researchers published a mindblowing paper called "On the Existence of Universal Simulators of Attention"
And it could solve the biggest bottleneck in artificial intelligence.
Every major AI model today, GPT-4, Claude, Gemini, runs on one core mechanism.
Attention.
It is the engine of modern AI. It’s how the model knows which words matter.
But it has a fatal, architectural flaw.
It scales quadratically. The more text you feed it, the exponentially more compute it burns. It is the reason AI is so expensive to run. It is the reason context windows have hard, unyielding limits.
Until today.
A new paper just dropped called "On the Existence of Universal Simulators of Attention."
They didn't just find a clever engineering workaround. They proved a fundamental mathematical reality.
A "Universal Simulator" exists.
This means you can perfectly replicate the exact cognitive power, mapping, and precision of the Attention mechanism without actually using it.
You can simulate the intelligence. Without the quadratic cost.
If this theoretical framework is translated into production code, it rewrites the economics of AI forever.
Infinite context windows. Zero quadratic compute walls. Models a hundred times more efficient than what we have today.
asfaan murthyulas retweeted
Over the past year, the Discovery team at @GoogleDeepMind, in a project led by student researcher @dsantosmarco, has made new discoveries in one of the most beautiful open problems at the edge of computation: Finding the 6th Busy Beaver (BB) number.
While most AI math systems solve human-generated problems, like theorems, this work tackles a different aspect of mathematical creativity: Discovering hidden problems.
For years, the BB Community has pursued one of theoretical computer science’s most captivating collaborative quests: finding the simplest programs that run the longest before halting. Because determining whether these programs ever stop is uncomputable, no universal algorithm can solve it, meaning new mathematical techniques need to be found and invented to make progress.
In this way, the BB challenge serves as an open ended domain for mathematical creativity and, in particular, a generator of hard math problems called Cryptids. These are tiny programs whose halting behavior is governed by and equivalent to deep, unsolved math problems in number theory, similar to the Collatz or Goldbach conjectures. Historically, the BB community had recognized ~20 Cryptids in BB(6), and the best problems were selected for the Busy Math Olympiad (BMO).
Building on their work, Marco built a pipeline powered by Gemini 4 Argon that:
1️⃣ Simulates raw Turing machine traces to compress recurring patterns.
2️⃣ Conjectures mathematical transition rules & runs synthetic experiments to test edge cases.
3️⃣ Proves the equivalence formally in Lean 4.
Using this pipeline, Marco and the team discovered 20 new potential Cryptids and verified 33 more machines previously studied in the BB community. All the proofs were open sourced and shared in the community discord channel and we hope that they will help the community to make progress towards resolving BB(6). We also contributed related halting/non-halting proofs directly to the community GitHub repository.
Finding these Cryptids shows hidden links between simple programs and number theory, and helps mathematicians map the limits of computation. I'm also excited about Gemini 4 Argon's ability to help researchers both solve hard problems and accelerate scientific discoveries, of which Busy Beaver is just one example.
Work led by our student researcher @dsantosmarco with paper contributions from @vivek_veeriah, @Dahoas1 , Andrew Bolt, @zuza777 , Satinder Singh, and myself with additional contributions from Hussain Masoom, @maxencefaldor , @MoritzFirsching and Petar Sirkovic.
Read more about it in our paper: arxiv.org/abs/2610.22603
asfaan murthyulas retweeted
Banger paper on evolving agent interfaces.
Pay close attention to this new trend if you build with agents.
Like optimizing harnesses, it's remarkable what you can achieve with agents that have access to better interfaces.
Computer-use agents usually act through interfaces that people design by hand, so an agent may type a spreadsheet cell by cell when one line of code would do. Simular's Agent S4 has a meta-agent that builds those interfaces instead. It reads the task agent's trajectories, with no success labels, and writes new tools and notes on when to use them.
With the evolved interfaces, an open 27B model reaches 21.3% on OSWorld V2, 3.8 times that of the same model with hand-built interfaces.
That is above GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their cost.
Population-based selection keeps the interface repositories that do best on validation tasks the meta-agent never sees, so the changes carry over to new tasks.
On ScienceBoard, the evolved interfaces raise success from 41.4% to 58.6% with the same model. The code is open source.
arxiv.org/abs/2610.28601
Chat with Paper: academy.dair.ai/papers/evolv…
asfaan murthyulas retweeted
Interesting work from Anthropic.
They reframe MLP neurons as a stronger basis for interpretability than commonly assumed.
It is standard practice to study circuits with transcoders, because individual MLP neurons are considered too polysemantic to interpret.
They find that many active transcoder features in late layers are close copies of single MLP neurons. They measure this across 13 open-source transcoders.
These neuron-aligned features are also the only ones with a stable link to the underlying MLP. The other features change more between training runs, don't match the MLP's Jacobian, and move less from their random initialization.
That makes MLP neurons a stronger starting point for interpretability than usually assumed. It also means the non-neuron transcoder features need better validation before they are used to explain model behavior.
arxiv.org/abs/2610.24521
Chat with Paper: academy.dair.ai/papers/compa…
asfaan murthyulas retweeted
Highly recommended read from Google DeepMind and colleagues.
There is a lot of interest in self-evolving agents, but they're hard to get right.
This work highlights how important and difficult memory and retrieval are in agents.
The big question in this paper is whether self-evolving agents keep learning without forgetting.
They tested ReasoningBank, GEPA, and Meta-Harness on four task streams that drift in topic, workflow, or domain, built from BIRD, BFCL, and ScienceWorld.
The weights stay frozen, and the agents adapt by rewriting prompts, memory, and skills.
Self-evolution improves overall scores on most streams. But more adaptation consistently means agents forget more.
Methods that rewrite a shared prompt or workspace make global edits, so they interfere with unseen tasks and overwrite older knowledge when the stream shifts.
Memory retrieval keeps updates more local, but it still pulls in memories from other domains and collides when similar queries need different workflows.
Static benchmarks hide both problems, so test self-evolving agents on a stream that changes over time.
arxiv.org/abs/2610.25858
Chat with Paper: academy.dair.ai/papers/are-s…
asfaan murthyulas retweeted
シャノンは通信の送り手と受け手の間のコードブック共有を前提に通信理論を築き，通信の意味論的側面を切り捨てました。では学習済みVAEのエンコーダとデコーダは同じコードブックを共有しているのか？再構成が完璧でも，標準的な学習指標では判定できないことを証明しました
alphaxiv.org/abs/2610.14012
Perfect reconstruction can't show whether codes read off a VAE's latent space come back as written. I constructed two VAEs: exact inference, same per-example ELBO and code histograms. One reads every encoded latent back as written; the other, none.
alphaxiv.org/abs/2610.14012
asfaan murthyulas retweeted
QD-Rec: Open-Ended Evolution of Recommendations via Quality–Diversity
Blog: pub.sakana.ai/qd-rec
GitHub: github.com/SakanaAI/QD-Rec
arXiv: arxiv.org/abs/2610.23470
Open-ended search has been used to discover algorithms, generate scientific ideas, and develop self-improving agents. With QD-Rec, we explore how it can support a different kind of task: helping people make decisions.
Which company should an acquirer buy? Which customers might benefit from a product? Which papers should a researcher read? In each case, the task is to choose among existing options, not generate new ones. At first glance, a finite, predefined candidate pool seems to leave little room for open-ended exploration.
However, a fixed set of options does not imply a fixed set of reasons for choosing them. The same company might be worth considering for its technology or its potential to open up a new market. The search over these reasons remains open-ended even when the options themselves are fixed.
QD-Rec searches over both what to recommend and why. Instead of aiming for a single best answer, it builds a diverse collection of recommendations, each pairing a candidate with a natural-language hypothesis explaining why it fits a particular decision maker’s goals, interests, and background.
The system combines three components. A contextual bandit learns which items to investigate next, balancing promising candidates with less-explored ones. An LLM generates hypotheses by adapting or combining the earlier recommendations. An archive retains new recommendations when they offer a novel reason, or replaces a similar recommendation when the new one is judged better.
We evaluate QD-Rec on acquisition target discovery, customer targeting, and paper recommendation. Compared with variants that remove evolutionary hypothesis generation or contextual bandits, QD-Rec achieves the highest mean LLM-rated quality in most settings and usually builds the largest archive. We also found examples where earlier recommendations with a low quality became stepping stones for later discoveries.
These experiments are a starting point. Practical applications will require richer, up-to-date information and expert human feedback to assess whether the proposed QD-Rec holds up beyond evaluation by an LLM judge.
We are releasing the code for QD-Rec so that researchers and practitioners can explore open-ended recommendation with their own decision-making tasks.
asfaan murthyulas retweeted
Very important work. One of the things missing from local learning (PC etc) is fast weights. It is missing no more!
The connection to deltanet follows so naturally here.
asfaan murthyulas retweeted
科研 Agent 会提 idea 已经不稀奇了，难的是知道哪条真的值得继续做。
这篇 DiscoveryRSI 想让 Agent 学会筛选和改进研究假设。
它会围绕一个假设不断生成、修改和组合方案，同时要求每个假设都有明确证据、实验设计、判断标准和替代解释。
更关键的是，研究过程会反过来训练下一轮 Agent。
好的假设会保留下来，失败轨迹则用来定位研究流程哪里出了问题，再继续更新模型和 Harness。
两轮自我改进后，假设质量从 4.86 提升到 5.39，提升主要来自「可证伪性」和「数据是否真的能回答问题」。
还有一个挺有意思的结果：
DiscoveryRSI-27B 生成的假设中，有 6.3% 和模型知识截止后发表的研究方向相似；GPT-6 是 2.5%，Claude Opus 5 是 2.3%。
这个指标当然不能直接等同于真正发现，但方向很值得关注。
科研 Agent 的自进化，不该只学会写得更像论文。
它还要逐渐学会判断，哪些问题值得追，哪些证据真的够，以及下一轮研究该怎么做得更好。
📎 arxiv: arxiv.org/abs/2610.23839
asfaan murthyulas retweeted
1/ The tokenizer is a new scaling axis!
We introduce an approach for diffusable representation learning that's principled, scalable, and works when DINO/LeJEPA collapse.
We get joint tokenizer+diffusion scaling laws and it's compute-optimal vs raw diffusion
w/ @drfeifei @GordonWetzstein @jcjohnss @ericryanchan @vincesitzmann @KyleSargentAI
Thread, Paper, and Blogpost below
