SWE @google Prev: CIS @PennEngineering, https://xitter.cf/t.co/2lyvZq0GOO 🇺🇸🇪🇺🇮🇹 尼克
Palo Alto, CA
Joined January 2011
- Tweets1.5K
- Following479
- Followers271
- Likes26.6K
Nick Fausti 👾 retweeted
Meet EmbeddingGemma 2, our first natively multimodal open model for on-device embeddings.
It expands beyond text to unify code, images, audio, and video in a shared space. 🧵
Nick Fausti 👾 retweeted
EmbeddingGemma 2 from @googlegemma, our compact 740M-parameter AI model built specifically for on-device and edge apps, has arrived.
While the first-generation model delivered best-in-class text embedding, EmbeddingGemma 2 natively understands it all, connecting text, images, audio, and video in one shared space. This means you can search across different formats like text, images, audio, and video, without having to manually organize, translate, or label them first.
Nick Fausti 👾 retweeted
Two major advances for Western science were:
1. The development of logic, and
2. the realization that we can discover causal relationships.
Inspired by Turing, I have tried to put the two together...
~Conjecture Institute Advisor @yudapearl
Nick Fausti 👾 retweeted
Things are heating up on Terry Tao’s blog. In “If Math Is More Than Proof, We Need to Better Celebrate the Rest of It,” Grant Sanderson of @3blue1brown proposes “open exposition problems” -- rewarding the work of making math genuinely understandable.
terrytao.wordpress.com/2026/…
Nick Fausti 👾 retweeted
It is inevitable that all AI will converge towards symbolic learning (i.e. modeling data by finding the shortest symbolic program that explains it), since that is the optimally efficient form of AI. But there can be more than one evolutionary path to this destination.
🤖🧠NEW PAPER🧠🤖
(The result of an 8-year project!)
LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it?
Our finding: LLM representations have implicit symbolic structure!
Link in thread ⬇️
1/n
![Overview of the paper.
Title: The Emergent Symbolic Structure of Artificial Neural Networks
Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it?
Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds](https://pbs.twimg.com/media/HRJ5Jr5W0AAQ-sN.jpg?name=orig)
ALT Overview of the paper. Title: The Emergent Symbolic Structure of Artificial Neural Networks Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it? Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds
Nick Fausti 👾 retweeted
I'm so excited that our @theworldlabs team has achieved a major milestone today! Introducing Atlas - a first of its kind multimodal world model trained from scratch! 🚀
Atlas is capable of generating frames with pixel-perfect camera control, reconstructing large scenes from as few as one single input image, simulating space-time by reframing videos, natively outputting 3D spaces from one or more input images, composing multiple posed images into a consistent 3d world, and more! This is the best camera conditioned world model ever, opening doors to many possible use cases from VFX to robotics. I'm so so so proud of our team!♥️
Nick Fausti 👾 retweeted
Every student needs to read "You Are NOT Dumb, You Just Lack the Prerequisites" by @lelouchdaily.
"It’s like walking into a movie halfway through—you can’t understand the plot because you missed the beginning."
Unfortunately, those who need to hear it most, seldom do.
Amazing.
modelrift.com/blog/openscad-…
A fluorescent-protein spin qubit | Nature nature.com/articles/s41586-0…
Nick Fausti 👾 retweeted
That one neuron connects to about 7,000 others. Your brain has 86 billion of them. Do the math and you get somewhere around 100 trillion connections inside your head. More connections than stars in 1,500 galaxies.
And each connection point is way more complicated than anyone expected. A Stanford lab found that every single connection contains about 1,000 tiny switches that can store memories and process information at the same time. So your brain is running roughly 100 quadrillion switches right now, while you read this sentence.
The wild part is the power bill. Your brain runs on 20 watts. That’s less energy than the light in your fridge. The world’s fastest supercomputer needs 20 million watts to do the same amount of raw calculation. A million times more power for the same output.
We’re still nowhere close to understanding how any of this works. In October 2024, a team of hundreds of scientists finished mapping every single connection in a fruit fly’s brain. Took six years and heavy AI help. That fly brain had 140,000 neurons. Yours has 86 billion. Google and Harvard also mapped a piece of human brain last year, a speck smaller than a grain of rice. That speck alone contained 150 million connections and took 1,400 terabytes to store. The lead scientist said mapping a full human brain at that detail would produce as much data as the entire world generates in a year.
A tiny worm had its 302 brain cells mapped back in 1986. Almost 40 years later, scientists still can’t fully explain how that worm’s brain keeps it alive. Your brain has 86 billion of those cells, each one wired to thousands of others, each wire packed with a thousand switches, all of it humming along on less power than a lightbulb.
Nick Fausti 👾 retweeted
Cool use case of AlphaFold, this is just the beginning of digital biology!
this is actually insane
> be tech guy in australia
> adopt cancer riddled rescue dog, months to live
> not_going_to_give_you_up.mp4
> pay $3,000 to sequence her tumor DNA
> feed it to ChatGPT and AlphaFold
> zero background in biology
> identify mutated proteins, match them to drug targets
> design a custom mRNA cancer vaccine from scratch
> genomics professor is “gobsmacked” that some puppy lover did this on his own
> need ethics approval to administer it
> red tape takes longer than designing the vaccine
> 3 months, finally approved
> drive 10 hours to get rosie her first injection
> tumor halves
> coat gets glossy again
> dog is alive and happy
> professor: “if we can do this for a dog, why aren’t we rolling this out to humans?”
one man with a chatbot, and $3,000 just outperformed the entire pharmaceutical discovery pipeline.
we are going to cure so many diseases.
I dont think people realize how good things are going to get
Nick Fausti 👾 retweeted
Solid mathematical ideas almost always outperform contrived engineering tricks.
For years deep learning has been dominated by increasingly complex architectural hacks: CNN blocks, attention layers, channel mixers, residual pathways, normalization stacks.
Every few years a new architecture is announced as if it were a revolution.
One of the most famous examples was Kaiming He and Residual Networks (ResNet). At the time he was paraded around the AI world like a celebrity because residual connections supposedly “solved” deep learning.
But these were largely engineering patches.
Now something much more interesting appeared.
A new architecture called CliffordNet returns to mathematics — specifically Clifford Algebra, developed in the 19th century by William Kingdon Clifford.
Instead of stacking arbitrary modules, the model is built around the geometric product
uv = u·v + u∧v
A single algebraic operation that simultaneously captures inner product structure and geometric interactions.
In other words: the math already contains the interaction mechanism.
No attention blocks.
No mixer layers.
No architectural spaghetti.
The result:
• 77.82% accuracy on CIFAR-100 with only 1.4M parameters
• roughly 8× fewer parameters than ResNet-18
And with strict O(N) complexity.
The paper even suggests that once geometric interactions are modeled correctly, feed-forward networks become largely redundant.
A good reminder for the AI community.
Engineering tricks can dominate for years.
But eventually mathematics shows up and deletes half the architecture.
Paper:
[arxiv.org/pdf/2601.06793?utm…)
19th century geometry just walked into computer vision.
Nick Fausti 👾 retweeted
It's astonishing to witness the improvement in public education--reading, math, attendance, grad rates--in Alabama, Louisiana and Mississippi. But we're busy fighting culture wars rather than scaling up what these three states have done. Please do read: nytimes.com/2026/02/09/opini…
Nick Fausti 👾 retweeted
for levin, the distinction between 'living things' and 'machines' isn't a tenably distinction
everything is a 'smooth metamorphoses process' where engineered systems are increasingly part of the ' intersecting continua' of life
his words: 'we need to get on board with this'
Nick Fausti 👾 retweeted
🎯 "Progress creates identity.
Identity creates consistency.
Consistency creates more progress.
Your brain does not need motivation, it needs evidence that you are improving."
Most devs wait for “motivation” to learn system design, Rust, distributed systems, whatever.
But motivation usually shows up after you get a small win.
A few examples that change your brain:
+ The first time you debug a production issue fast, without panicking
Suddenly you feel like you belong in the room
+ The first time you read logs/traces and you can actually explain the bottleneck
Now performance is not scary anymore
+ The first time you refactor a messy module and future you thanks you
You start enjoying clean architecture
+ The first time you ship something and users actually use it
You stop caring about tutorials and start caring about outcomes
Progress creates identity.
Identity creates consistency.
Consistency creates more progress.
So for any engineer stuck right now, do not aim for huge goals. Aim for daily proof:
1. Solve one small bug every day and write the postmortem in your notes
2. Read one incident writeup a week and summarize the lesson
3. Build one tiny project feature weekly, deploy it, measure latency/errors
4. Teach one concept in public, even if it is basic
Your brain does not need motivation, it needs evidence.
Once you have evidence you are improving, you will not need “discipline” anymore, you will feel offended to skip.
Nick Fausti 👾 retweeted
I used to think Sapiens was a great book. Sweeping, provocative, the kind of book that makes you feel like you finally understand the big picture of human history. It's on every CEO's bookshelf, assigned in universities, praised as a masterwork of synthesis. Yuval Noah Harari is treated as one of the serious thinkers of our time.
But something nagged at me. Some passages felt off. Claims that human rights are just figments of our collective imagination, not real things, just stories we tell ourselves. That nations, laws, money, justice, doesn't exist outside our heads. That meaning itself is a delusion we've invented to cope. That we're far more powerful than ever before but not happier. That hunter-gatherers had it better because they had no dishes to wash, no carpets to vacuum, no nappies to change, no bills to pay.
That sounded depressing to me, but was perhaps just the realistic scientific worldview? What it meant to see the world clearly, without comforting illusions.
Then I read The Beginning of Infinity by @DavidDeutschOxf. Deutsch has a concept he calls 'bad philosophy.' Not philosophy that's merely false, but philosophy that actively prevents the growth of knowledge. Ideas that close doors rather than open them. That makes problems seem unsolvable by design.
After soaking in Deutsch's framework (it's dense, a bit like digesting a delicious whale), it becomes clear: Harari's books are riddled with bad philosophy. They're smuggling nihilism in under the guise of scientific objectivity. Some examples:
On meaning: "Human life has absolutely no meaning. Humans are the outcome of blind evolutionary processes that operate without goal or purpose... any meaning that people inscribe to their lives is just a delusion."
On human rights: "There are no gods in the universe, no nations, no money, no human rights, no laws, and no justice outside the common imagination of human beings."
On free will: "Humans are now hackable animals. The idea that humans have this soul or spirit and they have free will, that's over."
On progress: "We thought we were saving time; instead we revved up the treadmill of life to ten times its former speed." The Agricultural Revolution? "History's biggest fraud." We didn't domesticate wheat, "it domesticated us."
On our cosmic significance: "If planet Earth were to blow up tomorrow morning, the universe would probably keep going about its business as usual. Human subjectivity would not be missed."
On the future: "Those who fail in the struggle against irrelevance would constitute a new 'useless class.'" Homo sapiens will likely "disappear in a century or two."
This is bad philosophy. It tells us our problems are cosmically insignificant, our solutions are illusions, and that progress is neither desirable nor within our control. It's also perfect nonsense. No one would ever go back to being hunter-gatherers. Would you rather worry about your kid spending too much time on Roblox, or face the 50% chance she won't reach puberty?
And our so-called "fictions"? They ended slavery. They gave women equal rights. They solved hunger. They eradicated smallpox. They turned sand into computer chips. They got us to the moon, and hopefully soon, to Mars and beyond. These "fictions" are already reshaping the universe, and over time they may become the most potent force in it.
Now compare Deutsch:
"Humans, people and knowledge are not only objectively significant: they are by far the most significant phenomena in nature."
"Feeling insignificant because the universe is large has exactly the same logic as feeling inadequate for not being a cow."
"Problems are soluble, and each particular evil is a problem that can be solved."
"We are only just scratching the surface, and shall never be doing anything else. If unlimited progress really is going to happen, not only are we now at almost the very beginning of it, we always shall be."
Where Harari sees a species of deluded apes stumbling toward obsolescence, Deutsch sees universal explainers, the only entities we know of capable of creating explanatory knowledge, solving problems, and potentially seeding the universe with intelligence.
The difference isn't academic. Ideas shape action. If you believe life is meaningless, progress is a trap, and humans are hackable animals with no free will, how does that affect what you build? What you fight for? What you teach your children?
Harari's books sell because they flatter a fashionable pessimism. They let readers feel sophisticated for seeing through the "delusions" everyone else lives by. That smug cynicism is corrosive. And it's everywhere: in schools, in media, in bestselling books. More than half of young adults now say they feel little to no purpose or meaning in life. This is what happens when you teach an entire generation bad philosophy. Less progress, less health, less wealth. Less flourishing. And ultimately, a higher chance that civilization and consciousness go extinct.
Fortunately, there's another equally well-written, but much truer, account of homo sapiens, appropriately titled 'The Beginning of Infinity'. And this one smuggles no despair in by the backdoor. But let's give Harari credit where it's due. He is right about one thing: if planet Earth blew up tomorrow, we wouldn't be missed. Because there'd be no one left to miss us, just a careless universe, blindly obeying physical laws. We are the only ones who can miss, but we're not going to. We're going to aim, hit, and keep going.
Full credit for the amazing meme to @Ben__Jeff
