Design lead, creative tools @ Google Labs 🧪 | Views are my own.
San Francisco
Joined August 2009
- Tweets10.1K
- Following1.8K
- Followers3.2K
- Likes20.3K
Spun up a little home app last week. Sonos, Google Home/Nest, HomePods and Apple TV in one place: I can see what's playing on every speaker, pull up my Spotify, move music across ecosystems, and set sleep timers. Weather and the NYT are in there too. It's been great.
@gcduncombe with another amazing exploration
It might not look like it at first glance but I actually think this is the wildest thing I've made with AI!
Every pixel of this video (and all the animation) is pure math functions. No 3D models, no meshes, no textures, no render farm, no keyframes, no rigging. It's completely done with ~1,500 lines of shader code that Opus 5.5 wrote while I art-directed the scene in natural language.
Music and SFX generated with @AdobeFirefly 🧵
Nick Matarese retweeted
Od kiedy bawię się animacją, a stukną niedługo cztery dekady, czekałem na narzędzia, które pozwalają animować w sposób bardziej organiczny i kontekstowy.
Cała tradycyjna nakładka techniczna na animację komputerową jest wręcz absurdalnie przekomplikowana.
Działało to trochę jak bramka dostępu. „Patrzcie jakie to jest strasznie trudne, lepiej zostawcie to specjalistom, bo nikt inny nie pojmie głębi i złożoności tego magicznego procesu”. W sumie mało jest innych branż, które ze skomplikowania narzędzi zrobiły cnotę, jednocześnie czyniąc proces nieludzkim i nieintuicyjnym.
Mi od początku tej zabawy marzyło się coś innego. Dynamikę sceny czy ruchu postaci trudno buduje się na abstrakcyjnych krzywych beziera, na wielokrotnie zagnieżdżonych hierarchicznych szkieletach, ale łatwo ją pokazać gestem, palcami, ciałem, czy choćby i zagwizdać.
Wreszcie można. Ludzie są multimodalni więc fajnie, że maszyny w końcu też. Wreszcie zamiast „przesuń kontroler kącika ust 5 mm w górę, i zwiększ zmrużenie oczu, przesuń kontroler brwi w górę o 3mm” – będzie można powiedzieć „uśmiechnij się” .
Czego sobie i nam życzę.
Nick Matarese retweeted
Lots of discussion out there about our next model(!), so I wanted to give an early look as soon as possible. Introducing Gemini 4 Argon!
It shows frontier performance in complex workflows, cyber defense and software engineering. Teams are using it extensively at Google, from coding to quantum computing, great feedback.
Here’s a look at the benchmarks:
Take this a step further. Have an agent start up or cancel a sub based on what you watch. Only the 3 months of march madness or only during Football season and never worry about “can I watch the game”
Uh oh. Agents will likely break the entire subscription economy.
Stanford research shows ~50% of all subscription revenue comes from people simply forgetting to cancel.
But agents don't forget.
Muse, Instinct, and Grok Bot can already go through your accounts, find every subscription you forgot about, and make you decide if you want to keep it.
And the same research found people are 4x more likely to cancel when they're forced to actively decide.
Once personal AI agents are on everyone's phone, a lot of subscription companies are going to really feel it.
Nick Matarese retweeted
Software is increasingly treated as disposable in the AI era. Yet we spend half our waking lives looking at screens! We don’t want to spend that time with zombie products—monotonous, vacant, and uncared for. We want products with soul. 💜
I go deep on this in my talk from @lennysan's Summit: youtube.com/watch?v=GLvFTMtw…
25 million people using @FlowbyGoogle is wild to see. Absolutely love building this thing. Here’s to the next 25M! 🌊
Come create magic with us!
We are hiring research scientists and engineers🧑🔬 on the Gemini Omni team! Both in the US (Mountain View / SF) and Europe (London / Zürich).
Apply here: google.com/about/careers/app…
Read it the other night and in full agreement here. I was more responding to the established industry (photography, film, SWE, 3D, etc…) being quick to find problems with the quality. This thought that anyone testing Astra this weekend is actually worried about the quality of the surfacing or code. They are just seeing what it can do.
We are in the messy middle. As you said the calm before the storm.
The current wave of “OMG WE ARE COOKED BECAUSE AI CAN NOW DO X” will die down and then repeat as capabilities advance. This cycle is common in technological advancement. People test the systems with things they can verify. Call it a prompt challenge. Call it an eval. The derivative content, programs, systems in the early weeks is more a sign of life and finding the edges. Understanding the medium. Once well enough understood creation of new and novel begins.
Common behavior in evolution. Mimic what you know to test if it can be done. None of these testers make games. We saw similar behavior with early Assistants. Common questions were well known riddles like “how much would would a woodchuck chuck”. Rocket League acts more like a verifiable test. Something sufficiently advanced and well known to judge progress. They arent trying to make games and any of them who do will realize derivatives have no audience.
Nick Matarese retweeted
I texted someone inside a simulation.
She had to go upstairs and find her phone before she could text me back.
The video model didn’t decide what happened.
This is Argos, my stateful simulation engine, rendered through H3 using Fal Director. Thanks to @fal for the generous credits helping me bring it to life.
Underneath the video is a persistent world: characters, objects, locations, and systems. The characters have their own cognition, memories, and goals. They choose actions; the simulation determines what actually happens.
Her phone has a location. Messages have delivery and read states. Typing a reply takes time. If I move the phone before she gets there, it’s gone—even if she remembers leaving it there.
Argos turns those changing world states into instructions for H3 to depict.
You can also see her character thought stream. The video currently trails the simulation a little; I’m working on aligning the two.
Next: connect this to SMS, so you can text someone inside the world and receive their reply. Or whisper an impulse into her thoughts and see whether she follows it.
The bigger ambition is a world that keeps growing while its characters inhabit it—with agents writing and maintaining the simulation’s systems.
I’ll be building more of this in public.
Not just a video that looks like a world. A world that keeps existing underneath the video.
👀
DialKit has a new home: dialkit.dev
Give the wheel a spin to see some of the amazing and creative ways folks are using it 🎛️