
Full Stack SWE @sundayrobotics - UC Davis / PL Research - Prev. SWE Intern @ Capital One & Sunain & Lucid Automation
Redwood City, CA
Joined December 2018
- Tweets137
- Following1.7K
- Followers340
- Likes1.4K
Pinned Tweet
A lot of great work happening here @sundayrobotics. Check out the latest model ACT-2, that can learn new behavior from a single example, and achieve 99.1% accuracy.
This is going to be fun! luma.com/freestyle-26a2
Jake Roggenbuck retweeted
New benchmark dropped bw.swerdlow.dev/report
Jake Roggenbuck retweeted
We are going to need a different AGI benchmark. Where is the goalpost moving next?
Thank you for the book recommendation @akshaysunday!
Excellent so far!
Jake Roggenbuck retweeted
Someone asked me recently why I’ve become interested in aesthetics after having spent most of my life more interested in STEM-adjacent topics. I hadn’t really considered the question consciously before, but I’m certainly thinking about aesthetics more than I used to. I think it’s a confluence of things:
• Many things today are ugly and far uglier than they used to be or need to be. Once you see this, it’s kinda hard to stop perceiving it. (Early twentieth century phone boxes versus modern phone boxes; old water fountains versus new water fountains; etc.) As someone with a naively meliorist assumption that most things should be getting better rather than worse, it’s all a bit perplexing: why did we stop doing things nicely? Is it a choice? Was there a malevolent spell cast upon us? This vein led me to think more about modernism and why much of art became more intentionally "challenging", grotesque, opposed to prettiness, rebarbative, dissonant, etc. Can or should anything be done about this? Is this just how things ought to be?
• Relatedly, much of modernism involved a kind of explicit repudiation of cultural continuity and represented a schism with prior practices. This is maybe most evident in American architecture, where the International Style exhibition in 1932 initiated the displacement of a rich tapestry of prior styles. This cultural break seems important and interesting to me, and I suspect that the rejection had important consequences outside of the aesthetic domain. Samuel Hughes has been exploring this question in his recent writing at @WorksInProgMag; @RuxandraTeslo is also pulling on this thread. Elaine Scarry wrote about how beauty inspires creation. If so, the inverse may also be true: ugliness inhibits it.
• It’s clearly the case that changes in the aesthetic domain can at least inspire progress in other places. Petrarch helped set some of the preconditions for the Renaissance which in turn fostered the scientific revolution and Enlightenment. Things like World Fairs (the 1851 Fair at the Crystal Palace recorded 6 million admissions when the population was 21 million) reflect the popular interdependency that used to exist between aesthetics and material development.
• @tedgioia and others have written about stuck culture and how so many domains seem to have ceased to straightforwardly advance in the way that they did up until the nineties or thereabouts. This is obviously peculiar and interesting. What changed, and what does it mean? Is it about the internet and fragmentation? Is it about a loss of supply? Is it just about having reached the zenith of various mediums?
• I’m generally interested in markets and the dynamics of creation. In aesthetics broadly, I find the reflexivity between supply- and demand-side factors to be very thought-provoking. There’s a natural desire to view satisfaction of individual preferences as the yardstick to measure market success, but things get interesting and even a bit unsettling when we start to think about how the supply might start to shape the demand. I often think about this in the context of food. Why is food so much worse in Germany than many of its neighbors? Germany certainly doesn’t have less material ability to produce good food; indeed, Germany is richer than the countries around it. There’s probably something about German food supply chains that is impoverished relative to France and Italy, but the Germans themselves don’t seem too upset about it. It just seems that the Germans are stuck in an objectively worse market equilibrium than their neighbors: the food is bad and they’ve gotten used to it. The obvious question then is where else these kinds of reflexive patterns apply, and where else we’re stuck in some objectively inferior equilibrium, even if preferences are in some superficial sense being sated.
• While this is an extremely banal and obvious point, I hadn’t until recently thought much about or internalized how much one can study reasonably objective things ("the status of women in society", say) through artwork. (Thanks to @_alice_evans for opening my eyes here.) In this vein, I’m pretty excited about the possibilities over the coming years in computational art analysis. I want something that’s conceptually similar to Google Ngram timelines but for the visual arts.
• We've always tried to do things well at Stripe. I've come to see that attempting to do them beautifully is often a helpful way to break out of standard practices and to do something with greater novelty and in a way that might have other benefits besides. (Also, excellent people want to do great work because it is intrinsically satisfying. Explicitly allowing aesthetic considerations to carry weight avoids having to justify every assessment with some kind of torturous empiricism.)
• In his Nobel Lecture, Solzhenitsyn said that, among the Platonic virtues of goodness, truth, and beauty, that beauty is special, for it possesses a unique kind of irrefutability. He notes that arguments, writing, and philosophical systems can all be predicated on misapprehensions, but that “a true work of art carries its verification within itself.” He proceeds to observe that when goodness and truth are threatened, the “ever surprising shoots of beauty will still force their way through.” There is a lot of specious and motivated reasoning in the world today and plenty of questionable value systems. I don’t think that beauty directly reflects any definitive trait, but I’m intrigued by the idea that it can be a marker of deeper metaphysical coherence.
Jake Roggenbuck retweeted
Indoor 5k to inaugurate our new floor was one hell of a company event!
Props to
@delmohtasham
Jake Roggenbuck retweeted
🌀 35 dizzying loop-de-loops
🚧 1 unbuilt office floor
🏃🏻♂️ 5 kilometers
🫶 The ultimate team endurance test
Jake Roggenbuck retweeted
One perfect robotics video, with unlimited takes, is not enough. The real world demands robots that are resilient under chaos and disruptions.
In our live ACT-2 demo, @tonyzzhao throws everything at Memo: distractions, undone folds, twisted pants, even covering its head. Watch Memo recover with endless patience.
Sunday Robotics' Tony Zhao runs a fully autonomous robot demo live and unscripted, and it corrects itself every time you mess with it:
@brexton: "I want to confirm for the audience right now that this is fully autonomous. There's no teleoperation, no one behind the curtain, and this is real time. And we did not prepare this demo before. Why do you think seeing real-time demos is so rare for robotics? What the audience is seeing right now is not that common."
@tonyzzhao: "In order for a live demo to work, it's really stressing all parts of the system. From the bottom is the reliability of the hardware, then the software system, then the model. Because we're a full-stack company, we're able to do excellent work across all of them. We have these robots running pretty much every day, and it just keeps getting better as they practice."
@brexton: "So kids can bump into Memo, people can drop clothes, and it will correct itself just on the fly."
@tonyzzhao: "Act Two is the first robotic policy that can handle the diversity of homes in a way that is this reliable. You can see as I'm twisting it, if I twist it in again, it will keep doing it. So Memo has infinite patience."
@sundayrobotics
Stunning demo from @tonyzzhao showing off Memo and the new ACT-2 model.
A few great things I saw in this live demo:
- Memo was able to operate even with a shirt obstructing its view from the hat camera
- Memo was able to quickly recover when Tony threw more shirts on top of what Memo was folding
- Memo can very precisely fix twists in pant legs and shirts
- Memo can be given new skills just with software updates
Incredible demo!
Sunday Robotics' Tony Zhao runs a fully autonomous robot demo live and unscripted, and it corrects itself every time you mess with it:
@brexton: "I want to confirm for the audience right now that this is fully autonomous. There's no teleoperation, no one behind the curtain, and this is real time. And we did not prepare this demo before. Why do you think seeing real-time demos is so rare for robotics? What the audience is seeing right now is not that common."
@tonyzzhao: "In order for a live demo to work, it's really stressing all parts of the system. From the bottom is the reliability of the hardware, then the software system, then the model. Because we're a full-stack company, we're able to do excellent work across all of them. We have these robots running pretty much every day, and it just keeps getting better as they practice."
@brexton: "So kids can bump into Memo, people can drop clothes, and it will correct itself just on the fly."
@tonyzzhao: "Act Two is the first robotic policy that can handle the diversity of homes in a way that is this reliable. You can see as I'm twisting it, if I twist it in again, it will keep doing it. So Memo has infinite patience."
@sundayrobotics
Please help @flooburrito find the missing cardboard cutout of himself. He said he’d buy me lunch tomorrow if I repost it.
🚨 I am officially asking X to help track down my missing body double🕵️♂️
8 weeks ago, @brainstub accidentally shipped a life-size cardboard cutout of ME to the wrong address
If you’re using me as a coat rack, wall decor, or worse.. pls reach out
Pictured: The note we left after our failed investigation
Interesting to hear about @bscholl’s “Confusion List”, where he kept track of topics that he didn’t fully understanding at first principles. He’d try to take an item off that list each week, except this list kept growing, but he still found it useful.
Startup School is great so far! Looking forward to tomorrow.
@ycombinator @boomsupersonic
Looking forward to hearing @garrytan speak at @ycombinator’s Startup School!
Looking forward to hearing @garrytan speak at @ycombinator’s Startup School!
Sunday Robotics CEO @tonyzzhao believes home robots only need to master a handful of chores before they become indispensable.
"What is the minimum number of tasks that a robot needs to do to justify its own existence so that people love having it in their homes? I actually think the list is pretty short. The tasks are pretty repetitive. You just need to learn it and just keep doing that over and over again, like folding shirts or loading dishwashers."
Jake Roggenbuck retweeted
my CEO's favorite way to tell me he thinks someone is full of shit is to say that they "speak with low information density" (yes he's a scientist)
Jake Roggenbuck retweeted
We just hit a weird milestone: our model became more reliable than your average home WiFi.
Just like everybody else, we thought cloud inference was the obvious choice. Yet 2 days into the ACT-2 eval, our mind completely changed.
If our hero @ArpitKalla didn’t cook, this video wouldn’t exist 🧵
Jake Roggenbuck retweeted
A lot of great work happening here @sundayrobotics. Check out the latest model ACT-2, that can learn new behavior from a single example, and achieve 99.1% accuracy.
Jake Roggenbuck retweeted
Replying to @JakeRoggenbuck
I bet you wish you could learn that rubics cube from a single demo
































