Co-founder and CEO @SafetyWing (YC18), building a global social safety net. Tweeting about remote work, philosophy, and the future.
San Francisco, CA
Joined April 2009
- Tweets3.2K
- Following608
- Followers2.3K
- Likes18.8K
Look, just take a guess at how to pronounce their name when the call starts. They can correct you if they want to.
It is surprising how many times in my life I have explained exponential growth to people I have worked with, across teams and projects. It does not mean growing fast, it means growing as a function of itself, resulting in a stable growth %
Insider language happens when people prefer to sound knowledgeable about a field instead of being clear.
Paid ads are like heroin for startups. At first, it feels great, wow look, free customers. Then you track deeper you realise the customers convert worse, buy less, and churn faster.
It also takes your dignity away. The only way you get good at ads is by generating alternatives and keeping the winners. After a while, you optimise into clickbait, and end up looking pathetic.
People think intellectual events improve by becoming more polished. My experience is often the opposite. improvised speech is more interesting and more authentic. You endure more bad talks, but the tradeoff is worth it.
Most conferences are over prepared and too vain. If you are making a conference right now, can you please try less to impress. It is really boring to sit through.
(Btw thanks @JoinEdgeCity for being different, you were the exception that made me think of this)
The most obvious business model for space may be real estate. If you can credibly build a city there, you can pre-sell it from Earth.
Not 'own a piece of the moon' gimmick. But actually pre-sell city real estate, and use that funding to fund the construction of it
Request for startup: Map the resources on Mars before everyone else arrives. Make satellites or drones to take hyperspectral images, maybe magnetic surveys of Mars. Then sell the map of where the valuable minerals are to other companies. This will spur mining and help colonists, mining companies, and city builders.
At the beginning of someone's career, you can hire Steve Jobs. There was a point in time where you could hire Elon Musk. But you can't hire him now. So if you're only hiring late career people, you're ruling out the highest-output people.
When you visit the apartment of someone on long-term disability, you realise the UBI debate is missing something.
Money is only a small part of the equation. The harder problem is helping someone build a life worth living.
In the Norwegian fairy tale Soria Moria, the protagonist Halvor sees a shining castle far away. After seeing it, he can't live an ordinary life anymore.
I think this is the best (only?) fairy tale about vision. Its called vision because it's a kind of seeing.
You can see something that doesn't exist yet, something you can't arrive at through reason.
That's what founders do, they move toward what they've seen, and that is why you can have high confidence that something can exist, even though it isnt real yet, because youve seen it.
One of the great mistakes of my life, was that I literally took a ten year break from startups in service of normality. And tbh there was no pot of gold at the end of that quest. I am strange, and I can contribute much more in the ways I’m weird than in the ways I’m not.
In Norwegian fairy tales, the hero is often Espen Askeladd. Humble, optimistic, naive, and ready for adventure. He represents humble improvisation, creativity, and simplicity. I think its such a great example for founders.
The Universal High Income debate is missing a piece.
As someone who's tried to work this out in the Norwegian government and Nomad Citizen. If the number is high, universal high income doesn't work.
Either the amount is low, or the tax rate very quickly becomes impossible. This base math doesnt get cancelled at higher level of productivity.
Weird history of modern regulation: We think agencies, directorates, and departments are a natural part of government.
But this intermediate layer of rule making didn't exist in the West before the 1960s, it was imported by soviet style directorates.
Since then, these directorates have been given huge mandates to make rules without lawmakers, and we've created huge amounts of regulatory texts as a consequence.
Simplifying these regulations, and maybe even bringing lawmaking back under the authority of parliament/congress is a great AI use-case. AI won't solve consensus building, but it can simplify the rules we already have.
Stanford recently held an AI Social Science conference. This quote from a friend of mine is worth thinking about. A thread.
news.stanford.edu/stories/20…
A startup mistake I made so you don't have to:
In the early days of Superside, we were a hot company out of YC, we had a red carpet rollout, were given the option to choose valuation ourselves, and thought higher the valuation = better. That turned out to be a mistake.
Next round, we had great trajectory, but we weren't able to raise because investors priced us like a Series A when we didnt hit Series A milestones. It took us years to recover.
The lesson I took was to ignore outlier rounds you see. Instead look at the median valuations. The median data is available. You can make slight adjustments if you are over/under on revenue/growth relative to median.
This way you get a reasonable price, can close all rounds with more certainty, all your investors get a good deal so you build relationships over time, and the next round becomes way easier.
1/ The singularity is already here. It's just not everyone has passed the event horizon yet.
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.
This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:
- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.
This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
github.com/karpathy/nanochat…
All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.
And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
