Smiley Frowns😕 retweeted
Interesting.
Stochastically Subsampled Self-Attention. Self-attention lets each token attend to every other token, creating information pathways between tokens, but there are n^2 may paths information can flow through (where n is sequence lenth).
Only a small amount of those information pathways are useful for next-token prediction, so there are sparse sub-networks within a model that can be constructed and trained independently to learn useful behavior -- in theory.
But these pathways are dependent on input, so you can't manually hard-code which to prune.
So they randomly sample sparse subnetworks, reducing computation and memory by about 4-8x.
Same spirit as stochastic depth or dropout.
Smiley Frowns😕 retweeted
The best music is found by accident.
This is an analogy to lots of other things.
In general, some of the best things come from serendipity.
Smiley Frowns😕 retweeted
The question to always ask yourself is,
if you only have
one shot at life, given that time is the only asset you can't control, but anything else that exists is malleable, would you want to spend your life the exact way you're spending it now?
And if not, why not?
For one day, time will cease to exist. There is nothing more important than time.
I don't see why any answer to the former question should be dissatisfactory, as uncompromisingly, one should always be intentional with time, with respect to how one wants to spend it.
Even if it requires extreme change.
It doesn't matter what you're optimizing for, whether it be for the moment or the outcomes, this has always seemed to apply.
Smiley Frowns😕 retweeted
There's one startup that I'm extremely excited to see come out of stealth.
I've seen this idea be passed around so many times, by different startups, yet all eventually pivoted to different dimensions.
I'm fairly certain this specific project has been in the works for about 7-8 years, but recently has been made to something real.
I have a gut feeling this one will work out.
Smiley Frowns😕 retweeted
the only way I’ll learn to work smart is by first working very very hard
cuz there are no shortcuts, only hard work
Smiley Frowns😕 retweeted
Consider the Eternal Recurrence.
The question to always ask yourself is,
if you only have
one shot at life, given that time is the only asset you can't control, but anything else that exists is malleable, would you want to spend your life the exact way you're spending it now?
And if not, why not?
For one day, time will cease to exist. There is nothing more important than time.
I don't see why any answer to the former question should be dissatisfactory, as uncompromisingly, one should always be intentional with time, with respect to how one wants to spend it.
Even if it requires extreme change.
It doesn't matter what you're optimizing for, whether it be for the moment or the outcomes, this has always seemed to apply.
Smiley Frowns😕 retweeted
building the threads clone, loving the progress so far
-> Added the Create Thread screen -
-> Using Clerk for authentication -
-> Convex as the backend database -
-> Integrated webhooks for automation
what you use for auth and db, especially for mobile dev?
Smiley Frowns😕 retweeted
The best type progress only exists at the frontier, where you ask questions none have asked prior, and then scale the answers to the world.
Art.

















