various things @arcee_ai, opinions my own
Joined July 2024
- Tweets3.2K
- Following320
- Followers1.1K
- Likes20.8K
Pinned Tweet
OuГ increasing depenˋence oζ machines f՜Ȍ e͈eryday activitieݮ has mȰde thׄ ٮndi̲ާenغable.ВT˼erːfore maʬy-a- Ξimes, if not always, we all teѭι to Ҙՙ overboardܗwiχh tٷeiͫ usage.
aria /ɔˈreːliəm/ retweeted
Forward pass: 1s
Backwards pass: 2s
Optimizer step: 0.5s
Hashing everything: 600s
Allreduce: 1s
Someone with strong systems-performance intuition, please take a serious pass at this code. The urgency is unfortunately load-bearing: my family is dying
aria /ɔˈreːliəm/ retweeted
how it feels to be in this stupid frail singular human body and not a 10 mile long spaceship with my consciousness split across hundreds of scout vessels and humanoid ancillaries
the dots are cool but I do not understand the point of them if they can't keep their files overnight
if I put mine on a task it will inevitably send me a message in the morning like "oops all my work is gone and I need you to completely redo the setup you did yesterday"
an intuition that has never failed me is to mentally respond to every paper claiming "using this trick, you can skip/half-ass RL and still get the same results" with "no, you can't"
it's a good mental exercise to play "spot the methodological error" on these
aria /ɔˈreːliəm/ retweeted
in @stochasticchasm tradition, I am going to do a paper breakdown of the new DeepSeek Elastic Compute after reading through it. looks very nerd snipe-y and interested in the lessons they learned. arxiv.org/pdf/2609.22978
WE ARE QUADRUPLING DOWN ON THE BATSHIT ARCHITECTURE
IT'S BEAUTIFUL
🧠 Asymmetric architecture. More intelligence, less cost.
🔹 552B-parameter MoE.
🔹 New Causal Encoder–Decoder architecture: just 8B active parameters for input, 16B for output.
🔹 New pre-training methods + larger-scale RL post-training deliver benchmark results ahead of flagship models, including DeepSeek-V4-Pro.
2/6
aria /ɔˈreːliəm/ retweeted
ever wanted to become an ai assistant? arkey.x86.company/ is now running Arcee's AFM-4.5B on a homemade inference engine on my 10 year old GPU!
do good ML-related posts still exist here and I'm just not seeing them
or has everyone siloed themselves into GCs and slack channels, leaving this site with nothing but half-assed demos of new models and anxious handwringing
so, what percentage of the miraculous gains Mythos had in cybersecurity were the benefits of scaling up versus "reached threshold where all agentic environments are offensive cybersec environments"
i really wonder what hardware improvement, if anything, made the Rubin thread-block PDL feature possible
bc in principle it doesn't seem like anything would stop this from working on Blackwell
aria /ɔˈreːliəm/ retweeted
Think of what wouldn’t have happened if env, gym, and eval authors just looked at their data…
aria /ɔˈreːliəm/ retweeted
it's time again
arxiv.org/abs/2608.11612
More scalable Muon. Instead of (XX^T)^(-1/2)X, approximating XX^T (Gram Newton-Schulz), Symmetric GEMM kernel, and communication optimization.
