Proud to be working on open models
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-W…
Philip Monk retweeted
.@essential_ai's rnj-1 model is now on Ollama!
ollama run rnj-1
8B parameter, open-weight dense model trained from scratch. The model is optimized for code and STEM with capabilities on par with other state of the art open-weight models.
Let's go! 🚀🚀🚀
ALT ollama run rnj-1
It's been a blast to lead the infrastructure effort to train this model. I'm excited to see it out in the world!
We are beyond thrilled to share our first flagship models, Rnj-1 base and instruct 8B parameter models. Rnj-1 is the culmination of 10 months of hard work by a phenomenal team, dedicated to advancing American SOTA OSS AI.
Lots of wins with Rnj-1.
1. SWE bench performance close to GPT 4o.
2. Tool use outperforming all comparable open source models.
3. Mathematical reasoning (AIME’25) nearly at par with GPT OSS MoE 20B.
….
Philip Monk retweeted
Today, we’re excited to introduce Rnj-1, @essential_ai's first open model; a world-class 8B base + instruct pair, built with scientific rigor, intentional design, and a belief that the advancement and equitable distribution of AI depend on building in the open.
We bring American open-source at par with the best in the world.
You all have it so easy today with your petaflop gpus. In my day we had *floppy disks* that could only handle a few hundred kiloflops/s
Philip Monk retweeted
[1/2] We at Essential are driven by mission to advance fundamental research guided by first principles, rigor and sharing research openly.
If you're a PL guy who engages seriously with the problem the answer you come to is Jax and/or torch.compile, not rust
Protip: function in the presence of uncertainty in your own mind. Your tweets will be worse but you will be more aligned with reality
Philip Monk retweeted
[1/5]
🚀 Meet Essential-Web v1.0, a 24-trillion-token pre-training dataset with rich metadata built to effortlessly curate high-performing datasets across domains and use cases!
Leaving aside the main point of the post, I think people underestimate the degree to which the previous generation was influenced by stories of the one before that. There are more total stories now, but the number that affects any particular person is probably not any higher
that jony and sam video has me thinking about something - i'll try to explain it
the previous generation of silicon valley there were not yet too many stories of silicon valley
people did the things they were trying to do, went through twists and turns and then later the stories were written - and they truly are incredible
but now our heads are full of these stories - we can't do the thing without thinking "hey this is that thing they talk about"
and as soon as that happens it inevitably becomes performative - people moving to sf and posting a pic of their mattress on the floor
everyone want to tell the stories in realtime - it feels like a major plot point in the book that openai recruited jony ive!
and so content is produced with all the gravitas that this plot point deserves as though we're already looking at it in hindsight
the video really feels like two people meeting up again after accomplishing something together and reflecting on how they got there
except this isn't the end, it's actually the beginning, we don't know where all of this ends up - it could be a disaster
navel gazing has become so rooted in our culture to the point where it becomes the official marketing approach of multi billion dollar companies
This was a super interesting project to work on. It showed up when we started using muon at larger scales.
Muon is a serious competitor to AdamW, but it's tricky to scale up.
Our infra team has made fundamental advancements in parallelizing Muon on large scale distributed clusters. We're extremely happy with the result and it's now a part of our pretraining pipeline.
🔗Check out our new blog post on this:
essential.ai/blog/infra
essential.ai/
@ashVaswani @pcmonk @ishaankshah @karlstratos
There's a new version of this post
I don't know who needs to hear it, but for 100s of years there has been a significant tribe that believes basically this. Something in their worldview hits inf or nan and they start believing the end of the world is in the next 10 years. Reject it, don't let it make you impotent.
This and the related thread is a good example of precisely where I differ from a lot of trads. I don't actually think the Colossus or Louvre was/is more interesting than what finance and tech has brought us.
