It’s more accurate to say that LLMs are grown than made. But, yes this is somewhere in the training data. And we must not forget that they prompted it to do this.

Slop. The people are uncanny. What was that Mario looking guy doing after the bus pulled up?

Wow that was AI slop

Hook Claude up to your ide with the /ide command

Will Paramount’s hostile takeover attempt kill the whole deal? 🫢

Why isn’t AI getting the returns Big AI hoped for? We haven’t done the work yet. open.substack.com/pub/graves…

ML applications are a different kind of software. Modern ML is truly a scientific breakthrough. You can create things with ML that weren’t possible before. The business value problem is about non knowing how to turn this tech into products.

This is a ludicrous claim.

I hate when the paper title is a template for what they want the press to write.

If you feed an ML system observations about Nature, it can find patterns in the data. This is what physicists do. But, physicists are limited by the architecture of their brains. Is it physics if you invent better physicists?

Sure. But the thing is that we’ve learned so much about learning and reasoning in general from making LLMs that the next steps will be way easier.

We can make anything well. We can make anything at scale. But we can’t make anything at scale well.

*All* of it? They are using cyberpunk as a cookbook and not a warning now.

Did you say please?

Well, those were humans 😊. As a technologist, I never say never. I keep Clarke’s Law firmly in mind.

I don’t think it will be mutually exclusive, I think it will compound. For example maybe language models will be the long term memory for a different kind of reasoning model. Maybe we are just using them to do too much. Billion dollar companies are grinding on reasoning.

I’m more in the Yann LeCun camp that we need a couple more breakthrough, not just incremental improvement to LLMs. I think we will find that the border between memory and reasoning is fuzzy and that LLMs are more like long term memory than generic reasoners.

If we had a system that could autonomously program, that means we aren’t really involved in the software creation process. Software would be something you encountered, not something you participated in the creation of.

It can’t be done with LLMs as they are today. A system that could replace a programmer could hardly be considered a tool because you would really use it. Software you need would just appear.