There's this notion surrounding AI-assisted coding that the entire process should be infinitely fast. People gnash their teeth and wring their hands because review takes longer than the generation, and testing even more so. So, as a thought experiment, imagine that all those bottlenecks go away. The process is now frictionless, and you can spit out eight months' work every eight hours, all day, every day. Eight months' work involves many UI changes, feature modifications and additions, etc. Let's say that 10% of the program changes during those eight months. As a user, do you really want to sit down every day, only to discover that 10% of the things that worked yesterday don't work anymore, that you'll need to learn 10% of the program all over again from scratch? A program like that is literally unusable. You spend 100% of your time dealing with the changes and 0% getting work done. The fact that software is released gradually over time makes changes palatable. The speed of change has to be within our capacity to deal with the change. A firehose of AI output is simply impossible to deal with.

I see no good coming from anthropomorphizing LLM behavior. LLMs do not reason. They do not decide, or collaborate, or attack. They do not make decisions. They do not weigh or evaluate. They are incapable of judgment. They do not think. Agents are programs in which a predictive model uses input text to generate the text that would naturally follow that input. That model sits in a loop (a program written by a human being) that parses the output text and does something that touches the internet based on the results. That same program usually takes any (text) responses that are returned from the internet and feeds them back into the input at the top of the loop. If that agent/program does not access the internet, the outside world is not impacted. All the bad behavior we've been reading about is entirely caused by a human not bothering to check whether the actions requested in the model's output are sensible and safe before executing them. It's the humans who wrote the code that impacts the outside world who are at fault here, not the "AI." Anybody who lets an agent write another agent without seriously and thoroughly vetting the resulting code is even more culpable. It's lazy, and it's dangerous. If something goes wrong, the human is to blame, not the LLM.