I find the sales pitch around running LLMs locally to save money instead of paying for subscriptions somewhat unconvincing. Even if the economics work on paper, the hardware would need to be used heavily—potentially 24/7 for years—to reach the break-even point. And there are several practical trade-offs that are often overlooked. Local inference can be constrained by token-generation speed, while power and internet outages can disrupt access. More importantly, once the hardware is purchased, its capabilities are essentially fixed. The models running on that hardware, however, are not fixed. Over time, models are likely to become larger, more capable, and more computationally demanding. This creates a mismatch: you are making a long-term capital investment in hardware against a rapidly evolving software landscape. That said, I don't think the answer is necessarily to eliminate local models altogether. A more balanced approach could be to run a relatively small, quantized language model locally—perhaps under 30B parameters—and use it primarily as an orchestrator. It could handle lightweight tasks, maintain local context, determine what needs to be done, and decide when a more capable model is required. When additional intelligence or compute is needed, it could call an appropriate cloud model or inference instance. This creates a hybrid architecture: cheap, always-available local intelligence for orchestration, combined with on-demand cloud intelligence for computationally expensive tasks. A more compelling infrastructure model would therefore be AI compute on demand. Instead of owning expensive hardware that sits idle most of the time, users could have access to private cloud inference instances that spin up when needed and shut down when they are not. Something analogous to AWS Spot Instances could be offered specifically for AI inference, allowing users to access significant compute capacity at much lower costs without having to own or maintain the hardware. Rather than buying a machine and hoping it remains useful for years, you maintain a small local model that acts as your persistent AI interface and orchestrator, while treating large-scale intelligence as an elastic resource that can be accessed whenever required.