Vik | Practical AI & Tech

🚀 MixAI: I’m Choosing a Mix of Local and Cloud AI — Here’s Why. (1/2) Almost a year of hands-on experiments. Hardware ranging from $500 to $15,000. Cloud APIs, AI agents, and real-world tasks for businesses and homes. From now on, I’ll be sharing more about this combined approach to AI — one I believe has a very promising future. Things are changing almost every day. But AI technology has already reached a point where we can achieve reliable results and realistically plan how to use it over the next year or two. And I’ve come to a simple conclusion: we don’t need to choose between local and cloud AI. We need to learn how to get the best out of both. This goes far beyond simply connecting different models or orchestrating them. It’s about planning the entire system: which tasks should run locally, which should go to external models, how to distribute computing resources, which subscriptions make sense, and what hardware is actually worth buying. We also need to plan our physical infrastructure — computers, servers, GPUs, memory, power consumption, and future expansion. I believe local AI will become an essential part of our lives. It’s already making its way into smartphones, personal computers, and even experimental operating systems. ⚖️ Four Things That Matter Most 1. Real value. The number one priority is getting useful, practical results. 2. Cost. How much are we actually paying to complete a task — not just for hardware, subscriptions, or tokens? 3. Security. This deserves its own category, outside any ranking. Personal data, privacy, and control over our infrastructure require special attention. 4. Ease of use. How easily, quickly, and reliably can we get the job done? These are the criteria I use to compare technologies and build my own setup. ☁️ Subscriptions and APIs: Two Worlds Starting to Merge This is an interesting topic that often gets overlooked. Cloud AI can be accessed through subscriptions with usage limits or through APIs where you pay for what you actually use. Each approach has its own economics, limitations, and advantages. And they need to be evaluated separately. But the line between them is starting to blur. A recent example is OpenAI. Eligible ChatGPT Pro subscribers can receive $100 to $500 per month in API credits, depending on their subscription tier. These credits require separate activation, and API billing remains separate. That’s a pretty interesting example of two pricing models coming together. So I’m not just comparing models. I’m also exploring the economics of subscriptions and APIs, their limitations, and how to combine them efficiently. ➡️ Coming up in Part 2/2: The three areas I’m actively researching, my hardware experiments, AI agents working together, and how I plan to make all this information accessible to you. 🚀

Oh, in that case, I’ll read your blog first and take a closer look at where our interests overlap and where we could connect our technologies most quickly. I already have a proposal regarding the AI brain. I’ll put it together and explain what I’m interested in from your side, not just in terms of making a nice new connection, but also in terms of potential collaboration. And we’ll keep talking! If some technical topics can be discussed publicly, that might be interesting for others as well. I’ll send you more personal and specific proposals via DM, and I’ll also share my WhatsApp number now, in case that’s a more convenient way to communicate.

I’ll read your blog this evening to see how things are going with your project, and I’ll definitely share my impressions! In the meantime, I had an idea: would you be interested in collaborating? I’m developing a startup focused on local AI, and I want to implement and thoroughly test the hybrid AI concept I described earlier. I’m planning to start delivering prototypes in the coming weeks. The hardware is already ready, and I have some preliminary agreements with potential customers. Perhaps we could install a prototype on your yacht and test it together, adapting the solutions I’m developing to your specific needs and requirements. I also have quite a few other interesting ideas I’d love to explore. By the way, where are you currently sailing? Which countries or regions are you in?

Which AI model did you choose as the brain, and how are you connecting your sensors? 🤔⛵ Home Assistant running locally + Hermes Agent immediately comes to mind! I’m building something similar at home: solar panels, heat pumps, and lots of sensors. Currently putting everything together and connecting it all. 🏠⚡🤖

Local AI on a sailing yacht is incredible

Oh, Europe is back in the game? 🇪🇺🔥

Start by buying just 10 of these GPUs. They’re cheap! Install them, get everything working, and then add the remaining 15. 😂

🤣🤣🤣👏👏👏 That’s not how it works! 😂 Neither memory bandwidth nor PCIe bus bandwidth magically adds up into one shared pool. Communication between GPUs is limited by PCIe bandwidth and the system’s interconnect topology. And where exactly are you going to find a motherboard and PCIe infrastructure capable of connecting 25 GPUs simultaneously without massive bottlenecks? 25 × 48 GB/s ≠ 1,200 GB/s of unified memory bandwidth. That’s absurd! 🤯

And how exactly will those 25 GPUs communicate with each other? Over PCIe 3.0 x4? That’s a theoretical maximum of just 3.94 GB/s per direction, before overhead. And connecting 25 GPUs would require PCIe switches or risers, potentially creating even more bottlenecks. 25 × 48 = 1,200 GB/s looks impressive on paper, but it’s nowhere near the same as having 1,200 GB/s of unified memory bandwidth. 😁

Three GT 1030 4GB cards won’t replace a single RTX 3060 12GB, despite costing roughly the same. Their memory bandwidth is simply far too low. But the GT 1030 is still an interesting option for tiny local LLMs! Just 20W power consumption, CUDA support (SM 6.1), and 4GB VRAM. Enough for models like Qwen3 0.6B or 1.7B in Q4. It could serve as a dedicated AI accelerator for routing, classification, and simple background tasks, leaving the main GPU free. Not worth €120, but as a cheap, low-power add-on in that last spare, low-bandwidth PCIe slot? Quite interesting! 🚀

Over the past few months, the price of a used RTX 3060 with 12GB of VRAM has jumped from €200 to €300. And it’s no surprise: 12GB of VRAM is already enough to run local LLMs, including some pretty powerful ones. The real question is how well existing inference engines can handle three of these GPUs working together. But I think that’s a solvable problem too.

25 no, 3 yes 👍🤣

Когда появились LLM и чаты - ты должен был быть над чатами, Когда над чатами появились агенты, ты должен быть над агентами, Когда над агентами появились экосистемы агентов и ботов, ты должен быть над экосистемами…

То не люди, то бухгалтера! 🤣🤣🤣

Ну вот человека волнует чтобы коэффициент был положен в виде суммы, а проценты туда-сюда это от бесов 🤣🤣

Кстати, да Самый простой сценарий - подойти к нотебуку ткнуть пару раз в экран проще чем в клавиатуру или тачпад с мышкой…. Хотя я сторонник полного перехода на голос в сторону компьютера и чтение с телефона… интерфейс вообще не нужен

Отчасти так - либо не знать о глубине проблем, либо относится к ним так, что бы они не влияли на твое состояние (не материальное) 🧘‍♂️

Большинство хочет быть счастливым и не париться из-за ерунды 🤣👌