MICROSOFT CHARGED HUNDREDS FOR THESE WINDOWS 7 PRO AND OFFICE 2016 LICENSES AND HE READ THE KEYS OUT LOUD ON CAMERA BEFORE SCRAPPING ALL 7 MACHINES THEY CAME ON 00:45 he flips the sticker toward the lens and gives it away without ceremony, "if anybody wants this dell license, yep there you go, windows 7 pro, there's the key" seven decommissioned office dells -> pull every drive first because they came out of a workplace -> the licenses are worth more than the hardware now -> ram prices are at record highs and none of it helps, these are ddr2 and ddr3 2gb sticks -> options for the drives are seven overwrite passes, a drill bit, or a sledgehammer -> he is leaning toward the sledgehammer microsoft charged real money for every one of those keys and the machines they were printed on are now worth less than the cost of hauling them away, while the boards that actually run local models are liquidated bitcoin hardware and the models themselves are free downloads, which is the whole inversion in one driveway the honest part: none of these seven boxes will run a model, ddr2 has nowhere near the bandwidth and the cpus predate every instruction set that matters, so the correct move really is the scrapyard even with ram this expensive 7 decommissioned dells + ddr2 and ddr3 2gb sticks + windows 7 pro and office 2016 keys + seven overwrite passes or a sledgehammer watch and save it, then stop renting the agent, elon put the same harness on github for free and the full $400 to $25 build is in the article below

OPENAI AND ANTHROPIC BOTH SHIP A SYSTEM PROMPT ON EVERY $200/MO SEAT AND NEITHER LETS YOU READ IT. HE OPENED THE BOX ON HIS OWN HARDWARE AND MADE THE MODEL ANSWER IN ELVISH 01:22 he sets the prompt and the model just complies, "but how is it authentic elvish, we will do the exact same prompt again" open lm studio, pick a model, open the panel on the right -> type the system prompt yourself -> "you are qwen 3.5, you respond entirely in old english, helpful, a bit cheeky, supremely intelligent" -> save it as a preset -> ask for a two paragraph mother's day story -> "hark, here in lies the tale of two souls" all the way through -> swap the prompt to elvish -> it answers in quenya and translates underneath for the mortal ear every hosted ai has one of these running before your message lands and none of them let you see it, they call it a trade secret, whole communities exist to extract them by prompt hacking, and locally it is a text box on the right side of the window that you fill in yourself the honest part: lm studio has no memory between chats, only inside a single thread, so if you want continuity you run it as a backend and build your own front end, or wire it to write into an obsidian vault and read back, and he says memory is the part he has sunk the most time into and it still makes silly mistakes lm studio + qwen 3.5 + a system prompt you wrote + saved presets + developer mode as a backend + obsidian for memory watch and save it, then stop renting the agent, elon put the same harness on github for free and the full $400 to $25 build is in the article below

ANTHROPIC BILLS $200/MO AND OPENAI BILLS $200/MO. THIS FANLESS BOARD UPDATED ITS ENTIRE OS IN 20 SECONDS, IDLES AT 86 DEGREES AND HOSTS THE SAME STACK FOR $3 IN ELECTRICITY 00:34 the whole update is over before he finishes the sentence, "well that was quick and easy, probably 20 seconds later it said it's done and restarting device" 86 degrees. 20 seconds. $3 a month. open zimaos settings, 1.5 sitting on 1.61 -> click update -> 40% downloaded before he can narrate it -> done, device restarting, refresh the page, logged back in -> btop is already installed, cpu, memory, disk, network, processes, 33 color themes -> two 2.5 gigabit ports, one still unused -> no fans, no cooling fight, no license synology sells you a box with an os you cannot leave, unraid charges for a key, and this thing ships a full web ui with a linux system monitor baked in and updates itself in less time than it takes to read a changelog, which is the entire argument for owning the always-on node instead of renting one the honest part: a fanless board at 86 degrees is not running a 70b model, this is the machine that stays awake and holds your files, the inference lives on whatever card you point it at zimaboard 2 + zimaos 1.61 + btop + 2x 2.5gbe + fanless at 86f + no license key and no subscription watch and save it, then stop renting the agent, elon put the same harness on github for free and the full $400 to $25 build is in the article below

GOOGLE CHARGES FOR GEMINI IN THE CLOUD AND GAVE AWAY GEMMA 4 FOR FREE. IT RUNS ON A MINI PC WITH NO GPU AT ALL, AN N97 AND 16GB OF RAM, AND IT STILL LOOKED AT A PHOTO AND FOUND TWO KITTENS AND YELLOW WILDFLOWERS here is the whole setup, download to answer: LM Studio, a 600MB installer -> it checks your hardware and recommends a model that will actually fit -> gemma 4 E4B, 8B parameters, just over 6GB -> load it, start a chat -> text works, vision works, all on the cpu of a tiny windows box. and here is the honest part. it did 3.6 tokens a second and spent 41.37 seconds thinking before it said hello. that is slow. it is also a machine with no graphics card doing multimodal AI that did not exist on consumer hardware two years ago. this is what the floor of local AI looks like right now, and it is five things: no gpu is no longer a no -> a low power N97 with 16GB of shared ram runs an 8B multimodal model -> not fast, but real, and the floor keeps rising every release turn thinking off before you judge the speed -> 41.37 seconds of that reply was reasoning you can switch off with one toggle -> most people test with thinking on, see a minute of waiting, and decide local is unusable let the app pick the model -> LM Studio reads your hardware and recommends what will fit instead of letting you download something that crashes -> this one feature removes the most common beginner failure vision is the surprise -> drag in a photo, get a description, no upload anywhere -> for sorting, tagging and describing your own images that is already enough it can serve the rest of your house -> developer mode exposes it on your network so other programs can call it like an api -> one small box becomes the model endpoint for everything else you run google sells gemini by subscription and ships gemma for nothing. the free one now runs on hardware that costs less than a few months of the paid one. my take, and it is the uncomfortable one: the question stopped being whether local AI runs on normal hardware, it does, and started being which jobs are fine at 3.6 tokens a second, which turns out to be more of them than people assume. read the article below. that's how you start with what you already own.

RING AND NEST SELL YOU THE CAMERA, THEN CHARGE MONTHLY FOR THE PART THAT REMEMBERS. HIS CHEAP WIFI CAMERAS WENT FURTHER: WITH THE SUBSCRIPTION CANCELLED THEY KEPT UPLOADING CLIPS TO A SERVER HE HAS NO ACCESS TO here is the whole problem, doorbell to server: cheap wifi cameras everywhere, doorbell, driveway, floodlights, indoors -> without a plan they give you a live view and one snapshot every 5 minutes, and that's it -> meanwhile they keep phoning home and uploading footage he can't see -> and they congest the home network doing it. so the subscription was never paying for the upload. the upload happens either way. you pay for the right to look at your own front door. the replacement, and why it is five things: poe, not wifi -> reolink cameras, one ethernet cable each for power and data, a poe switch in the rack -> slower to install, and the only version where the camera has nowhere else to send anything storage you can walk up to -> every clip lands on the nas in the closet -> if you can't physically unplug the drive holding your footage, it isn't your footage the AI part stays in the house -> a local model reads the clips and writes alerts that describe what happened, "person at the door with a package", not just "motion detected" -> that is the exact feature the monthly plans sell, running on a box that never leaves the network searchable, not scrollable -> every event goes into a database, so "every time a car pulled in this week" is a query, not an evening of scrubbing -> this is the feature that used to justify the cloud, and it is local now higher upfront, zero monthly -> reolink costs more at checkout and the build quality shows -> after that there is no bill, and no company deciding what your cameras do next year amazon and google turned the camera into a storefront for a subscription. the hardware was the cheap part on purpose. my take, and it is the uncomfortable one: a camera that uploads your house to a server you can't open is not a security product, and a local model now does the one thing people were paying the cloud for. read the article below. that's how you keep the smart part and cut the cord.

OPENAI PUTS SORA BEHIND A $200 PLAN AND GOOGLE PUTS VEO BEHIND AN EVEN BIGGER ONE. HE GENERATED FOUR FULL 20 SECOND VIDEOS AT HOME ON TWO INTEL CARDS, AND A 2 SECOND CLIP WENT FROM 26 MINUTES TO 46.5 SECONDS here is the whole pipeline, prompt to finished clip: flux 2 or z-image makes the stills -> wan 2.2 turns each still into a 2 second, 49 frame clip -> clips get stitched into themed 20 second pieces -> flowers, tokyo, lava, a pixar-style one -> all of it rendered on one workstation with two intel arc pro b70s, nothing sent anywhere. the first outputs were unusable, warped and smeared. the same hardware, after tuning, went from 26 minutes a clip to 46.5 seconds, and the quality jumped with it. nothing new was bought between those two numbers. this is where video generation actually is right now, and it is five things: the stack is open and it is good enough -> flux 2 for images, wan 2.2 for motion, both free to download -> the gap to sora and veo is real, but it is a gap in polish, not a gap in whether it works the first run lies to you -> terrible output and a 26 minute render is what a default setup looks like -> almost everyone who tried local video quit here, one config away from 46.5 seconds image first, then video -> generating a strong still and animating it beats asking a video model to invent everything -> this two-step shape is the whole trick behind consistent-looking clips short clips, stitched -> 2 seconds at 49 frames is what fits comfortably, and 20 second pieces are built from those -> nobody is rendering a minute in one go on consumer cards, and the stitch is where the storytelling happens vram decides what you can run, not brand -> two intel cards at 32GB each hold models that a single gaming card cannot -> nvidia is not required for this, and that alone surprises most people openai and google charge a monthly plan for the right to generate video on their servers, with their limits and their filters. this runs on a desk, with no queue and no cap. my take, and it is the uncomfortable one: local video generation is about one year behind the paid tools and closing fast, and the people paying $200 a month mostly never saw what a tuned local setup produces. read the article below. that's how you get from 26 minutes to 46 seconds.

NVIDIA WANTS $1,600 FOR A 16GB CARD. THIS $80 BOARD HAS 16GB TOO, AND A CHUNK OF IT STAYS LOCKED UNTIL YOU TYPE TWO COMMANDS AND CHANGE ONE BIOS NUMBER TO 512 here is the whole unlock, usb stick to usable memory: pull the nvme -> boot a usb stick -> type blk0: then flash.nsh -> firmware P3.00 writes block by block, don't lose power -> clear cmos, date resets to 01/01/2021, that's how you know it took -> set the integrated graphics to forced and the UMA frame buffer to 512MB -> F10, done. that last setting is the whole video. the board shares one 16GB pool between cpu and gpu, and a big fixed carve-out wastes it. a small frame buffer lets memory get handed to the model on demand instead of sitting reserved for a display nobody is looking at. this is the part nobody puts in the listing, and it is five things: the spec sheet number is not the usable number -> 16GB printed, and how much a model can actually grab depends on a bios field most people never open -> "it only fits a small model" is often a setting, not a limit firmware is a performance setting, not maintenance -> an old revision on the same silicon behaves differently, and a fleet on mixed firmware benchmarks like a different product -> flash first, test second, always clear cmos means clear it properly -> jumper, battery out, hit power ten times, wait -> overboard on paper, and it is the step that saves you from chasing ghosts for a day one psu can feed two nodes -> a 500W supply runs both boards, so node two costs a board and a cable, not a second build -> this is how a cluster grows one $80 step at a time document the two commands -> blk0: and flash.nsh is the entire procedure, and nobody remembers it three months later -> write it on the usb stick itself nvidia sells 16GB with the memory already unlocked and charges for that convenience. here it costs one usb stick and knowing which number to type. my take, and it is the uncomfortable one: half the "this hardware is too weak for AI" verdicts are made on factory bios defaults, and the fix is a menu nobody scrolls down to. read the article below. that's how you get the memory you already paid for.

AMAZON RING AND GOOGLE NEST BOTH CHARGE A MONTHLY SUBSCRIPTION TO LET YOUR CAMERAS REMEMBER WHAT THEY SAW. HE PULLED EVERY ONE OF THEM, WIRED REOLINK POE CAMERAS INTO A NAS, AND PUT AN AI ON TOP THAT SEARCHES THE FOOTAGE FOR FREE here is the whole swap, cloud to closet: doorbell, floodlights and indoor wifi cams, all tied to a monthly plan -> reolink poe cameras bought during black friday -> one poe switch, one box of ethernet, one cable per camera for power and data -> every frame recorded straight to a nas in the rack -> local AI processing on that nas tags what happens in each clip -> a searchable database of your own house, and no monthly bill attached to any of it. the cameras were never the product. the subscription was. without it the cloud cams still give you a live view, but the recording, the history and the "person detected" alerts all sit behind the plan, which is exactly the part that runs perfectly well on a box you already own. this is the same move he already made with photos. google photos became a local immich server years ago, and the security cameras are just the next account to cancel. this is subscription unwinding, and it is five things: the cloud feature is almost always storage plus a small model -> ring and nest charge monthly for keeping clips and labelling them -> a nas stores the clips and a local model labels them, and both are one-time purchases poe removes the reason cameras went wireless -> one ethernet cable carries power and data, so there is no battery, no charging and no dropouts -> wifi cameras are easy to install and that ease is how the subscription gets in searchable video is the killer feature and it is local now -> "show me every time a car pulled into the driveway this week" used to need a cloud plan -> the same query runs against a local database with a model that never leaves the house privacy is not a side benefit, it is the product -> every frame of your front door on someone else's server is a policy decision you did not make -> on a nas in your closet the policy is yours it is work, and he says so -> running cable through a house is the actual cost of this, not the cameras -> a weekend of wiring against years of monthly bills is the honest trade amazon and google do not sell cameras, they sell the right to look back at what your cameras saw. that right costs a nas and a weekend. my take, and it is the uncomfortable one: every smart home subscription is a storage bill with a small AI model attached, and both of those got cheap enough to own outright about two years ago. read the article below. that's how you cancel the next subscription and keep the feature.

NVIDIA'S CARDS TALK TO EACH OTHER OVER A PROPRIETARY BRIDGE THEY DELETED FROM CONSUMER CARDS. TENSTORRENT'S TALK OVER PLAIN 800 GIGABIT ETHERNET, USING THE SAME CABLE ANY DATACENTER ALREADY HAS IN A DRAWER here is the whole path, card to card: two Intel Arc B70s in one machine -> when they need to exchange anything it goes down the PCIe slot, through the chipset, and back up into the other card -> every hop costs you -> two Blackhole P150a instead -> QSFP 800G ports sit on the cards themselves -> they connect directly to each other, over standard ethernet, and the motherboard is never involved. that choice is the whole company. nvidia's interconnect is proprietary, expensive, and removed from anything you can afford. tenstorrent picked an off-the-shelf networking standard, which means the fabric is a cable, not a licence. the second-order effect is the interesting one, and it is five things: standard ethernet means the slot stops mattering -> the cards only need pcie to be fed at startup, after that they talk to each other -> x1 mining risers become viable again, which is how you get density without a server motherboard a cable is a market, a bridge is a moat -> qsfp optics and dacs are a commodity with dozens of suppliers and decades of price pressure -> nvlink is one supplier, one price, and they decide which cards get it 120 capable cores instead of thousands of tiny ones -> a GPU is a convoy of trucks on a straight road, great for uniform bulk work -> this is a switching network, built for work that keeps changing shape and handing off no video out at all is the honest signal -> these were never graphics cards with an AI sticker, they are data movement devices -> the fastest way to understand hardware is to look at what the vendor deliberately left off he refused to benchmark them against each other and that is correct -> a convoy and a rail network do not share a metric, and forcing one makes both look bad -> every "X vs Y" video comparing these is measuring the wrong thing on purpose nvidia sells the fabric as the product and charges $25,000 a card for it. tenstorrent made the fabric a networking cable and open sourced everything above it. my take, and it is the uncomfortable one: nvidia's moat was never the silicon, it was making the interconnect proprietary and then removing it from the cheap tier, and the first serious attack on that is a standard ethernet port. read the article below. that's the difference between buying a chip and renting a fabric.

NVIDIA PUTS THOUSANDS OF TINY CORES ON A CARD. TENSTORRENT PUTS 120 BIG ONES AND AN 800 GIGABIT CABLE BETWEEN THE CARDS. THAT IS THE SAME ARGUMENT AS 40 CHEAP AGENTS VERSUS 5 SMART ONES, AND MOST PEOPLE PICK WRONG here is the whole split, core count to topology: a GPU is thousands of tiny identical calculators on one fast bus, which is a convoy of trucks down a straight road -> a Blackhole P150a has 120 far more capable cores and treats the whole thing as a switching network -> two Intel cards talk to each other by going down the PCIe slot, through the chipset, and back up into the other card -> two Tenstorrent cards talk over QSFP 800G directly, card to card, never touching the motherboard. that is not a spec difference, it is a different opinion about what the work looks like. uniform work wants the convoy. work that keeps changing shape and handing off wants the switching network. you are making the same call every time you build an agent system, and it is five things: many small workers wins when the work is uniform -> forty identical extraction jobs, one model, no coordination, pure throughput -> this is the convoy, and a GPU or a rack of cheap boards is the correct hardware for it few capable workers wins when the work branches -> a planner, a coder, a reviewer and a verifier are not interchangeable, and splitting them into forty pieces makes it worse -> more workers past that point adds coordination cost and subtracts accuracy the expensive part is always the handoff -> Intel cards pay motherboard latency every time they talk, and agents pay context-rebuilding cost every time they hand off -> whoever shortens the handoff wins, which is exactly what an 800G cable on the card is for no video out is the tell -> these are not graphics cards wearing an AI label, they are built for moving data between units -> the clearest signal of what hardware is actually for is what the vendor left off it pick the shape before you pick the model -> forty cheap workers on a bad topology is slower and more expensive than five on a good one -> and five expensive workers on uniform bulk work is the most common way people burn a budget anthropic and openai sell one enormous brain and charge you for its generality. most real workloads are either forty identical small jobs or five that need to talk properly, and neither of those is what a $200 seat is priced for. my take, and it is the uncomfortable one: the core count argument and the agent count argument are the same argument, and the answer is the same in both cases, which is that the topology matters more than the units. read the article below. that's how you pick the shape before you pay for the parts.

ANTHROPIC CHARGES $200 A MONTH SO SOMETHING IS ALWAYS AWAKE FOR YOU. THIS FANLESS BOARD DOES THAT AT ROUGHLY 8 WATTS AND WAKES THE EXPENSIVE MACHINE ONLY WHEN A REQUEST ACTUALLY ARRIVES here is the whole shape, always-on to burst: a Zima Board 2, passively cooled in a machined case, silent by design -> dual 2.5 gigabit ethernet, two SATA ports, a PCIe Gen3 slot on the side -> it sits on 24/7 and costs almost nothing to leave running -> it holds the queue, the router and the API endpoint -> when real work lands it wakes the big rig, hands off, and the big rig sleeps again. the expensive part of local AI is not the tokens, it is the idle. a workstation doing nothing still pulls 40 to 120 watts, and that is the number that turned a $1,920 pallet into a $6,000 power bill in the last video. the always-on box and the compute box should not be the same machine. that is the whole idea, and it is five things: separate the front door from the factory -> a fanless 8W board can hold your endpoint, your queue and your web UI forever -> the GPU machine only needs to exist during the seconds it is actually generating idle watts compound, peak watts do not -> 40W of idle for a year is roughly the same money as a few hundred hours of real inference -> nobody measures idle because it never shows up in a benchmark dual 2.5GbE is the feature that makes this work -> one port to your network, one to the compute box, so wake-on-lan and handoff are clean -> this is the difference between a hobby setup and something the rest of the house can actually use the PCIe slot means it can do small jobs itself -> a low power card in that slot handles embeddings, classification and speech to text without waking anything -> route by size: tiny jobs die here, big jobs get escalated passive cooling is a reliability spec -> no fan means no fan failure, which matters for the one machine that is never supposed to go down -> the loud rig can be loud, it is only awake for minutes a day openai and anthropic sell you a warm machine that is always ready. that readiness is real, and it costs them a fortune at scale. at home it costs about 8 watts. my take, and it is the uncomfortable one: most people who priced local AI compared a sleeping subscription to a rig that never sleeps, and the fix is two machines instead of one. read the article below. that's how you stop paying for idle.

NVIDIA'S REAL MOAT WAS NEVER THE CHIP. TENSTORRENT'S DOCS SAY UBUNTU 22.04, HE FOLLOWED THEM EXACTLY, AND THE CARDS VANISHED OFF THE BUS. THEY CAME UP FIRST TRY ON FEDORA, WHICH IS NOT IN THE DOCS AT ALL here is the whole install, box open to working card: two Blackhole P150a, 32GB each, 1,532 grams, no video out -> detected on first boot, then gone -> reseat, swap slots, single card, other slot, bios, new PSU -> support responded fast and could not explain it either -> moved one card to a different machine running Fedora -> up immediately, LLMs and image gen and speech to text all running. nothing about that sequence is a silicon problem. every hour of it was drivers, kernel versions and a motherboard that did not agree with a card. that is the entire reason cuda is worth more than any chip nvidia has ever taped out. this is the part of the AI hardware story nobody prices, and it is five things: the supported distro list is the product spec -> if the docs say 22.04 and you are on 26, you are the integration team now -> and if it then works on a distro that is not listed, the docs are describing what they tested, not what works every non-nvidia card costs you a week you did not budget -> amd users have been saying this about rocm for years and it is the same story -> the chip being good is necessary and nowhere near sufficient vendor support quality is a real spec -> they answered fast, engaged properly and still could not solve it, which is honest and also tells you where the platform is -> compare that to opening a cuda issue and finding the answer from 2019 they open sourced the whole stack and that is the long play -> you can read every layer, which you cannot do with cuda -> it matters in year three, not in week one, and week one is where most people quit two cards is a dev kit, not a deployment -> the product is the fabric, and the fabric means 32 of them in a Galaxy Box -> for one person who wants an LLM running tonight, this is the wrong purchase and he says so plainly nvidia spent fifteen years making the software boring. that is the moat, and it is why a better architecture still loses two days to a bus that stopped seeing the card. my take, and it is the uncomfortable one: everyone benchmarks silicon and nobody benchmarks the install, and the install is where every nvidia competitor has died for a decade. read the article below. that's the cost nobody puts in the comparison.

EVERY FRONTIER MODEL SHIPPED THIS YEAR IS MIXTURE-OF-EXPERTS, WHICH MEANS ROUGHLY 92% OF THE PARAMETERS SIT IDLE ON EVERY SINGLE TOKEN. NVIDIA'S ENTIRE ARCHITECTURE IS BUILT FOR THE OTHER 8% here is the whole bet, chip layout to model shape: a GPU is thousands of identical tiny calculators on one high speed conveyor -> that is perfect for a dense model where everything moves through the same pipeline -> a Tenstorrent Blackhole is not that, it is separate buildings connected by a fabric, built around moving data rather than crunching it -> and the models everyone ships now are sparse, so most of the factory is idle while a few rooms do the work. gpt-oss 120B activates about 10B parameters per token. the other 110B are loaded, powered and doing nothing. on a monolithic card that waste is invisible because it is the only shape the card knows. this is an architecture bet, and it is five things worth understanding: dense models made GPUs king for a reason -> everything in a dense model flows through one pipeline, and one enormous fast factory is the correct answer to that -> nobody is claiming GPUs are bad, they are claiming the workload changed underneath them sparse models reward data movement, not raw math -> when only a fraction of the network fires per token, the expensive operation becomes routing to the right expert -> that is a traffic problem, and traffic is what Tenstorrent designed for multimodal makes it worse for the monolith -> text, vision, audio and image generation inside one model is not one factory, it is four wings that have to talk -> separate units with a fast fabric between them starts looking elegant instead of exotic agents push in the same direction -> a model that watches a video has to do audio, language, subtitles and vision in one pass -> more distinct kinds of processing in one request means more routing and less uniform crunching the bet only pays if models keep getting sparser -> if the industry swings back to dense, GPUs keep winning outright and none of this matters -> every frontier release for the past two years has gone the other way nvidia sells the best possible answer to the 2022 question. the 2026 question is where the data goes, and it is the first time in a decade that a non-GPU company has a real answer to it. my take, and it is the uncomfortable one: people are buying hardware optimised for dense models to run models that are 92% idle, and the only reason it still works is that nvidia is fast enough to waste that much and still win. read the article below. that's the difference between buying compute and buying a fabric.

NVIDIA DELETED NVLINK FROM CONSUMER CARDS. JIM KELLER BUILT A COMPANY WHERE THE INTERCONNECT IS THE ENTIRE PRODUCT, OPEN SOURCED THE WHOLE STACK, AND THE CARDS STILL DROPPED OFF THE BUS AFTER TWO DAYS here is the whole install, unboxing to working: two Tenstorrent Blackhole P150a, 32GB each, no video out because they are not GPUs -> detected on first boot, then detection vanished -> reseat, swap slots, single card, bios, nothing -> moved one card to a different machine on Fedora and it came up immediately -> LLMs, image gen and speech to text all running on it now. so the pitch is not "faster than a GPU". the pitch is "your vram can actually be one pool again", and the entire software stack is open source while nvidia's is not. this is where they are honest and most vendors are not, and it is five things: the interconnect is the product, not the chip -> two cards is a dev kit, 32 in a Galaxy Box is the actual machine -> buying one or two to run a big model is buying the wrong half of the idea the stack is immature and they say so out loud -> the documentation targets Ubuntu 22.04, he was on 26, and that mismatch is a real day of your life -> support answered fast and could not explain the dropout either, which is what a developer platform actually looks like open source is the long bet -> you can read every layer of their stack, which is not true of cuda, and that matters more in year three than in week one -> people underrate this until the first time a driver blob decides their card is obsolete future architectures are the reason to watch -> as models move past plain mixture-of-experts into shapes that need constant cross-chip traffic, on-card meshing stops being a nice feature and becomes the requirement -> right now, for one person who just wants an LLM running tonight, it is not the purchase nvidia charges $25,000 for a card because pooled memory is the product. tenstorrent put the pooling port on a card you can actually buy and open sourced everything around it. my take, and it is the uncomfortable one: the company most likely to break nvidia's grip is not shipping a faster chip, it is shipping the connector nvidia deleted, and it will take two more years of driver work before anyone notices. read the article below. that's the difference between buying a chip and buying a fabric.

NVIDIA'S LAPTOP 4080 DOES THE SAME WORK AS A DESKTOP 3080 Ti AT A THIRD OF THE POWER. THAT RATIO IS THE DIFFERENCE BETWEEN A $6,000 ELECTRICITY BILL AND TWO SOLAR PANELS here is the whole swap, desktop to solar: a desktop 3080 Ti finished one work unit off a single EcoFlow River 2 battery -> the same job moved to a laptop with a 4080 -> laptops are engineered for efficiency by default, so the whole machine pulls 121W under full load and 40W at idle -> two to three times more efficient on identical work -> solar kicks in at 97% battery and the runtime estimate jumps from one unit to nearly four hours -> three work units on a cloudy day, and two more panels arriving. that ratio is the entire story and almost nobody measures it. everyone benchmarks tokens per second. nobody benchmarks tokens per watt, and the second number is the one that decides whether local AI is cheaper than a subscription or four times more expensive. this is efficiency engineering, the layer nobody benchmarks, and it is five things: mobile silicon is binned for watts, desktop silicon is binned for peaks -> a laptop 4080 is tuned to hit a thermal and power budget, which is exactly the constraint a 24/7 inference box has -> the desktop card wins a benchmark and loses a year measure the wall, not the card -> the GPU reported one number, the meter on the battery reported the whole machine, and they are not close -> screens, RGB, fans and chipset are real watts, and closing the lid turned several of them off airflow is worth more than a bios tweak -> lifting the laptop off a tablecloth moved throughput up immediately -> that is free performance sitting under every machine on a desk right now throttled mode is often the efficient mode -> full 175W against a throttled profile is the comparison worth running, because peak watts rarely buy proportional work -> most of these rigs sit at the flat part of the curve and pay for the steep part once the panel covers the draw, the marginal cost is zero -> a 121W machine is inside the range of a couple of panels and a battery, and at that point the electricity argument disappears entirely -> this is the missing half of every local-versus-cloud comparison anthropic and openai charge $200 a month and their power bill is invisible to you. the moment yours is covered by sunlight, that part of their price stops being a real advantage. my take, and it is the uncomfortable one: the local AI debate keeps getting settled on hardware price when it is actually settled on watts, and a laptop chip at 121W beats a pallet of cheap boards on the only metric that compounds. read the article below. that's how you price the watts before the hardware.

GLM 5.1 AND MINIMAX 2.5 BOTH FIT ON A $1,920 PALLET OF DEAD CRYPTO BOARDS. THEY RUN AT ONE TOKEN PER SECOND, AND THE ELECTRICITY COMES TO $6,000 A YEAR here is the whole wall, first node to giving up: one BC-250 with 16GB, about 14.5 usable once the server is up -> second node added, just under 30GB combined, pipeline parallelism works -> third and fourth node and the returns are already collapsing -> at 12 or 24 nodes the model loads and produces roughly one token per second -> memory bandwidth is the constraint, not compute, not capacity -> and at 50% uptime across a year the power bill lands at five to six thousand dollars. that is the number nobody posts. the boards are $80. the vram is nearly free. the electricity costs more than the two subscriptions he was trying to cancel. pipeline parallelism over slow links means every token walks through every node in sequence. adding nodes adds capacity and adds latency at the same time, and past three or four the latency wins. this is the arithmetic nobody runs before buying a pallet, and it is five numbers: capacity and speed are different purchases -> 24 nodes holds enough vram for GLM 5.1 or Minimax 2.5 and delivers one token per second of it -> a model that technically fits and practically crawls is a screenshot, not a setup mixture-of-experts is the only shape that survives this -> a 120B model with 10B active per token reads far less memory per step, which is exactly what a bandwidth-limited rig needs -> dense models on a slow fabric are hopeless, MoE on the same fabric is usable, and architecture matters more than node count here power is the real bill and it arrives monthly -> $1,920 of boards once, then five to six thousand a year to keep them powered -> one efficient modern GPU does the same work for a fraction of that, and the pallet price stops mattering by month four a few of these is a great deal and a lot of them is not -> two or three self-contained nodes, each holding its own model, is genuinely cheap capability -> the curve does not extend, it falls off a cliff, and the cliff sits around node four know which number you are optimising -> cheapest vram: the pallet wins outright -> cheapest tokens: it loses badly, and tokens are the thing you actually wanted anthropic and openai charge $200 a month each and their electricity is somebody else's problem. that is a real part of what the price buys. my take, and it is the uncomfortable one: the local AI pitch is always priced on hardware and never on watts, and once you add the power bill the honest answer is a few nodes at home and a subscription for the rest. read the article below. that's the arithmetic before you buy the pallet.

ANTHROPIC AND OPENAI CHARGE $200 A MONTH AND YOU NEVER SEE A FAN, A DUCT OR A FIRMWARE FLASH. HERE IS WHAT $1,920 OF DEAD MINING BOARDS ACTUALLY COSTS YOU: 24 BOARDS, THREE AT A TIME, EIGHT BIOS ROUNDS BEFORE A SINGLE TOKEN EXISTS here is the whole gap, pallet arrives to cluster online: 24 boards at $80 each -> flash firmware three at a time, eight rounds -> a fan and a printed duct keep them alive -> then the OS and the tooling -> and the chassis carries power and air only, no data between nodes. read that last part again. the rack is not a cluster. it is 24 independent computers sharing a power bus and a fan. every node runs its own model and answers its own queue, and nothing coordinates them except code you write yourself. this is the part missing from every "just run it locally" reply on this website. the hardware is cheap and the provisioning is not, and the subscription you pay is mostly for someone else having already done it. this is provisioning engineering, the layer between buying hardware and producing a token, and it is five things nobody budgets for: firmware is a performance setting, not a formality -> he found boards shipping on different revisions inside the same lot, and the newer one behaves differently -> a fleet on mixed firmware produces inconsistent benchmarks and you will chase that for a week cooling decides how many you can touch at once -> three at a time, because that is what one ducted fan covers during a flash -> density is not a rack problem, it is an airflow problem, and it caps every maintenance task you will ever run power and data are separate purchases -> the 12-slot chassis delivers electricity and air and nothing else -> people see a rack and assume a cluster, then find out every node is an island, which is perfect for many small models and useless for one large one count your maintenance passes before you buy the lot -> 24 boards at three per pass is eight cycles of flash, verify, swap -> multiply that by every future firmware update, because you own that now the tooling is the real asset -> whatever you write to image, configure and dispatch across 24 nodes is worth more than the boards themselves -> it outlives the hardware, and it is the one thing that does not arrive on a pallet anthropic and openai charge $200 a month and you never see a fan, a duct, a firmware revision or a flash cycle. that is the honest description of the product. my take, and it is the uncomfortable one: local AI is not expensive, it is laborious, and the subscription is a labour contract that everyone keeps describing as a compute bill. read the article below. that's the work between a cheap pallet and a working cluster.

THE SAME $80 BOARD WENT FROM 25 TOKENS A SECOND TO 175 WITHOUT A SINGLE HARDWARE CHANGE. THAT 6.8x IS MOST OF WHAT SEPARATES YOUR LOCAL SETUP FROM THE $200 A MONTH YOU SEND FABLE 5.1 AND GPT ASTRA here is the whole climb, stock install to serving speed: ollama 0.19 on a BC-250: 25.67 tokens per second -> llama.cpp lands an update: 37 -> governor clock unlock, a community trick that raises ram and cpu clocks: 54.99 -> batched serving with multiple concurrent requests: 174.95 on one node -> same board, same silicon, same power envelope, 6.8x. the last jump is the one nobody at home makes. 54.99 is what you get sending one request at a time. 174.95 is what you get when several are in flight together, because the weights are already loaded and the bottleneck is reading them, not thinking about them. that is not a trick. that is exactly how every API provider runs. they batch your request with strangers' requests against one resident copy of the model, and the difference between their cost and your bill is largely that number. this is serving engineering, the layer between a model that works and a model that is fast, and it is five moves that cost nothing: batch before you buy anything -> one request at a time leaves roughly two thirds of your throughput on the floor -> if you have agents, a household, or any queue at all, you are already the multi-user case and just haven't served it that way update the runtime before you update the hardware -> llama.cpp alone took this board from 25.67 to 37, which is 44% for a git pull -> people benchmark once, decide local is slow, and never rerun after the next release unlock the governor -> the single biggest jump here came from raising ram and cpu clocks, not from any model change -> stock power profiles are tuned for the workload the hardware shipped for, and yours is not that workload fork the runtime if your hardware is unusual -> he maintains a llama.cpp fork specifically for the BC-250, because generic builds leave real performance behind on odd silicon -> this is the part that separates a benchmark from a machine that actually serves then scale by node, not by card -> 24 of these at 175 tokens a second each is roughly 4,200 tokens per second for a rack of dead mining boards -> no interconnect needed, because each node holds a full model and answers its own queue openai and anthropic sell batched throughput on resident weights and call it a subscription. that is the actual product, and a $80 board with the right flags does 175 tokens a second of it. my take, and it is the uncomfortable one: almost everyone who tested local AI measured a single-user, stock-clock, outdated-runtime configuration and then compared it to a provider running every optimisation in this list, which is not a comparison, it is a default. read the article below. that's how you get the other 6.8x before you spend a dollar.