CEO @CUDO_Compute, Passionate about creating innovative technology to improve the world with sustainable computing.
UK
Joined August 2009
- Tweets65
- Following949
- Followers2.4K
- Likes790
Matt Hawkins retweeted
Compute capacity gets the headlines. Power availability is what actually decides who builds, where, and how fast.
Power availability and grid capacity now rank as the leading factors in AI infrastructure location decisions.
But having power on paper isn't enough.
A site needs the capacity to support the workload, a realistic route to getting that power online and the resilience to keep it running. Land without a clear power pathway is an option, not an asset.
As @WSP’s Anushka Devaser puts it:
“The biggest misunderstanding is assuming securing power is only about price or MW capacity.”
AI infrastructure needs more than an available MW figure. It needs firm, resilient power that can support the workload in practice.
That’s why land, power and compute need to be considered together. Decisions around one quickly affect what’s possible with the others.
Read our Land. Power. Compute. research below.
Matt Hawkins retweeted
21.3% → 55.2% MFU.
That’s more than a 2× difference in useful work on comparable-era hardware.
GPT-3 175B achieved 21.3% MFU. MegaScale reached 55.2%.
The difference is not just the GPU. It’s the system around it.
DeepSeek-V3 is a useful example.
It ran on 2,048 NVIDIA H800 GPUs, hardware built on the same Hopper architecture as the H100 but with NVLink bandwidth reduced from 900 GB/s to 400 GB/s for export compliance.
That single difference in interconnect reshaped the entire training strategy.
The team avoided tensor parallelism because it was inefficient under the constrained NVLink, built a bidirectional pipeline scheme to overlap computation and communication, and relied on eight 400G InfiniBand NICs per node to compensate for the loss of intra-node bandwidth.
The chip was nearly the same. The configuration determined what the cluster could actually do.
Even after that hardware-aware co-design, third-party analysis estimates the run landed at around 23% model FLOPs utilization, a figure the DeepSeek team never had to disclose and one that rarely appears in headline infrastructure specifications.
GPU model and count only tell part of the story. What matters is how effectively the entire system turns that hardware into useful compute.
Read the article below.
Model choice will keep changing.
The need for reliable land, power and compute will not.
Taking control of an AI model is getting easier. Running it well is not.
Look at the last four weeks.
Meituan open-sourced LongCat-2.0 (1.6T params, ~48B active).
Moonshot shipped Kimi K3 (2.8T params) this week, the full weights went public. Largest open model ever released.
Moonshot paused new subscriptions within 48 hours of launch. Not for lack of demand, for lack of compute to serve it.
Now anyone can download K3. Moonshot's own guidance: run it on a cluster of 64+ accelerators.
Once a workflow becomes core, the questions change:
What does each output cost?
Where does the data go?
Can we guarantee capacity?
Can we switch models without rebuilding?
Open ≠ cheaper. Utilisation and system design decide the economics. Self-hosting wins when a workload runs hot and steady. Otherwise a managed API usually wins.
Either way, someone still has to secure the compute and run it under load.
Capital markets noticed. @Togethercompute just raised $800M at an $8.3B valuation.
Open models don't remove the need for AI infrastructure. They create more buyers for it.
At @Cudo_Compute we build for both sides. Start on demand, prove the use case, move to dedicated or managed capacity as usage gets predictable.
Model choice will keep changing. The need for reliable land, power and compute will not.
If you’re looking to run multilingual LLM’s, @StabilityAI have the highest performing small footprint LLMs. Great for running at very low cost on GPU cloud. How long till someone has this running on your pair of sunglasses when abroad 😎
Remarkable to see Claude 3 Opus not just finding the ‘needle’, but also questioning the ‘haystack’ itself! 🧠 This meta-awareness points towards a new horizon in AI, where machines aren’t just tools but participants in problem-solving.
What could this mean for the future of AI-driven research and do you find this scary ? #MachineLearning #AI #Innovation
Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.
For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.
When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.
Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:
Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.
Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.
This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.
One of the next exciting areas in media for GPU power and #AI is lip-syncing. Alibaba researchers unveiled EMO, an AI that adds lip-syncing to videos. Just give it a still image and any audio, incredible! These aren’t just lip-syncing they are face syncing!
Amazing news that the industry as a whole is moving to #carbonneutrality .One step at a time, we should get to the stage where all blockchains and crypto can be carbon neutral.
Yesterday I was pleased to host a meeting between @elonmusk & the leading Bitcoin miners in North America. The miners have agreed to form the Bitcoin Mining Council to promote energy usage transparency & accelerate sustainability initiatives worldwide.
Could you fit one of these under your desk? #IBM’s 50 Qubit quantum computer shown off at #CES2020 #IBMQuantum still a long way off #QuantumComputing effecting crypto but it has great potential for shared computing
Working with @AMD in Toronto, let’s make better use of people underused hardware for #CloudComputing
It's great to see how the #sharing #economy continues to #develop, harnessing unused #computing power to further medical research and other revolutionary breakthroughs
The DreamLab app which harnesses the unused processing power of idling mobile phones has been used to identify anti-cancer molecules in a range of foods. @ImperialSandC research in partnership with @VodafoneFdn | via @BBCTech bbc.co.uk/news/technology-48…
Great to see #Bournemouth get coverage for growing local #tech companies, there are so many great companies in the South West making a global impact
It would be incredible to see the UK follow suit in plastic recycling machines for credit on transport facebook.com/worldeconomicfo… #Recycle #plasticwaste
Although technologies are around longer, the core time periods you see below are similar to the major technology timelines we saw with the internet and VoIP #blockchaintechnology
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more
Proud to be talking at StartUpU this Saturday @thisworkspace , if you want to hear @pierslinney from Dragons Den, myself and many other great entrepreneurs speaking about how to build a business come on down. eventbrite.co.uk/e/startup-s… #screwitjustdoit #Entrepreneur #startup
