I listened to a call on AI infrastructure trends with @AlanWeckel, Co-Founder of the 650 Group (market research firm) – very smart / thoughtful guy with a solid handle on the data center, communications, and information technology markets. Some thoughts below, with a few personal notes on Nvidia at the bottom... Hyper scalers and large cloud players $MSFT $GOOGL $MSFT $AMZN $META hinting at increasing capex in 2025. Q1 saw 30% growth in capex from hyper scalers. Projecting ~25% growth this year and roughly the same for 2025 (huge growth in % and absolute $ terms). Capex more than doubles over next 5 years – weighted toward GPU and AI ASICs. Ties to what Jensen has been saying. Apple $AAPL starting to spend more on servers. Tier 2 cloud providers (think CoreWeave & Lambda Labs) and sovereign AI will continue to have larger influence on market. CoreWeave recently raised ~$7bn, which is basically the same as what Apple and Oracle $ORCL are spending. To be clear, all the tier 2 cloud providers have very little chance of reaching top tier hyper scalers – the numbers are just that big. Interesting sovereign AI markets to keep an eye on: Japan, South Korea, and wealthy Middle Eastern countries. There’s hope in the market that non-AI spend will recover this year, but that probably won’t happen. Non-AI server spend will probably stabilize around current levels and resources will be re-allocated elsewhere. Spend will be related to hunting power / limiting power consumption vs. participating in Intel $INTC refresh cycles. AI perceived as too important not to spend on. Like I said above, non-AI server spend is expected to be flat through 2028 vs. AI/ML server spend more than doubling. Could see workloads move to other continents where power is more readily available. Like how most of Japan’s data center’s sit in Pacific Northwest. On the power point, liquid cooling $VRT reduces power consumption. Cold plate / direct to chip cooling is the technology for current data centers. Immersion is most efficient but still in test phase, while direct is an older technology and ease of use gives it an instantaneous advantage. Cold plate gets to ~40% power savings just by focusing on high heat parts of the server. Cooling makes up larger % of AI/ML server rack (50-100kW per rack) spend vs. traditional x86 server rack (5-15kW per rack). Power consumption is big wall out there in terms of being able to deploy AI going forward. Won’t be much delay in terms of spending with Blackwell $NVDA generation. These are relatively small volumes in units relative to what’s already in data centers. This is inefficient but facilities can support, for now…but as we move forward, it becomes a huge constraint – not a single data center out there today is built for AI. These will start coming in 2025 and 2026 – Jensen refers to these as “AI factories.” Industry did terrible job – stopped planning data centers during pandemic and is now paying the price. Again, right now getting away with it with existing infrastructure given AI relatively small number of units. This is what Jensen refers to when he talks about the massive transition from traditional to accelerated compute infrastructure. Spend on AI focused semis went parabolic in late 2022 / early 2023…AI % of data center semi spend went from <20% in early 2018 to >70% in late 2023. Expect a lot of hyper scale vertical integration with Nvidia, which is favored in high end training. People are extremely enthusiastic about Blackwell and Blackwell will be supply constrained. But in a couple of years, the move toward inference could create a more disaggregated ecosystem. Inference will need purpose built lower power ASICs $MRVL $AVGO – the right ASICs for the right workload. Certainly, a risk to Nvidia’s dominance, but I don’t think we are there yet. Also depends on how embedded Nvidia systems / software become and how they plan to attack the inference market. As well as what ongoing training requirements look like. We’ll see. Marvell has great traction with a hyper scaler and imagine they will continue to gain traction with others. Doesn’t mean Broadcom won’t be competitive. AMD will be favored as the second merchant chip behind Nvidia, but still needs to prove itself with MI300, etc. Won’t throw AMD into a high-end Llama model or Chat GPT anytime soon. As we move beyond Blackwell, all the ODMs will be itching for a piece of the enterprise market. All won’t be successfully, but those that are will be competing with Super Micro $SMCI and $DELL. Networking about to take off, which is a good trend for Arista Networks $ANET given they over-index on AI. AI networking market will grow from $8.6bn in 2023 to $24.9bn in 2028. Ethernet and InfiniBand will both be relevant. Astera Labs $ALAB market will get a lot more competitive. Broadcom missed the re timer market and they weren’t happy about it. A few separate thoughts on Nvidia… Worries about a quarterly deceleration ahead of Blackwell launch (customers holding off on buying more Hopper chips because they are waiting for the next gen architecture). July quarter could be first quarter in a while with <10% sequential sales growth. Very excited about Grace Blackwell (GB200). Best ROI from a compute, power, and space perspective. Bank of America thinks every high-end Blackwell rack could be ~$3mm vs. the ~$25-300k for a high-end hopper rack. Opportunity to gain share. China H20 shipments should benefit back half but raises restriction risks. Last quarter, the market was excited about the ~40% inference mix metric – not sure if we’ll get an update here. Gross margins should start to normalize in the mid-70s vs. 6 straight quarters of expansion to ~77% (trough was ~56%). Could be from higher Blackwell cost or more aggressive pricing vs. competitors. As I’ve said, the magnitude of price increase from Ampere to Hopper (about double) is much larger than what we will see from Hopper to Blackwell. I continue to own this stock and try not to play the expectations game (expectations are high), but it sounds like investors are looking for ~$26bn sales in Q1 (street ~$24.6bn) and ~$28bn guide for Q2 (street ~$26.7bn). Looks like the street is ~$31 EPS for CY 2025, which implies low-mid 20s earnings growth. At $31, the stock is trading just <30x earnings, which doesn’t feel unreasonable for the most innovative company leading one of the most important investing and technology trends of our generation. Using more aggressive assumptions, I can get mid-30s earnings power in CY 2025, which implies ~26x earnings.