seikixtc retweeted
Turns out you can run enormous Mixture-of-Experts on Mac hardware without fitting the whole model in RAM by streaming a subset of expert weights from SSD for each generated token - and people keep finding ways to run bigger models
Kimi 2.5 is 1T, but only 32B active so fits 96GB
I got a 1T-parameter model running locally on my MacBook Pro.
LLM: Kimi K2.5
1,026,408,232,448 params (~1.026T)
Hardware: M2 Max MacBook Pro (2023) w/ 96GB unified memory
Running on MLX with a flash-style SSD streaming path + local patching.
This is an experimental setup and I haven’t optimized speed yet, but it’s stable enough that I’ve started testing it in an autoresearch-style loop.
#LocalAI #MLX #MoE
seikixtc retweeted
More progress.
I got a 1T-parameter model running locally on my MacBook Pro.
LLM: Kimi K2.5
1,026,408,232,448 params (~1.026T)
Hardware: M2 Max MacBook Pro (2023) w/ 96GB unified memory
Running on MLX with a flash-style SSD streaming path + local patching.
This is an experimental setup and I haven’t optimized speed yet, but it’s stable enough that I’ve started testing it in an autoresearch-style loop.
#LocalAI #MLX #MoE
seikixtc retweeted
This guy beat me to it
Guess I have to go big or go home
I got a 1T-parameter model running locally on my MacBook Pro.
LLM: Kimi K2.5
1,026,408,232,448 params (~1.026T)
Hardware: M2 Max MacBook Pro (2023) w/ 96GB unified memory
Running on MLX with a flash-style SSD streaming path + local patching.
This is an experimental setup and I haven’t optimized speed yet, but it’s stable enough that I’ve started testing it in an autoresearch-style loop.
#LocalAI #MLX #MoE
I got a 1T-parameter model running locally on my MacBook Pro.
LLM: Kimi K2.5
1,026,408,232,448 params (~1.026T)
Hardware: M2 Max MacBook Pro (2023) w/ 96GB unified memory
Running on MLX with a flash-style SSD streaming path + local patching.
This is an experimental setup and I haven’t optimized speed yet, but it’s stable enough that I’ve started testing it in an autoresearch-style loop.
#LocalAI #MLX #MoE





