@ObsidianLib

Goo - a .NET declarative desktop UI framework: https://xitter.cf/t.co/gCNIM647xo SharpTUI - a lean .NET TUI Framework: https://xitter.cf/t.co/aPXnMhEFWh

Joined July 2024
  • Tweets5.5K
  • Following420
  • Followers258
542Media
Goo speaks for itself. Cross-platform UI framework in .NET on Vulkan. OSS. Try it out :)
1
1
1
11
435
🐉longhorn update🐉 #bnt #blocknet
1
4
125
awesome the first #blocknet block mined on the longhorn pool happened last night! Not many people on the pool yet, so it all went to one miner. If you have an AMD CPU/GPU, get in on the pool: longhornpool.net/
1
3
6
99
LONGHORN is LIVE. longhornpool.net A #BNT #blocknet miner built for AMD, plus a zero-fee pool. Pool 0% fee Miner 2% fee The lowest total fees for AMD miners.
1
4
7
471
longhorn. soon. #bnt #blocknet
1
1
5
215
blocknet miner specifically tuned for AMD, anyone? Long Horn 🐉😈 #bnt #blocknet
2
4
6
581
Elation retweeted
tokate
@tokatedevOct 15
The first two gratis donations have been contributed to @_gnoland, an interpreted and fully-deterministic implementation of the Go programming language. See them here: github.com/gnolang/gno/pull/… github.com/gnolang/gno/pull/…
🤖 Made with AI

fix(gnovm): allow comma-ok results in interface variables by obselate · Pull Request #6354 ·...

Fixes #6343 Use untyped bool assignability for comma-ok map indexes and type assertions. Add filetests for empty interfaces, named bools, true and false results, and incompatible targets. Update ...

github.com
2
4
43
local models take a long time to test...might as well make something to keep you mesmerized as it works.
2
77
Elation retweeted
EgorBo
@EgorBoOct 12
GPT + Opus (via Copilot CLI) helped me uncover 180 JIT bugs, each with a minimal C# repro using only safe code 😶 - most had been hiding from us for many releases github.com/dotnet/runtime/is…

AI bug hunting in JIT (tracking issue) · Issue #134482 · dotnet/runtime

List of issues (each with a minimal C# repro, no IL, no opt-in configs, no unsafe/UB) filed against JIT with the "AI bug hunting" script. Just so I can see the current status (how many of...

github.com
14
29
6
285
27,314
Gloop 🤝 Tern
6
146
Elation retweeted
tokate
@tokatedevOct 12
Donating inference to your favorite OSS project will soon be beautiful and easy. Set up your repo, donate to another repo, browse issues, save profiles with your preferred harness, model, and effort. Give your inference a purpose 🌞
1
2
4
65
Bus use: stock llama.cpp reached ~145GB/s on Ornith. These engine tweaks reach ~171GB/s over a whole pass and 234GB/s in its best kernel. The memory floor for a pass is ~17.4ms, this implementation reaches 23.5ms so still some headroom left at the same quant.
2
26
Caveats: custom ggml + libllama under stock llama-server Speed moves ~2.5% run to run and more by prompt. Pooled routing changes the model, a preliminary and partial SWE-bench run showed about 11/19 resolved rate with my pi harness, 10m per task.
1
19
The takeaway so far: capacity is free but bandwidth is not. The ideal model is ~100GB of small experts behind a thin trunk, reading ~3GB per token. Ornith has the read pattern right, but does not fill the hardware's capacity.
16
Draft head: only the 65,536 most common tokens of 248k, stored at Q4_0. Used for drafts only, every token is still verified by the full Q8_0 model so the output does not change. It only moves draft acceptance rate.
1
16
Results so far, Q8_0 @ temp 1.0 32k ctx: Decode with MTP: 59.3 -> 99.7 t/s Decode without MTP: 53.1 t/s Verify pass: 33.8ms -> 23.5ms Prefill: ~1,100 t/s TTFT: 126ms short prompt, 1.5s at 1.6k tokens 37.9GB file, ~43GB loaded
1
27
Next lever is MTP and Ornith ships its own draft block. It drafts 3 tokens, then one main pass verifies 4 positions. The trunk is read once per pass, not once per token. So, a thin trunk makes verification cheap. ~2.7 tokens kept per pass.
1
24
more replies
Then the engine, a stock verify pass is ~2,400 GPU nodes and each small kernel costs about 5us. I fused them: MoE layer 42 nodes -> 1 op, linear attention layer -> 3 ops. Now 284 nodes and each weight row is read once for all 4 tokens.
1
11
There is a problem, though. 4 positions each pick 8 of 256 experts, so up to 32 expert sets per pass. The fix is pooled routing. The 4 positions share a pool of 12 experts and each takes its top 8 from the pool. ~60% less expert traffic. Routers tuned for the Strix.
1
12
Since you can't make the bus faster, you need to cut bytes/token. Ornith at Q8_0 reads a 2.17GB trunk once per pass + 1.10GB of experts per token. Stock llama.cpp running Ornith-1.5-35B-A3B: 45 t/s
1
32
Doing some AMD Halo Strix testing... First is picking a base model that fits the strix's architecture and is able to run as optimally as the hardware allows. Ornith 1.5 35B A3B fits this description as it is small-trunk MoE, allowing for small experts on the shared memory bus.
1
1
1
220
The Halo Strix used in testing has 128GB of memory but one shared bus at ~230GB/s. Single stream means your speed is bandwidth / bytes read per token. Size and compute is not the limit in this case. This raises the question, how do we saturate that 230GB/s shared bus best?
1
1
62
Load more