If you need fast append/insert/delete, usually the solution is not "just an array" but clever pre-allocation schemes and a mixture between linear and linked data to amortize the cost of updates while still keeping most of the linear memory benefits for sequential access.

Another downside is that the reference chain in a linked list adds a dependency chain between elements in the list that prevents potential OOO benefits and SIMD optimizations.

I see. I looked into uprobes again for github.com/jvm-profiling-too… but JIT code / code in anon mappings is still not supported. It seems that you can use mem eXecution watchpoints to trace function entry but something like return probes still seems quite elusive.

Almost 50 years ago mastermind Loriot created this visualization of what happens during a rolling upgrade if you implemented sharding naively (w/o consistent hashing) dailymotion.com/video/x2x2dh…

Would you like to see the dynamic tracing functionality or even use the linux infra for that?

Thanks, Matthias!

My job as a Rust Engineering Lead at a startup became victim of restructuring and I'm looking for a new role. I enjoy working on infra software, Open Source, with Rust, Scala, or whatever is needed. I'm also available contractually to support previous libs like Pekko Http.

Great stuff, maybe I should retry llama2.scala on Scala Native as well since it was also hampered e.g. by ByteBuffer issues.

Thank you, @scrollprize, for awarding me a prize in the Vesuvius Challenge for my Open Source contributions cataloguing released segments and running OS ink detection models on them. It has been a fun journey and great collaboration! blog.virtual-void.net/2023/1…

Thanks for egui :) For the first time in 15 years, I have been enjoying working on a gui after avoiding GUIs at all cost because of the lack of good options...

A short report on trying out different algorithms for selecting top-p elements from a distribution (aka Nucleus Sampling): blog.virtual-void.net/2023/0…

The whole point of cloud volumes is to be able to view storage detached from compute. Optimally, storage can be easily migrated to new compute in cases of problems. Quite a bummer if storage cannot be relocated in cases like this... @Hetzner_Online

Mmh, @hetzner cloud volume hangs while unmounting since one hour. No more actions on the server are possible, after shutting the node down other mounted volumes are also blocked, server cannot be restarted. Big share of cluster storage is unavailable => extended downtime.

Also, the day, when you first run a code generation model on your engine asking for suggestions about how to improve algorithms of the underlying engine 🤯 (the suggestions are mostly weird and broken, admittedly, but had some ideas to follow up) gist.github.com/jrudolph/fb7…

llama2.scala now has quantization support, AVX2 kernels, multithreading and can load some GGML models directly. github.com/jrudolph/llama2.s…

@karpathy's llama2.c is interesting because it removes all abstraction and breaks down inference into a single file of C (which can be seen as a lingua franca of computation because data structures map closely to actual memory layout and operations to CPU instructions)

Python's speed is mostly irrelevant since ML means instructing the GPU which matrix op to do next. In big models, each multiplication takes so long that it dwarfs the overhead of the language. If you are looking at state-of-the-art inference on CPU look at llama.cpp.

Sounds like an interesting experiment (but I don't have a GPU to test it on and the iGPU has max 2GB of UMA, so somewhat limited).