@gentengineeri
iAccount based inGermany
About this account
- Account based in
- Germany
- Connected via
- Web
Account-level information from X, not a live location or the device used for a specific post.
Advancing charting technology through cutting-edge engineering.
Joined June 2016
- Tweets673
- Following108
- Followers56
- Likes1.1K
Pinned Tweet
Who else can claim their charting platform works on a pirate ship?
Custom network protocol makes it happen.
AI already produces enormous amount of code that a good engineer needs to go through and understand. Otherwise your project will collapse under the accumulated technical debt.
If you have to stop at this to comprehend you have already lost.
How I comforted by bro @temidaradev after 10 consecutive red monthly solana:So11111111111111111111111111111111111111112 candles during the bear market.
(His first cycle, was about to sell at the bottom)
This post proves Javascript developers live in delusional Hypetrendland.
There are thousands of programming languages suitable for backend.
But no it has to be what is most hyped right now: Rust.
They say girls mature faster, but I believe girls just do what they are told / expected to and we mistake that as maturity.
Boys may see less meaning in school and rather go outside to play ball or, nowadays, play video games instead of learning.
Schools need to find out what kids actually enjoy, support those interests and help them develop skills in those areas earlier.
Maybe there should also be more of an entrepreneurial side to education as well.
Copper is the economic growth trade 🤝
Maybe there are good times ahead assuming the AI bubble doesn't collapse?
Took all ~1 Million solana:So11111111111111111111111111111111111111112 spot trades on Binance from 22 Sep 2026 and compressed them about 45x smaller 😳
Featuring:
Brotli @jyzg
Zstd
Snappy
LZ4
the Kanzi suite
and a custom compressor
The data was a 75MB CSV file, 9.5MB zipped.
I extracted timestamp, trade id, price, volume and aggressor side for each trade and packed them into a 33.5MB binary file.
Compressing the Binary as is for comparison:
Lowest we got was around 3.5MB with Kanzis TPAQX context-mixing coder. 10x smaller but painfully slow (10s).
Transformation Step:
I ran a series of custom transforms that lower the entropy of the data, then ran the benchmark again.
Everything converged to around 2 MB, about 15–20× smaller than the binary.
Kanzi still wins at 1.68MB, this time using its CM Predictor and BWT Transform.
LZ4 and Snappy trail at about a still very impressive 2.3MB even without an entropy coder. Previously they were half as good as Brotli and Zstd.
Takeaway:
I would consider Brotli to be one of the strongest general purpose compressors when you want to squeeze that last few % while still maintaining reasonable decompression speed.
Not shown in the chart but Brotli lvl 10 was 2.3x faster than lvl 11 while being only 0.3% bigger.
In database systems LZ4 and Snappy are picked because speed is preferred over maximum compression.
Zstd offers a great balance between speed and compression. Although high compression levels can become considerably slower.
Given the volume of data we expect to store, compression efficiency is paramount. Moving beyond LZ4 and Snappy could save us terabytes of storage. At the same time, decompression needs to remain fast, including on mobile devices.
With a specialized custom compressor, we manage to compress over 2x faster than Zstd with slightly better ratio and slightly slower decompression.
And most of the gain doesn't even come from the compressor itself. The transformation step matters just as much, if not more. Add a specialized compressor on top and you are golden.
Additional notes:
Benchmarked single-threaded on an M3 MacBook.
LZ4, Snappy and Kanzi are pure Golang libraries.
LZ4 and Snappy include some assembly.
Brotli, Zstd and the custom compressor use optimized C libraries.
@aquipongoalgo2
There is this "Six Degrees of Separation" idea, that any two random people can be connected through an average chain of 6 acquaintances.
I wonder how it scales across time frames. How many chains is a person from the 12th century separated to someone from today?