Making stuff fast at @redpandadata.
Canada
Joined December 2018
- Tweets12.6K
- Following176
- Followers4.2K
- Likes5.7K
Travis Downs retweeted
The latest release of the simdutf C++ library (6.0.0) brings in more convenient for C++20 users. While you used to have to provide both a pointer and a size parameter... often you can now just pass your container...
std::vector data{1, 2, 3, 4, 5};
// C++11 API
auto cpp11 = simdutf::autodetect_encoding(data.data(), data.size());
// C++20 API
auto cpp20 = simdutf::autodetect_encoding(data);
Link in the comments.
Travis Downs retweeted
to our friends/companies downloading #redpanda 1.5M times a day... should turn on caching 😂
Travis Downs retweeted
We're excited to hear @redpandadata's Travis Downs describe the practical experience of building high performance systems with C++20 in an asynchronous runtime. He'll also discuss tradeoffs in adopting a thread-per-core architecture. bit.ly/43RHzlV
#P99CONF #ScyllaDB
Travis Downs retweeted
For a long time I've been thinking that using a closed loop (sync) for measuring latency is wrong
Travis Downs retweeted
Measuring the memory usage of your C++ program lemire.me/blog/2022/11/10/me…
Travis Downs retweeted
some pretty cool tests we are doing on the #redpanda cloud of aggregate throughputs of 40GB/s. nbd .... more deets to come on Nov 15th. hopin.com/events/redpanda-op…
thanks to @trav_downs for the goodie you see below.... maybe we should do a 200GB/s test next? 🤣
#kafka
There is always something special about seeing a 30GB/s #redpanda cluster (10GB/s producer writes + 20GB/s for replicas). Very cool 😎
Travis Downs retweeted
10-20GB/s yes. We don't have anyone at 40GB/s yet. 400GbE not yet, but DM me if you want to test. This is the reason we have a dedicated Performance Engineering team
xitter.cf/trav_downs/status/1575…
Do you count cycles in your sleep?
Do you sweat over those last few μsecs?
Do you both love and hate Nagle's algorithm?
Join me in making future of event streaming wicked fast!
@emaxerrno has built a team of great people here at Redpanda and we're looking for a perf expert.
Travis Downs retweeted
New ARM instructions were just released! (With SVE 2.1, among many other things)
(I always use the exploration tools, but you can view them online too: developer.arm.com/documentat…)
developer.arm.com/downloads/…
Travis Downs retweeted
While everyone is enjoying all the #AMD #Zen4 performance reviews, here is my teardown of Zen4's #AVX512 implementation and architecture.
tl;dr - Intel has some serious competition now.
mersenneforum.org/showthread…
Travis Downs retweeted
y-cruncher v0.7.10 coming out soon with Zen4 AVX512 optimizations.
This was fun since there are no optimization resources yet for this chip. Had to do all the RE myself.😅
Travis Downs retweeted
New blog post: Faster zlib/DEFLATE decompression on the Apple M1 (and x86)
dougallj.wordpress.com/2022/…
ALT Graph of decompression performance, showing zlib-dougallj outperforming 3 alternatives by about 50% on each of the files in the Silesia Corpus. zlib-cloudflare is consistently second best, closely followed by zlib-ng, with zlib-apple further behind, outperformed by about 2x. The files have been compressed with zlib level 6, and throughput is measured in compressed mb/s on M1 using Apple clang 13.1.6. zlib-dougallj performs between 282 and 380mb/s, cloudflare performs between 186 and 276mb/s. xml: apple 122, ng 181, cf 194, dj 290. x-ray: apple 181, ng 190, cf 209, dj 330. webster: apple 132, ng 195, cf 205, dj 303. sao: apple 234, ng 260, cf 276, dj 380. samba: apple 151, ng 211, cf 221, dj 321. reymont: apple 131, ng 210, cf 213, dj 338. osdb: apple 173, ng 206, cf 224, dj 352. ooffice: apple 142, ng 170, cf 186, dj 291. nci: apple 119, ng 171, cf 191, dj 282. mr: apple 164, ng 187, cf 208, dj 329. mozilla: apple 153, ng 187, cf 199, dj 307. dickens: apple 148, ng 219, cf 238, dj 344.
Travis Downs retweeted
Given:
1. crc32 has throughput 1 on port 1
2. pclmulqdq has throughput 1 on port 5
3. pclmulqdq+pxor can emulate crc32
It seems that fastest crc32 code should divide input in half and issue a crc32 _and_ a pclmulqdq every cycle.
Code and numbers at corsix.org/content/fast-crc3…
Travis Downs retweeted
Filtering numbers quickly with SVE on Amazon Graviton 3 processors lemire.me/blog/2022/06/23/fi…
Travis Downs retweeted
How do Graviton2 Arm & Intel x86 processors stack up for data-intensive workloads? Don’t miss our webinar Wednesday with @LinuxFoundation to see the results of our research!
linuxfoundation.org/webinars…
Travis Downs retweeted
Posted an AVX2-vectorized-sorting benchmarking script cr.yp.to/software/sortbench-… covering djbsort, vxsort, vqsort. (vxsort and vqsort also support AVX-512.) The middle part of this Skylake graph is the part that matters for crypto, and also the base case for larger quicksort etc.
Travis Downs retweeted
How did I notice!?!? Excited for @jfbastien latest podcast episode! @TLBhit podcasts.google.com/feed/aHR… – at Evanston, IL
Travis Downs retweeted
Hello y'all fine Internet Folks!
Tonight's article is on Graviton 3 and a first look at the microarchitecture.
Hope y'all enjoy!
chipsandcheese.com/2022/05/2…
Travis Downs retweeted
Our best release to date. A story of dollars and safety 🧵
offloading data transparently to s3 is the single most important cost savings anyone doing streaming can do.
most importantly, your app does'nt change, not even one line of code. zero. just more $$$ in your pocket.
🚨🚨Redpanda 22.1 is here! 🚨🚨
Packed with goodies to further simplify your data streaming journey!
- Transparent tiered storage (GA)
- Centralized configuration
- Maintenance mode
- Rack awareness
- Consumer offsets
And more!
#kafka #redpanda
redpanda.com/blog/redpanda-2…
