
happy, following up on all the work we did for tsdb/metrics in ES, we are formalizing our learnings to a more generic columnar mode in ES, on an index level. Personally, still remember us adding doc values (columnar) to Lucene so many years ago (2012!), and so much since
We're introducing Elasticsearch Columnar Mode:
A new index mode that stores data once, in columnar form, with no redundant copies and no indexes the workload doesn't need.
Not replacing the document model. Adding a second way to organise data alongside it, for the workloads where columnar is the right shape: logs, telemetry, metrics, security events, AI retrieval.
One platform for search and analytics at the same level, on the same data.
Tech Preview in 9.5, GA in 9.6.
When turboquant came out, it got a lot of justified excitement. We took it for a Spin, and our year long work on Elasticsearch BBQ (better binary quantization) holds up better, especially in the abundant world of CPUs.
* also, we need to get Thom an upgrade from M2 ;)
Filtered vector search with restrictive filters just got 3-5x faster. Close to 10x on hyper-restrictive ones.
The old flow scored centroids without knowing if they contained matching vectors, loaded postings lists for nothing, decoded doc IDs that didn't pass the filter, and repeated until it had enough results.
The fix in Elasticsearch 9.4: a doc_id-to-centroid mapping that lets DiskBBQ skip straight to clusters with relevant vectors. Latency used to spike as filters got tighter. Now it stays flat or drops.
30x faster than Prometheus on gauge averages sounds like a typo. It's not.
Elasticsearch 9.4 rebuilt TSDS and ES|QL into a fully columnar metrics engine. Vectorized time series aggregations, zero-copy decoding, doc value skippers replacing inverted indices.
The result at 1.4M time series: queries that took Prometheus 30+ seconds finish in under 2.
jina-embeddings-v5-omni is now on Elastic Inference Service.
Text, images, audio, video. One index, one query.
• Best-in-class visual understanding under 1B parameters
• Beats models 20x its size on multilingual visual tasks
• Beats ByteDance Seed 1.6 on video (55.57 vs 29.30 on Charades-STA)
• BBQ quantization: 93% storage reduction, under 3% accuracy loss
• nano runs on commodity hardware without GPU
Introduction below with @florianhoenicke
Great work from the @elasticseclabs team!
We uncovered a new Brazilian banking trojan campaign: TCLBANKER.
What makes TCLBANKER notable isn’t just the malware itself, but how it spreads.
The campaign uses compromised WhatsApp and Outlook accounts to propagate through trusted user relationships, deploys targeted banking overlays, and incorporates anti-analysis techniques designed to evade detection.
For defenders, it’s another example of malware increasingly blending into legitimate user behavior and everyday communication channels, making detection harder and trust easier to exploit.
Our latest research breaks down the infection chain, propagation methods, evasion tactics, and detection opportunities observed across the campaign.
Read the full analysis: go.es.io/4ewvCKF
Impressive work by our team making Elasticsearch a kick ass columnar metrics store
30x faster than Prometheus on gauge averages sounds like a typo. It's not.
Elasticsearch 9.4 rebuilt TSDS and ES|QL into a fully columnar metrics engine. Vectorized time series aggregations, zero-copy decoding, doc value skippers replacing inverted indices.
The result at 1.4M time series: queries that took Prometheus 30+ seconds finish in under 2.
Shay Banon retweeted
Lots of noise around TurboQuant. It is good at MSE quantization. But claims around unique groundbreaking results are exaggerated. As it stands, I don't think it should be used for vector search indices. Tom Veasey shows why: elastic.co/search-labs/blog/…
Shay Banon retweeted
Preconditioning Vectors: Making DiskBBQ work for every vector, even non-normal ones. elastic.co/search-labs/blog/…
our journey with making Elasticsearch one of the best vector databases in the world continues, impressive and innovative work by our team
95 ns/vector vs 412 ns, Elasticsearch simdvec VS jvector scoring float32 at 32,500 vectors on x86.
But the number isn't really the story here. The story is that at scale, past L3 cache, memory latency beats compute every time and simdvec is built just for that.
At this depth, the kernel spends more time waiting for data than processing it. Prefetch too early and you've wasted the slot. Too late and you're stalling anyway. The window is narrow and simdvec times it right.
Explicit prefetch instructions on x86 pull the next vectors into cache while the current batch is still scoring. On ARM, interleaved loads do the same job differently. Either way, the pipeline stays fed.
Hardware counters show what that buys: 139K L1 cache misses drop to 19K. That's where the 4x gap lives.
The simdvec team wrote up the full architecture and benchmarks, including their hardware counter data.
Native promql support for metrics in Elasticsearch, fused right into our ES|QL so you can optionally fuse it. Pretty cool and tons of more metrics news (perf! Moar columnar!) coming soon!
If your team lives in PromQL, you no longer have to context-switch to query your Elasticsearch metrics.
The new PROMQL source command in ES|QL lets you run PromQL queries directly against Elasticsearch time series data, whether ingested from Prometheus Remote Write, OpenTelemetry, or other sources. Because it's built on the ES|QL engine rather than a separate query layer, PromQL output can be piped into filters, sorts, joins, and enrichments.
That's not something you can do in Prometheus.
The Axios attack again proved the status quo is broken. We caught it by using AI to diff package repo updates. Built Friday, caught the compromise Monday.
We open sourced the project so the community can iterate on it. We all need a higher floor for defense.
One of our researchers built an AI powered supply chain monitoring tool on a Friday afternoon.
The following Monday night it caught the Axios npm compromise before most people knew it existed.
Elastic Security Labs is open sourcing the tool.
Full story by @dez_ here: go.es.io/4bOfsuq
Shay Banon retweeted
We open sourced the tool used to detect the Axios supply chain compromise! I built it Friday after a red eye home from RSAC. Also, wrote up the full story, including the hectic moments after that first critical alert
github.com/elastic/supply-ch…
Shay Banon retweeted
We have discovered a massive supply chain compromise in the Axios npm package.
A backdoored maintainer account delivered a cross-platform RAT for Linux, Windows & macOS, targeting the Axios package, which has ~100M weekly downloads and is in the top five most popular Node.js packages.
We filed a GitHub Security Advisory to coordinate the disclosure, ensuring that the maintainers and the npm registry could act swiftly on the compromised versions.
Full analysis: go.es.io/4sHybxr
Shay Banon retweeted
We've got two days left to get these folks on at @SXSW! 👇
🌟 @MichelTricot — CEO & Co-Founder, Airbyte
🌟 Shay @kimchy Banon — CTO & Founder, @Elastic
🌟 @LindaMLian — CEO & Co-Founder, @CommonRoomHQ
🌟 Paul @kiwicopple Copplestone — CEO & Founder, @Supabase
Do your part by voting for them here: bit.ly/3UQo4al
If you do, they'll share how open-source communities offer a competitive edge, fueling faster innovation, unlocking developer mindshare, and laying the foundation for enterprise adoption.
OpenSearch moves away from Lucene, while we work hard to make Lucene and Elasticsearch better, the difference between faking it and actually building elastic.co/search-labs/blog/…
Shay Banon retweeted
yes!
one of the underrated use-cases of elasticsearch: speeding up your primary database for the presentation
We made search in HEY 7x faster by ensuring we're only pulling data directly from the ElasticSearch indexes for the presentation. Sub-100ms response times now on the median. Very noticeable! hey.com/new/#quick-search




















