@chdb_io

https://xitter.cf/t.co/vlD2sAhx3c

 localhost
github.com/chdb-io/chdb
Joined September 2023
  • Tweets882
  • Following19
  • Followers960
92Media
Hybrid agent memory search fits in one SQL query. Rank a user's memories by cosine, rank exact OOMKilled hits by recency, score each 1/(60 + rank), sum. No score normalizing. A memory at #13 and #47 beats a #1 in one list. chDB, 20k memories per user in a 1M-row table: 12 ms.
19
Field notes on ReplacingMergeTree from a real NSE price pipeline, with the trap reproduced in chDB: re-inserted keys stay duplicated until a merge, so any count or avg without FINAL is off. And it never retires a key you stop writing. hikmahtechnologies.com/blog/…

Best database for stock data: what I store NSE prices in, and what broke

The best database for stock data depends less on the engine than on how you key, overwrite and deduplicate prices. How I store daily NSE bars in Parquet and ClickHouse, with the ReplacingMergeTree...

hikmahtechnologies.com
48
The OneLake Iceberg bench now keeps a chDB lessons log. Two worth copying: size the filesystem cache to the disk, not the data (too big is ENOSPC mid-query, not eviction), and cap max_memory_usage so a heavy query fails cleanly, not the runner. github.com/djouallah/lakehou…

lakehouse_benchmark/LEARNING.md at main · djouallah/lakehouse_benchmark

https://djouallah.github.io/lakehouse_benchmark/. Contribute to djouallah/lakehouse_benchmark development by creating an account on GitHub.

github.com
1
109
MCP tip: before an agent queries s3('.../*.parquet'), let it price the scan. Format One lists objects without reading them: SELECT count(), formatReadableSize(sum(_size)) FROM s3('.../*.parquet', One) Public hits bucket: 100 files, 13.73 GiB, 1.2s in chDB. Too big? Sample one.
2
146
80–200 TB a day, in a single partition 📈 Jump Trading's logs are its record of every order and trade, and it has run them on ClickHouse for 7+ years. Arnaud Adant on why they chose it, at Open House San Francisco 2026 👇 Try ClickHouse on your own data, free with $300 in ClickHouse Cloud credits → clickhou.se/4s2Nhyc
3
4
15
1,250
Agent memory doesn't need a database server. It can be a folder: from chdb.session import Session s = Session("agent_db") 1M turns in MergeTree, 92 MB. A fresh process imports chDB, opens the folder and runs a 7-day GROUP BY in 0.12s. Copy the folder elsewhere; it still opens.
1
1
3
174
Loams is building a ClickHouse HTTP service on sandboxed chDB workers and measured the boot cost: 136.5 ms p50 from spawn to first SELECT 1, 1.1-1.4 ms for a second connection in a booted worker, and user/net/mount/pid namespaces add almost nothing. github.com/ostrium-labs/loam…

loams/docs/house/performance.md at main · ostrium-labs/loams

One bucket, every index: open-source, AI-native data platform in Rust. Hybrid retrieval on object storage, plus reactive data (Live), graph, SQL, Postgres and durable agent runs. Preview. - ostrium...

github.com
1
4
304
chDB chdb-core nightly builds are out. Daily wheels from main, which now syncs with ClickHouse master. macOS and Linux, x86_64 and arm64, Python 3.9+. Not for production. pip install --pre --extra-index-url chdb-nightly.vercel.app/simp… chdb-core github.com/chdb-io/chdb-core…
5
249
Agent memory tip: store embeddings as Array(BFloat16), not Array(Float32). chDB, 1M memories, 384-d, exact cosine top-10: Float32: 1.44 GiB, 156 ms BFloat16: 739 MiB, 112 ms Same top-10 in 19 of 20 random queries; the 20th swapped 1 of 10. One type change in CREATE TABLE.
1
135
Summing token usage from Claude Code transcripts? Dedupe first. One API response is logged once per content block, with the same message id and identical usage. On 1.9 GB of local JSONL: 76k usage lines, only 35k responses. A naive sum puts output tokens 2.7x too high.
2
171
One LIMIT BY fixes it. chDB, 0.8s over 1.9 GB: cd ~/.claude/projects python3 -m chdb " SELECT sum(o) FROM ( SELECT JSONExtract(line,'message','usage','output_tokens','UInt64') o FROM file('*/*.jsonl', LineAsString) LIMIT 1 BY JSONExtractString(line,'message','id'))"
1
72
Agent-written pandas: e = df[df.status == "error"] g = e.groupby("tool")["ms"].agg(["count", "mean"]) g.sort_values("count", ascending=False).head(3) 30M tool calls, 2.3 GB Parquet: pandas: 3.1s, 7-8 GB RSS import chdb.datastore as pd: 0.36s, 0.36 GB Same 3 rows, same order.
1
5
256
An options analytics pipeline (Kafka, ClickHouse, Postgres, FastAPI) tests each ClickHouse migration and query in embedded chDB, on synthetic data with known true vols. Two settings match clickhouse-connect's output; a lock covers FastAPI's thread pool. github.com/Suleyman040/optio…

GitHub - Suleyman040/options-analytics: eal-time options analytics pipeline in Python: implied...

eal-time options analytics pipeline in Python: implied volatility and Greeks from live market data, built with Kafka, ClickHouse, Postgres and FastAPI. - Suleyman040/options-analytics

github.com
3
6
279
Sonde watches AI agents via OpenTelemetry, ClickHouse and Grafana. Its SQL tests replay agent spans into embedded chDB exactly as the collector writes them, then run every view, dashboard panel, alert and documented query against it. No server needed. github.com/OlivMartins/sonde

GitHub - OlivMartins/sonde

Contribute to OlivMartins/sonde development by creating an account on GitHub.

github.com
2
137
Filtered vector search can quietly return fewer rows than you asked for. chDB, 500k agent memories, 384-d, HNSW index. Nearest 10 from the last 7 days (7.8% of rows): default: 1 row, 5 ms vector_search_filter_strategy='prefilter': 10 rows, 79 ms HNSW picks 10, WHERE drops 9.
2
120
WalShadow, ClickHouse Managed Postgres on Google Cloud and the chdb Postgres extension all shipped the week of Open House NYC. Sai Srirampur and Kaushik Iska's recap covers all three, plus the live demo: clickhou.se/4hLZzaI

Postgres at Open House NYC | ClickHouse

At Open House NYC, we showed how ClickHouse Managed Postgres and ClickHouse work as one stack, with a live WalShadow demo replicating Postgres into ClickHouse in under a second, plus the same-week...

clickhouse.com
9
1
17
1,945
A new database/sql driver for embedded chDB in Go gives each pooled connection its own chDB session, so one writer and several readers query in parallel in one process. No cgo, and a sibling HTTP driver runs the same code against a ClickHouse server. github.com/tiny-obs/chdb-sql

GitHub - tiny-obs/chdb-sql

Contribute to tiny-obs/chdb-sql development by creating an account on GitHub.

github.com
106
An MCP data server doesn't need a connector per source. Two tools are enough: describe(src): DESCRIBE file('t.parquet') query(sql): chdb.query(sql, 'Markdown') Swap file() for s3(), url(), postgresql(), iceberg(). DESCRIBE on a public S3 Parquet: 105 columns, under 3s cold.
2
1
247
nano, a Rust SIEM, lists embedded chDB as the query engine in its deployment docs. Hosted tenants each get their own chDB process over signed Parquet in R2. Self-hosted runs chDB lake query workers over Iceberg on RustFS. nano.rs/docs/concepts/deploy… @dansec_

Deployment models | nano SIEM

Hosted nano and self-hosted open core: how each stores, ingests and retains events, and what differs between them.

nano.rs
1
9
274
An agent tool can be one file. No server, no setup: # /// script # dependencies = ["chdb"] # /// import chdb print(chdb.query("SELECT tool, count() n FROM file('logs/*.jsonl') GROUP BY tool ORDER BY n DESC", "Markdown")) uv run it: 1M JSONL rows, 0.2s (first run ~5-10s).
7
344
Load more