building systems • low performance athlete at @strava
Joined January 2007
- Tweets761
- Following441
- Followers2.3K
- Likes609
Pinned Tweet
Prototyping is cheaper than ever, so I tried to imagine the simplest possible ClickHouse service.
The result is a side project called ClickHamlet. You run:
clickhouse client --host cloud.clickhamlet.com --secure --user <name> --password
And that's it. No sign-up form, real ClickHouse in the cloud.
moved @ClickHamlet to clickhouse 26.10. quiet upgrade, but a few changes land right where s3 hurts:
- reader refreshes skip GETs for directories that didn't change
- part removal is one batched delete instead of copying each file first
- metadata loads several times faster
the router also caps nested CTE expansion with max_expanded_ast_elements now, before it could grow without limit
i'm such a good manager i got my grok bots fighting each other, interrupting themselves, and spending more time arguing than doing the actual task
is this AGI, or did they just become a real engineering team?
Jordi Villar retweeted
added stripe billing to @ClickHamlet, first time doing it end to end:
- free and paid plans, checkout + webhooks
- usage meters: compute per GiB-minute, storage per MiB-hour
- costs counted from the moment an account starts paying
- the hour you upgrade is reported once, with idempotency keys on meter events
first customer: me, at 100% off 😅
added stripe billing to @ClickHamlet, first time doing it end to end:
- free and paid plans, checkout + webhooks
- usage meters: compute per GiB-minute, storage per MiB-hour
- costs counted from the moment an account starts paying
- the hour you upgrade is reported once, with idempotency keys on meter events
first customer: me, at 100% off 😅
impossible to escape from ai these days. yesterday, while having dinner, a lawyer and a cinema director were explaining their agents/bots setup..
does it make sense to build a clickhouse offering on top of s3?
last 30 days: $42.34 revenue at our prices, $51.78+ in aws (billing lags ~24h)
if you've run clickhouse on top of s3 you won't be surprised by the puts eating a big chunk of costs: ~410 rows/h inserted turned into ~5,500 PUT/COPY and ~8,000 GET/HEAD an hour, mostly doing merges.
you've to spend hours optimizing, packing small parts and refreshing every 5s instead of 2s did the trick for me
how does ClickHouse Inc make the math work?
no iba a ser menos y Codex no solo ha conseguido hacer que el proyector vaya como un tiro, sino que también ha arreglado el ajuste automático a la pantalla que fallaba más que una escopeta de feria
I didn't care about deployments when I started working on @ClickHamlet, it was an ugly bash script. Now that it's running for some users, it was time to add zero-downtime deploys:
- releases are immutable and prepared before the switch
- the controller leaves backends running when it exits, the next one adopts them
- a systemd socket unit keeps the ports open while it restarts, so connections queue instead of failing
- routers retry on a fresh connection if their pooled one breaks
- idle and drain timers live as annotations, so they survive the restart
Deployed with queries running, nobody noticed
Your servers start on the first query and stop when you're done, so an idle database runs nothing.
Reads and writes go to separate servers sharing the same storage, so heavy dashboards don't slow down ingestion.
Your first login creates your service, and you have 2 days to claim it from the dashboard link.
It's in early preview. Try it, break it, and tell me what's missing.
clickhamlet.com
Jordi Villar retweeted
spent the weekend making my side project's query path faster, measuring every step and re-running all of it on one machine:
443 → 2,209 queries/s through the router (5×)
2.39 → 0.31 ms of controller CPU per query (7.7× less)
no single big fix, just small changes that add up
spent the weekend making my side project's query path faster, measuring every step and re-running all of it on one machine:
443 → 2,209 queries/s through the router (5×)
2.39 → 0.31 ms of controller CPU per query (7.7× less)
no single big fix, just small changes that add up
