Cofounder and CTO of Tobiko Data. Building SQLMesh and SQLGlot.

San Mateo, CA
Joined December 2013
Pretty amazing for just 4 sparks.
1
3
322
the latest ds v4.1 flash is the real deal. it’s the first model that has actually beaten (let alone come close) to claude on my real world agentic benchmarks. my mind is blown. it’s finally here.
2
6
599
V4.1 and Opus 4.8 now tie on total time: 1,138 s against 1,139 s. V4.1 used 7% less input and 44% less output, and costs about 1% of Opus 4.8's price. This running on 4 sparks
1
90
all of these benchmarks are bullshit. on real agentic work, i have glm 5.3 flash vs opus 4.8, it's not even close. opus 4.8 wall clock is up to 8x faster. It's not just the tps because i'm running sparks, it's efficiency. Opus uses half the tool calls. local is not frontier yet.
7
1
29
3,761
trying out nvfp4 glm flash vs fp8, i think fp8 is better, nvfp has slightly faster decode but the mtp acceptance is lower and it is using more tokens in my local agentic coding benchmarks. gonna use fp8 instead but going to start looking at glm full and the new ds4 vision.
2
10
496
end to end it finished tasks at similar speeds or faster. even though decode was slower
129
after a week of dsv4 flash on 4 sparks, i've decided to switch back to glm 5.2. but i discovered that i can offload kv cache to disk to avoid long cold pre-fills on agentic sessions. 600k max context windows x 8!? amazing.
3
385
there’s a big issue with dsv4 flash on jasl vllm. after 80k tokens with parallel sessions, it can corrupt, been debugging it for multiple days now, anyone else run into this?
1
290
4 sparks with cooling serving glm / dsv4. can’t wait to see what’s next!
1
5
247
dsv4 flash (no quant) losing its mind. has anyone else seen this?
1
4
325
dsv4 flash on 4 dgx spark is fast, but the token efficient is way worse than glm 5.2, wall clock is similar on my internal coding benchmarks, but dsv4 uses 10x the amount of tokens. is it better? maybe. speed wise yes, but it thinks harder not smarter.
1
5
328
Toby Mao retweeted
“Pure Python” and “fast” don’t usually go together. We used mypyc to speed up SQLGlot by up to 5x without rewriting it in Rust or changing the codebase. This post breaks down how we squeezed as much performance from a large Python project as possible: 5tran.co/3OYg7AP
3
6
324
This week’s episode of @drilltodetail features @Captaintobs from @TobikoData on @Airbnb, #DataOps and the engineering innovation behind @SQLMesh
3
4
1,490
Exciting new feature for @SQLMesh. Custom signals / triggers. SQLMesh's built-in scheduler controls which models are evaluated when the sqlmesh run command is executed. It determines whether to evaluate a model based on whether the model's cron has elapsed since the previous evaluation. For example, if a model's cron was @daily, the scheduler would evaluate the model if its last evaluation occurred on any day before today. Unfortunately, the world does not always accommodate our data system's schedules. Data may land in our system after downstream daily models already ran. The scheduler did its job correctly, but today's late data will not be processed until tomorrow's scheduled run. You can now use signals to prevent this problem! Just write a function in Python to determine which intervals are ready to be processed. sqlmesh.readthedocs.io/en/la…
1
4
28
2,099
SQLMesh concepts with plans that apply to different environments (prod, dev) are elegant. Even `fetchdf` is integrated into the CLI. Also, on the right, you see SQLMesh auto-detecting the new columns as non-breaking and simply applying the (virtual) changes `y`.
7
57
3,429
It surprises me how most poorly most data orchestrators support batch size for backfills Meanwhile, @SQLMesh open source supports out of the box
Simplifying our data stack i.e. killing Airflow
4
1
45
6,701
⚠️⚠️Read this before you start using dbt's microbatch models. There are three large gaps that could lead to serious data issues. Due to fundamental architectural design choices of dbt, the microbatch implementation is very limited. At its core, dbt is a stateless scripting tool with no concept of time, meaning it is the user's responsibility to figure out what data needs to be processed. 1. dbt's microbatch can lead to silent data gaps 2. dbt's lack of scheduling requires manual orchestration which could lead to incomplete and incorrect data 3. Mixed time granularities in microbatch can cause incomplete data and wasted compute tobikodata.com/dbt-increment…
1
7
43
4,068
Tired of messy data pipelines? 🥹🫣 Check out the @SQLMesh + @dltHub integration for seamless metadata handovers, faster scaffolding, and incremental processing. 💻 Simplify your data workflows! 🔗 tobikodata.com/integrated-da… #DataEngineering #DataPipelines
1
4
1
13
1,124