Matthias Broecheler

In the age of coding agents and free code, the key challenge is maintaining clean abstractions to retain understanding. buildingwithdata.substack.co…

The real danger of AI is losing intellectual control. It's a concept coined by Edsger Dijkstra and highly relevant for the adoption of AI. Strategies for minimizing and managing intellectual control debt in AI transformations are in my Substack article: buildingwithdata.substack.co…

In Part 2 of "Why do I need a data engineering harness?" we are going deep and discuss how relational analysis surfaces issues with pipeline implementations that can escape the code review. docs.datasqrl.com/blog/p2-va… Join us at ODSC West on Oct 28 in SF. hubs.li/Q04rKfGf0

"Why do I need a data engineering harness? Isn't Claude Code good enough?" Coding agents can cause subtle breaks at the boundaries between systems. A data engineering harness derives the seams deterministically to avoid generation errors. docs.datasqrl.com/blog/p1-br…

I've seen a lot of AI automation projects stall out this year because AI wasn't applied correctly. Are you using AI as the implementor or as the reasoner? It makes a huge difference. Wrote about the distinction with a practical churn prediction example: buildingwithdata.substack.co…

Just launched: Flink SQL Runner — run Flink SQL apps in prod with zero boilerplate. Key features: •SQL & compiled plan execution •Kafka DLQ support •Env var injection •UDFs, JSONB, vectors, Postgres compatibility •K8s Operator friendly 👉 github.com/datasqrl/flink-sq…

“Batch or streaming?” is the wrong question. Most use cases live in the middle—and incremental data processing is the answer. ⚡ Fresh enough 💸 Cheaper than real-time 🧠 Simpler to operate New post: buildingwithdata.substack.co…

Build scalable, robust, and fast data APIs with just SQL? Mutations, queries, and subscriptions. Learn more: docs.datasqrl.com/blog/flink…

🚀 Building Flink apps shouldn’t feel like rocket surgery. In this video, we show how DataSQRL automates the data plumbing—so you can focus on business logic: youtube.com/watch?v=LiDefKUg… - Realtime analytics - Kafka + Flink + Postgres - Built-in tests & CI/CD - Developer Tooling

How to deliver value quickly in your data initiative? Practical advice for agile data development: •Show value early •Align the team •DataOps for quick iteration cycles buildingwithdata.substack.co…

“Just get it from the data warehouse.” That phrase derails more data initiatives than bad SQL. Daily snapshots ≠ responsive data. BI pipelines ≠ product infrastructure. Don't let the data warehouse dictate your architecture. Full post 👉 buildingwithdata.substack.co…

If ChatGPT had been invented in the 1950s, would we all be vibe-coding Assembly now?

🚀 DataSQRL 0.6 is here - The Streaming Data Framework •Full Flink SQL compatibility •Real-time serving for LLMs & user-facing apps •CI support Plus bugfixes and perf improvements. Read more 👉 docs.datasqrl.com/blog/datas… #StreamingData #DataEngineering #OpenSource

The data warehouse can accelerate your data initiative or derail it. Learn the 7 ways your warehouse helps (and hurts), and how to manage the relationship to your advantage. New Substack article 👉 buildingwithdata.substack.co… #DataEngineering #Analytics #DataOps #BuildingWithData

Building streaming apps or event-driven microservices with #Kafka or #ApacheFlink? Then you have to decide whether to preprocess your data in stream or query it at request time. Learn how to make the right decision: datasqrl.com/blog/preprocess…

⏳Temporal join is a secret superpower of stream processing engines like #ApacheFlink, allowing you to "time travel" for time-consistent joins. Learn more about temporal joins, when to use them, and why they are powerful: datasqrl.com/blog/temporal-j…

Simplify your #ApacheFlink app development with #DataSQRL 🐿️! This open-source tool compiles SQL into a microservice, integrating Flink, Kafka, #Postgres & API layer. Focus on app logic, not complex tech integration. Learn more: datasqrl.com/blog/simplifyin… #streamprocessing