Helping data teams preview, validate, and ship data changes with confidence. https://xitter.cf/t.co/BgSF4fEcve

The Data Pipeline
Joined November 2023
Recce 1.0 is now live on Product Hunt! producthunt.com/posts/recce-… Upvote and leave a comment to help us grow the Recce community and bring better data review processed to more data teams Thanks for your support! #OpenSource #Data #DataEngineering #Analytics #DeveloperTools #dbt
135
“I was never paying for the code. I was paying for the understanding. And some people are about to learn this lesson the hard way.” Josh Wills closes Data Renegades Ep. 13. youtube.com/watch?v=gmD1hxW3… #DataEngineering #AgenticCoding
10
“Does Claude need an emotion chip so it can feel shame?” Josh Wills, unprompted, on how engineers build judgment from scars and what the equivalent is for AI agents. youtube.com/shorts/6Htkjf4V9… #AIAgent #ClaudeCode
89
“Law school is a pie eating contest where first prize is more pie. And that is true of engineering management as well.” Josh Wills on why he came back to IC work. youtube.com/shorts/bTAcwf6uV… #EngineeringManagement #DataEngineering
16
Van Halen buried brown M&Ms in their rider as a canary test -- if the M&Ms were wrong, the pyrotechnics were wrong too. Code used to work the same way. Now it does not. Josh Wills on Ep. 13. youtube.com/shorts/-y7pWFcT5… #DataEngineering #AgenticCoding
9
Josh Wills and Claude deleted 120 valuable synthetic datasets while working in tandem. “You let the panic in for about five minutes. Then you get to work.” Data Renegades Ep. 13. youtube.com/shorts/q6Ia4ubqn… #DataEngineering #AgenticCoding
1
1
28
“You’re here to butter toast. Your job is to run my benchmarks. It is not to interpret things because you don’t have all the context here.” Josh Wills, 18 months into serious agentic workflow use at a frontier lab. youtube.com/shorts/IbyjGm9_g… #AgenticCoding #AIAgent
19
Josh Wills coined the 2012 data scientist definition. Now he is IC at DatologyAI building trillion-scale LLM pretraining pipelines. His calibration from 18 months with agents: execution yes, interpretation no. youtube.com/watch?v=gmD1hxW3… #DataEngineering #AgenticCoding
2
47
Netflix experiments took a week to run and failed half the time. Toby Mao migrated them to Spark. An hour. The infrastructure instincts from problems like this became SQLGlot. youtube.com/shorts/eG2hym9Q3… #DataEngineering #ApacheSpark #DataInfrastructure
12
"Don't get lazy." Toby Mao on what senior engineers should actually do with AI productivity gains. Hint: more benchmarking, more variation, ruthless review. youtube.com/shorts/kYrhHolPY… #AIAgents #SoftwareEngineering #DataEngineering
7
"nine of the 10 things I tried didn't run faster, but on the 10th attempt, while watching my cooking show, I was able to get significant speed ups." Toby Mao on AI-assisted exploration. youtube.com/shorts/0gcfn6qeu… #AgenticWorkflows #DataEngineering #AIAssistedDevelopment
11
“Everyone is a member of technical staff. Your role is to find impact and then automate it and then go find impact again.” Maxime Beauchemin on burning human tokens. youtube.com/shorts/WVHx9JytK… #AgenticCoding #FutureOfWork #AIAgent
3
“I started using cloud code and I was like, my God, this is changing everything.” Maxime Beauchemin on the Claude Code moment and 12 TMUX tabs that became Agor. youtube.com/shorts/oq4DVfN3R… #AgenticCoding #ClaudeCode #DataRenegades
25
"My experience at the big companies definitely set me up for a disadvantage." Toby Mao on building for petabyte-scale problems when most companies just use Snowflake. youtube.com/shorts/VRP6PMo3p… #Startups #DataEngineering #FounderLessons
23
SQLGlot v30 dropped on the day we recorded. They used Claude to refactor Python into MyPyC-compiled code, got rid of Rust, and hit 5x speed improvements. New episode with Toby Mao. youtube.com/watch?v=cRECaz2P… #AIAgents #PythonPerformance #DataEngineering
75
“I cannot see what you’re doing with your agents. I cannot see what I’m doing with my agents.” Maxime Beauchemin names the single-player default of every agentic IDE. youtube.com/shorts/rGX1D9sbS… #AgenticCoding #MultiAgent #ClaudeCode
20