@ApacheArrow / Databases / Compilers. (past @SDFLabs, VoDa, @Spotify). Rust/C++/TLA⁺🇧🇷 → 🇸🇪 → 🌎

Joined June 2008
Felipe O. Carvalho retweeted
ok I think the main thing we've learned this month is that computer programs get faster whenever they're ported to another programming language, and all we need to do is keep porting back and forth until every program only needs one CPU cycle to finish
44
76
8
2,234
28,340
Felipe O. Carvalho retweeted
Any startup that hires enough ex-Googlers ends up with an ad hoc, informally-specified, bug-ridden, slow implementation of half of Google infra
New on our Engineering blog: Creating in-house source control. In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale. Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: anthropic.com/engineering/re…
44
103
17
3,716
166,957
Felipe O. Carvalho retweeted
I can’t tolerate people being cruel to everyone else dealing with their emotions and the implications of ai changing everything about their work (and life!) have some decency man
22
29
1
517
15,550
Felipe O. Carvalho retweeted
Ok, now I understand why people want to get rid of unit tests. Apparently they overrely on mocks and there aren't proper boundaries between the different parts of the system. If that's the case, go ahead! You should have gotten rid of those before AI. :D
32
23
7
531
31,443
“After everything I’ve seen throughout my career, I can confidently say this: scaling too fast can be just as dangerous as not scaling at all. It's important to grow smart and strategically.” - Kyle, 23, Junior Analyst at McKinsey
42
77
8
2,593
78,854
Felipe O. Carvalho retweeted
Replying to @filpizlo
Do you really think that writing a program in assembly makes it 17x faster than writing it in Rust? I wish compilers had that much potential 🤣
2
1
46
1,955
Felipe O. Carvalho retweeted
4
2
66
2,482
Felipe O. Carvalho retweeted
With @_Felipe talking databasemaxxing
4
1
118
2,436
Felipe O. Carvalho retweeted
Replying to @chrsnjk @denbvk
Proper handling of NaN from std::clamp makes it require more instructions. Reasonable for a standard library function, but applications might want to enforce NaN-freedom as a property ensured by the callers.
1
12
530
Felipe O. Carvalho retweeted
This will lead to the same result we always got with human-written code. To satisfy the metric, the programmer/agent will spread control flow complexity out into different functions. It's all still there, but harder to understand because it's spread out. xitter.cf/asidorenko_/status/209…
> "Agents produce overly verbose code that's hard to read and understand" Try setting a Cognitive Complexity rule in Biome. This metric is specifically designed to test how hard the code is to understand
16
16
4
555
51,390
Felipe O. Carvalho retweeted
Most developers know that avg latency is misleading and they should use percentiles instead. They also know that if 90tile response time is 200ms, then 10% of the requests had response times higher than 200ms. Well, the last sentence is false. Or rather, true in theory. The only way to calculate true percentiles is by sorting and scanning all the data. Since this often means many millions of datapoints, we actually don't do that. We approximate. With histograms. How does that work? You create buckets of values: 0 to 0.05, 0.05 to 0.25, 0.25 to 1, 1 to 5, 5 to infinity. Then you accumulate: number of data points, sum of values and the number of data points that fall in each bucket. Now, if I ask you for the 90tile value, you start going over the buckets from lowest to highest latencies, and add the number of points in each bucket. Once you go past 90% of the data points, you know that your 90tile value is in this bucket. Then you give either the middle of the bucket or the upper bound as the estimate. Note that you don't know what % of the data is higher than the estimate. And also note that the quality of the estimate really depends on how the data ends up distributing between the buckets. It isn't uncommon for ntiles 90, 95 and 99 to all fall in the same bucket (if your distribution is "heavy tail"). And sometimes all or most of the data falls in one bucket and you can tell nothing. I recently had a case where we noticed the 99tile of response time for a specific request type was 10s. This was both high and suspicious - we have a 10s timeout in some places. But we couldn't find any request that hit these timeouts in our logs. Super weird! Until a smarter colleague realized (by looking at sample traces) that 10s was just the lower bound of the top bucket. The requests actually timed out at 60s. This allowed me to find the timeout they actually hit and fix the issue. So the lesson is: Averages are misleading. But percentiles are usually approximate and can be misleading too! Be careful not to trust the data blindly. #StatisticsSaturday
29
38
6
608
176,144
Felipe O. Carvalho retweeted
Replying to @mitsuhiko
Non-ironically: /goal reduce the binary size by strategically using dyn traits and inline(never) annotations in cold code paths Binary size is proxy metric for build times. Inlining creates work for LLVM which translates to binary size. Reduce binary size, get fast builds.
1
6
393
Felipe O. Carvalho retweeted
C++ is like jazz. It's insanely technical, with such breadth and depth. Most people, if they do like it, only prefer a subset of it. Like jazz, the deeper one goes into C++, the more the journey becomes a descent into madness. The madness is welcome and celebrated.
40
43
10
557
20,812
Felipe O. Carvalho retweeted
Replying to @mycoliza
Software Engineering is solved. I should be able to update the SHA256 of a file after an update to the first file byte without recomputing the hash for the entire file. /s
2
1
39
2,432
Felipe O. Carvalho retweeted
Replying to @caballerobrah
It would be funny, if not tragic, seeing engineers writing end-to-end tests that depend on network services and then being surprised when these tests become flaky. The low probability of failure of each test create a high CI failure probability when combined.
2
1
11
646
Felipe O. Carvalho retweeted
I've developed a physiological response to text like this. Eyes absolutely glazing over. It could be a message to myself sent from the future and the words would pass right through me.
47
56
22
1,641
121,879
Felipe O. Carvalho retweeted
Coding used to be the bottleneck and isn’t anymore so now there is a lot of brain power being spent at companies to play bottleneck whackamole while tokens are cheap. Problem: code review Multiple levels of automatic code review of AI written code doesn’t really increase trust, and AI code review still lacks the context behind WHY. So there’s all this effort to document everything by adding MCPs to every context store: jira, confluence, gmail, slack, zoom calls, google docs, whatever. The goal is to get to the first approve/reject recommendation before a human even looks at it. Problem: human time Human time is precious, your teams are overloaded, everyone is busy, and has their own brain split between 15 things, so let's intelligently @ people directly in public channels. Round robin it between reviewers, add leaderboards, daily reminders, and some public shaming. But wait. Someone says- can we just…skip human review? Problem: defect escape Whether or not we got a review from a human (rubberstamp or meticulous), we still need to prevent defect escape. Auto merge approved code to main which deploys to staging. Force it to bake on staging for X hours. Run every test suite against it (nothing new). Have adversarial agents use headless browsers to try and break your stuff. Should probably fix things too? Problem: blast radius This one isn’t new either. If you’re going to yeet out code that no human has looked at, you should probably reduce your blast radius with progressive deployments. Squeeze out a little bit, have an agent watch your error logs and dashboards, set a confidence threshold for rollback vs perseverance. Problem: incident triage If something breaks at 3am, unconnected to an ongoing deployment, you shouldn’t have to wait for a human. Have an agent triage the issue and link to relevant dashboards before the human can roll out of bed to get to their laptop. Problem: conflicts Your PR queue is a traffic jam. Agents will need to be constantly rebasing, resolving conflicts, and regenerating changes as main moves underneath them. Problem: trust boundaries How much rope can we give our agents? Under what conditions can we let yhem act without asking permission? Whose agent is going to be the first to screw up and leak PII? Gotta wrap it up, my toddler is waking up. What did I miss?
20
14
7
292
32,193
Felipe O. Carvalho retweeted
Replying to @justinmk @Neovim
That week when we all moved away from Github was crazy.
1
1
4
602
Felipe O. Carvalho retweeted
Replying to @_Felipe
"It is a myth that a DBMS operates on data on disk". (A mentor told me this years ago, but I'm sure there are many Stonebraker/Gray/Pavlo quotes to the same effect.)
1
11
1,064