Staff Software Engineer @datadoghq ex @Bluecore ex @facebook ex @yelp ex @CitynetLLC Recovering Network guy.

NYC
Joined March 2022
coxley.dev retweeted
Saying there's code so complex you can't understand it is like saying there's sentences so English you can't hear them.
235
160
39
6,603
269,318
coxley.dev retweeted
To those linking to pieces that are obviously 100% LLM generated: do you... actually read them? I can't make it through the FIRST PARAGRAPH without my brain hitting its LLM abort; did you actually READ the WHOLE THING? Fine, you no longer write, but do you also no longer READ?!
45
49
15
584
55,419
Goodhart's Law from a different angle: you can't legislate caring about the work. If you make a rule that describes what you'd get from someone who cared, that doesn't make them care. They'll just do what's needed to meet the bar. With agents it's worse because they can't care.
3
15
2
93
4,013
coxley.dev retweeted
Mitchell Hashimoto
@mitchellhAug 25
The Ghostty memory usage memes are dead with the upcoming 1.4 release, much to the dismay of internet trolls everywhere. ~10x less memory for a visible window. ~450x lower for a non-visible window. Very reasonable compared to other terminals with 10 tabs. This was the result of a lot of incremental hard work, but there were two big wins. One I talked about before which was scrollback compression. I want to note that these charts do NOT include scrollback, so with scrollback the results are more dramatic! The other big win is that we free almost all GPU resources when a window is fully occluded (covered by another window, minimized, etc.). We triple-buffer our screens so for large screens this has major savings. Those account for a large chunk of the improvements, but we also had dozens of incremental 1% wins here, a few KB there, etc. throughout the 1.4 cycle. And there are more coming! I noted before that Ghostty's memory waste was all in the app layer and we can see it here. libghostty has always been very tight in comparison and I've shown comparisons to other embedded terminal libraries elsewhere. And importantly, the charts below use the same grid sizes with a very small +/-2% difference due to differences in how cell metrics are calculated between terminals.
87
74
18
2,245
201,295
coxley.dev retweeted
The hardest part of software engineering is convincing people that a simple solution is actually better than an impressive one
39
96
12
1,034
37,437
coxley.dev retweeted
Megan Reynolds 👩‍💻
@meganreynoAug 24
WAL Street
58
157
41
1,694
132,549
coxley.dev retweeted
Replying to @alvindrakes_
there doesn't have to be an end goal you can just make useful things for people who want to use them, and live a happy life
33
222
35
2,028
274,313
coxley.dev retweeted
Tangentially: the only reason OP is able to "stop looking at assembly" is because a bunch of other people still do. There are compiler codegen bugs constantly, but OP never knows about them because other people who do read ASM find, report and/or fix them before they cause too many downstream problems. Another way to say this is that the "compiler" does not "get it right". A ton of people, constantly reading the ASM that the compiler outputs, work together to "get it right".... eventually :) So the proper way to look at this is, if you've stopped reading ASM, that's not because codegen was solved, but rather that you have implicitly decided to outsource to other people the problem of keeping codegen working reliably.
I might never look the source again. It was nearly a half century ago that I stopped looking at assembly because I trusted the compiler to get it right. This feels like that.
76
262
18
3,020
120,377
coxley.dev retweeted
Mitchell Hashimoto
@mitchellhJul 27
Btw this is why computer science matters. Yes, you can self teach it if you really want to (I personally wouldn't have had the discipline, so college was a way better force). However you learn it, the theoretical fundamentals matter and get applied to work every single day.
Mitchell Hashimoto
@mitchellhJul 27
If you're interested in "how" or "why": the major culprit is that each row in Alacritty has 32 bytes of metadata and each cell is 24 bytes. In Ghostty, every row and cell is represented by exactly 8 byte each. How do we do this? The first major culprit is styles. Alacritty stores the full cell style alongside each cell (foreground, background, underline, etc.). Ghostty stores a 16-bit style ID and de-dupes all styles into a look-aside custom reference-counted hash table. MOST cells are unstyled, and when there are styles MOST styles are shared, and when styles are shared MOST are repeated in a run (multiple cells with the same style in a row). Put this all together, and the tradeoff on compute to access it doesn't even end up being slower. Next, codepoints. Alacritty stores multi-codepoint graphemes (like, Emoji) by having an 8-byte nullable pointer to a `Vec`. This hurts doubly: (1) its almost always null (because multi-codepoint is rare) yet you pay an 8 byte cost on every cell and (2) every multi-codepoint grapheme triggers a heap allocation to make that Vec. Ghostty stores single codepoints inline, but multiple codepoints in a look-aside table. The memory for this table uses a custom bitmap-tracked chunk-allocator (since grapheme frequency follows a measurable curve we calculated by scanning various online texts). The presence of graphemes is marked by a 2-bit content tag in our packed 64-bit cell. To keep the key small in the hash table, its limited to a 16-bit unsigned int that is an offset from a base pointer. Okay, the astute systems programmer will quickly notice there are a lot of 16-bit integers and ask: so this is all limited to a max of ~65K values? Nay. We maintain our grid using a linked list of contiguous ~400KB memory chunks (which themselves are in a memory pool using a custom allocator to speed up alloc/free). Each memory chunk is limited to 2^16. If/when we reach a limit, we move to the next page. In practice, this really doesn't happen except under pathological cases... the important point is we handle it. Lots, lots, lots more details, but thats a 10,000 foot view. These things alone account for ~95% of the difference of our uncompressed vs. Alacritty's uncompressed memory usage. (Theres also a reason why Alacritty's data structures aren't trivially compressable but thats a whole other topic)
46
161
13
3,006
276,807
coxley.dev retweeted
Mitchell Hashimoto
@mitchellhJul 20
Replying to @_2c2c2 @Malix_Labs
I'm willing to engage with anyone. And the algorithm lifts this so others can too. I try to combat engagement bait with an attempt towards rational discourse as much as I can, because I don't want people to think that professional people actually work like memes do. When others don't respond in kind, it naturally reveals their true clown nature. That wasn't the case this time, again speaking in generalities.
1
21
976
coxley.dev retweeted
Mitchell Hashimoto
@mitchellhJul 9
Ghostty is getting automatic scrollback compression, resulting in 70 to 90% less physical memory usage. It happens incrementally when idle, so it had no measurable effect on IO throughput. I'm not aware of any other mainstream terminal that does this. Demo video below! The gains let us increase the default scrollback limit from 10MB to 50MB, because on average a full scrollback will still compress smaller than the prior limit. More history, for free. ("Unlimited", disk-paged history is on the roadmap too) Let's talk about cool implementation details, cause this was fun. First, the data structure and memory layout ("PageList") I wrote two years ago finally pays off! One of its traits is that screen memory is backed by a linked list of page-aligned, page-sized (or page-multiple-sized) blocks. Because each block is page-aligned and page-sized, we can use madvise to discard its physical backing while keeping the virtual address space reserved. Compressed pages therefore disappear from resident memory, but decompression is still guaranteed because the address space remains valid and we simply fault new pages back in as needed. We use the same trick for our memory pools, too. Unallocated pool pages don't count as resident memory, saving another couple of MB per terminal. This functionality is also available to libghostty-vt consumers via new `ghostty_terminal_compress` APIs. The consumer decides when the appropriate time to compress is and the APIs advise on compressability.
67
92
12
3,029
167,668
coxley.dev retweeted
Mitchell Hashimoto
@mitchellhJul 6
Ghostty is now indisputably the fastest terminal emulator at IO throughput, by a very large margin. On ASCII, Unicode, and CSI tests, Ghostty is more than 2x (double!) faster than any other leading "fast" terminal. These changes are directly in libghostty, too, so everyone wins. `time cat 150MB_ascii.txt`: - Ghostty nightly: 575ms - Ghostty 1.3.2: 1.5sec - Alacritty: 1.2sec - Kitty: 1.7sec - Warp: 3.8sec - iTerm2, Terminal: stopped after 60s `time cat 150MB_unicode.txt` (mixed languages): - Ghostty nightly: 536ms - Ghostty 1.3.2: 1.22sec - Alacritty: 1.05s - Kitty: 1.35s - Warp: 3.4s - iTerm2, Terminal: stopped after 60s `DOOM-Fire-Zig` (an IO test): - Ghostty nightly: 842fps - Ghostty 1.3.2: 532fps - Kitty: 485fps - Alacritty: 593fps - Warp: 577fps - iTerm2, Terminal: 60fps (yes, 60) To quickly address the "cat speed doesn't matter" naysayers: this is a direct test of how many bytes/second you can push through a terminal. It doesn't cover just "read big file" but also "how much can a TUI do". The tests above test various shapes of inputs (plain ascii, unicode/wide chars, csi-heavy loads, etc.). IO throughput is incredibly important. Most of these improvements apply to libghostty-vt consumers too, so any libghostty-based terminals will instantly see huge throughput improvements by simply upgrading (ABI compatible). I'll cover the exact improvements in a blog post in the future. These results are the result of 6 separate optimizations.
143
123
40
3,550
269,489
A common problem on large projects is solving problems on the wrong layer of the stack because you’re afraid to change the layer below. e.g., adding checks and retries because you don’t know why the server code isn’t transactional. This is how simplicity dies and LLMs will very happily let you do this. Pick your platforms well.
9
22
4
379
15,232
Related, but people often get these wrong (too): - The fallacy of composition - The fallacy of division For example, your house is “safe” because you build with “safe” bricks. Or, that to build a “safe” house your bricks should be “safe”. In fact, a house is “safe” when it is designed to be “safe” (even if/when the bricks are not)! Correctness is a design problem. And this should not be new to us as programmers. It’s the end to end principle. Probably the most important principle in designing distributed systems. And how resilient systems like the Internet work (a reliable system out of unreliable parts—being more reliable than a system that is only reliable if all the parts are reliable). For these reasons, this is also the basis on which TigerBeetle was designed: To be safe, even if a programmer gets some logic wrong (assertions will trip), or if the filesystem writes/reads to/from the wrong sector, or disk firmware lies about fsync (cf. Protocol-Aware Recovery for Consensus-Based Storage).
People often get this wrong: But per systems thinking, correctness is not a { language, static allocation, testing, etc. } problem. Correctness is a design problem.
3
15
104
8,775
In other words, correctness starts when you lift up your eyes: - You don’t “rewrite” for correctness (too late). - You don’t “not allocate” for correctness (not enough). - You don’t even “test” for correctness (again, too late). Rather, before all these (and more than these!) you treat correctness as an end to end systems problem: You “design” for correctness.
1
3
45
2,152
coxley.dev retweeted
Mitchell Hashimoto
@mitchellhMay 20
This is why PR diff speed matters. This isn't a dunk on GitHub specifically, because GitLab, Forgejo, etc. are all equal or worse. But this is the kind of thing that drives me nuts, because this is a core workflow and its slow enough I literally take my hands off the keyboard. Btw, when my mouse jiggles on the left, its because the page is literally skipping frames and I'm instinctively shaking my mouse to see if it'll respond. And on the keyboard input you can literally here me finish typing before a letter even shows up. For someone like me who is an expert at these tools, my brain navigates the tool dramatically faster than it can keep up, and that is not good. The tool should not get in the way.
109
128
26
3,161
367,800
coxley.dev retweeted
Replying to @Krongggggg
Some people have been saying this since the beginning, that the only way for Zig to succeed is to get a big tech patron. By now they've been saying this for almost 10 years :^) People just don't realize how efficient a tightly run open source project can be. So yeah we do need donations, but so far things have been working out.
3
63
1,738
coxley.dev retweeted
Replying to @GergelyOrosz
The hubris of assuming people will care to read something you couldn’t bother to write.
9
86
7
797
63,070
So much bragging about LoC changed and PRs merged. Just pick an ambitious project and brag about it when it’s done. The PRs aren’t the goal.
10
12
153
4,836
coxley.dev retweeted
us: we are struggling to figure out the best way to use coding agents, we don't have clarity yet everyone else: our team is moving at speeds unheard of, all our PRs are ai generated, we've cleared 6 years of backlog man we must really suck huh
165
99
22
3,058
185,014