Staff Engineer at @datadoghq, Ex-@github, Ex-@facebook
Joined February 2008
- Tweets11.7K
- Following18
- Followers3.1K
- Likes31
Pinned Tweet
Where Relay Modern was born, 2016 — also pictured: React (as Fiber was being born), Flow, GraphQL. Quite possibly my favorite time at FB.
Open-plan offices aren't always bad.
ALT Seating map at Facebook's MPK campus.
"latest information about the compatibility of Brother products with macOS Golden Gate 27. We update this list regularly. If your machine's model is not listed, there are no plans for it to support macOS 27."
😢 My printer is now obsolete.
support.brother.com/g/b/osco…
Meanwhile, my prompts look like "Remove the flicker. Make no mistakes."
Very few will care about the actual reinforcement learning details here, but watching me grope around for understanding in the chat log might be of broader interest: chatgpt.com/share/6aad4252-f…
I have often noticed that our estimated Q values are higher than the observed returns, sometimes substantially so, and that can’t be good for performance.
For policy decisions, only the relative values matter, but an offset negatively impacts bootstrapping, so I was excited to see this paper on Relative Value Learning:
arxiv.org/pdf/1901.09732
After struggling a bit with “the Banach space of bounded antisymmetric pairwise functions“, I realized all that is essentially going on is subtracting two value functions. Given this framing, Chat was able to simplify the machinery into a particularly elegant form:
Subtracting the mean TD error from the individual sample TD errors gives all the benefits of relative value learning. Summarized as "relative Bellman regression is TD-error centering".
Unfortunately, while you can create examples where this should be very valuable (someone should write a proper paper on it!), it didn’t actually improve performance on my tasks.
However, I think this has usefully narrowed down what is actually happening. Bellman iteration naturally corrects towards a correct absolute value, but only when the bootstrap values are also training targets. With Q-learning, you are often / mostly bootstrapping from a max-action that was not actually taken, so it never sees any downward Bellman pressure.
This is consistent with another result I have noted: learning state value functions offline from a frozen policy without actions doesn’t seem to suffer from any value overestimation.
And the winner for "Most Annoying Unnecessarily Incessant Autoupdate" goes to... Microsoft AutoUpdate!
Greedflation: YouTube Premium price hike just announced: 14.3%
Follows two years after the last price hike, of 16.7%
The pattern is repeated price ratcheting far excess of inflation.
That means I can't just have a normal profile repo: I'll have to make a profile repo with a big fat notice on top telling people who wanted to find my dotfiles that they have to go somewhere else. This might sound like I'm overblowing it, but my dotfiles repo has 1.2K stars.
Like Michael, I've been a SuperDuper user since 2006. The one backup I've done with the new version 4 so far went smoothly. First impressions are good.
Just had a skim and it looks interesting
In the tradition of @fabynou 's game engine books, Bas Smits (on X?) has made the comprehensive book on the Commander Keen games. Available free online and in print: forgottenbytes.net/
This is one of several walk-backs in Golden Gate that has me eagerly looking forward to the next macOS release. Sad that Tahoe was so disappointing in the first place, but still happy some of the errors are being unwound.
Unsurprisingly, Kagi mentioned a million times in this discussion news.ycombinator.com/item?id…
My favorite iOS keyboard now ships with a privacy-invading cloud sync feature that nobody asked for.
Twitter engagement hacking: the "notifications" that somebody famous who you don't follow posted something.
AI slop PRs flooding popular open source projects are making it hard for real humans to be heard in all the noise. Super annoying.
Minor correction: "Copilot customers are now paying the price"
