founder in sf. orbital, unboxed, dailymile. did you see the runner’s hair?

san francisco
Joined February 2007
are they? the tension in agent autonomy is that giving them a longer leash requires watching them closely first. trust comes from seeing where they fail, tightening harness, repeat. open to an agent being the reader of that transcript though ofc.
The days of reading transcripts up close are over.
1
72
one more small prompt climbs towards another false summit day fades, i await
60
ever wondered whether most software never seems to get better because writing code takes time or because most teams don't know what good looks like or what's worth building? now that code basically writes itself, i think we have our answer...
The speed at which a product is developed doesn't inherently make the product better or worse. The number of commits doesn't make the product better or worse. The number of people or agents working on it doesn't make it better or worse. The number of hours you’re pouring into it doesn’t make it better or worse. Working on a weekend, or late into the evening, doesn’t make it better or worse. Talking about these things sounds like it’s talking about the product, but it’s not talking about the product. Those are all development metrics and styles of work. They don't speak to the product itself. They're all back of the house. Important to those making the work, but not inherently important to the people using the work. Development metrics and style say nothing about fit. They say nothing about quality. They say nothing about clarity. They say nothing about usability. They say nothing about utility. They say nothing about performance. They say nothing about how customers feel when they use your product. It's like reviewing a restaurant by talking about how many cooks were in the kitchen, how long the shift lasted, how many orders they fired, or how late they stayed cleaning up. None of that tells you about the meal, the product, the customer’s experience. Mistaking making for the thing you've made is like confusing the kitchen for the meal or the blueprint for the building. They're connected, but they're not the same thing. One of them matters a whole lot more than the other.
142
this was always the bar for product eng. what's changing is the speed of delivery, which puts downward pressure on product decisions toward the IC. (also, PMs are converging from the other side, they can build now too!)
“taste” isn't enough… The three tiers of agent powered developers: 1️⃣ Can implement products: Can use agents to build code, with great tests. 2️⃣ Can implement products, 𝘄𝗶𝘁𝗵 𝗴𝗿𝗲𝗮𝘁 𝘁𝗮𝘀𝘁𝗲: Your feedback, which can keep up with the pace of code creation, lets you ship *good* products. 3️⃣ Has a deep empathy for their market and user and tons of surface area with them to stay up to date: Taste isn't enough! You need user empathy. Everyone keeps saying “taste”, to define a company's value when code is cheap. But “taste” is incomplete. User understanding, subject matter expertise, etc. These things are no longer the domain of just Product people, they need to be internalized by all and orgs should be designed to max out this surface area. I've said the difference between a good data scientist and a great data scientist is domain expertise: “When you have an idea how the business works, you can make more complex assumptions and develop hypotheses further out from the baseline. Bigger leaps, adequately tested, help you move faster and find unique information.” This now applies to software engineers.
101
was thinking about this today: you have an agent whose final responses sometimes need semantic classification during a session. it requires model judgment, but not extended thinking, and should add little or no user-visible latency. weighing a couple approaches.
1
55
the main model could label its own response through hidden metadata requested by an instruction added to the system prompt. no extra model call, but relies on compliance and there's a small risk the instruction affects other response behavior.
1
23
or, after generation, make a second model call with extended reasoning disabled to classify the final response. you could do this sync or async, but would probably want a deterministic prefilter to avoid an extra call for every final response. (a local model might work here too)
10
agent runs often fail after expensive model calls or tools with real-world side effects and this problem gets amplified on serverless. i built kassette to make agentic workflows durable without needing to add a workflow service or sql db.
1
1
67
most stacks already have half of durable execution: a queue or job runner that retries work. kassette supplies the other half by journaling completed steps to object storage (or an fs). retrying will replay journaled work then continue at the first unfinished step.
1
1
32
in theory this makes a lot of sense, but it requires that sonnet actually knows when to ask for help. in my experience, current models do a poor job of judging their own competence (RL rewards confidence and task completion), a weaker model even more so. weaker models will need to be fine tuned for this use case.
A few patterns we frequently use with Fable 5: Use Fable 5 as an "advisor." An executor (Sonnet 5) calls Fable 5 for guidance. Most tokens are billed at the lower executor rate.
1
72
managing business and product outcomes across the huge number of teams in larger orgs has traditionally been too much work. i suspect agents end up both requiring this context and freeing people up to actually do it.
Just coming off of meetings with a couple dozen enterprise IT leaders discussing AI agents. Here are a few of the common themes that stand out: * Lots of conversation that you have to solve an operating model challenge to get the full benefits of AI. Most companies have orgs that have always operated in siloes; but agents are most effectively when they are tied to a process, which often cuts across these siloes. So the big question is how do you start to deploy centrally managed agents that can work across organizational boundaries. Who manages these agents? How do they get deployed and adopted? * Data fragmentation remains a major issue for most organizations. As long as data remains highly fragmented and not in standard formats, or data is not available to the right people and agents, enterprises are dealing with issues around being able to get answers from agents that are accurate or that conform to their business practices. This cuts across both systems with structured data (product metrics or revenue figures) and unstructured data (product roadmap or customer contracts). * Clear sense that companies need to figure out what their core data moats are going to be in the future. If everyone has access to roughly the same superintelligence from the various models, then the context that you feed the models becomes proprietary value in the future. Capturing this data and getting it into a format that agents can use becomes very important. * Everyone is trying to figure out the right metrics to manage to for AI adoption. General consensus that tokens are not the right metric per se, and people leaning more toward business outcomes (in an ideal world). For business outcomes (like more revenue or more shipped product), though, you have to get close to each individual workflow to figure out if it was successfully transformed with AI so it’s harder to manage top down. * Growing view that enterprises are going to live in a multi-model world. Lots of interest (though early in actual adoption) in layers that can route workloads to different models (frontside or open weights) for cost or performance reasons. Also enterprises are trying to figure out what things do you give to the models directly vs. what do you separate as horizontal systems and context so you can swap any system in and out. * Talent for driving AI adoption and implementation still remains a major issue and topic. Many view it as something you necessarily have to train for internally due to a shortage of talent being trained on this in the outside. As an aside, this feels like it remains a huge opportunity for those that get very good at deploying and management agents in an enterprise since most companies are looking for these skills. * The best use-cases for AI tend to be those that fundamentally change the work being done instead of just replacing an existing process and doing it more efficiently. Companies are working through their versions of this individually because it’s different per industry, but this often remains both the most exciting and higher upside uses of AI. Many more topics discussed recently, but overall it’s clear that there’s a ton of change going on with much more to come.
1
37
I made a giant javascript powered flipdisc display for my office with @korevec. Here is a little bit about flipdisc displays, and how to make your own! - a thread 🧵
26
89
9
1,068
138,678
Ben Weiner retweeted
Everyone knows the internet is increasingly full of noise. But it’s also filled with so, so many good reads (😊). I’m thrilled to announce Smashing, a new app that helps you find the gems. We’re building a trusted community to curate the web. Join us at Smashing dot xyz 🌐 💎 1/
10
9
6
53
10,885
after almost 8 years in-house at apple, my wife has joined DLA's patent litigation group as a partner. excited for her big change and new challenges!
“The San Francisco office is pleased to welcome Jessica to the firm and excited about the impact that her IP litigation experience will have on our robust IP litigation and patent appellate practice,” said Jeff Tsai, managing partner of our SF office. dlap.pr/czfJjEEiuA7nc9HW6
3
203
Ben Weiner retweeted
When you build products for developers, you’re a chef cooking for chefs. I wrote about why you should care: kathykorevec.substack.com/p/…. Now don’t burn the sauce! 👩‍🍳
8
20
2
227
64,929
nodding my head the whole way through, espec: "I’ve seen many developer tools come to market that seem to rush to creating a website, whereas a plugin is much more appropriate." it's so easy to get caught up in what you're building and lose sight of who you're building it for.
I've worked in the developer tools space for over a decade and finally wrote down my favorite principles for shipping great developer experiences. Here is the first article in a series: Minimize switching contexts. Let me know what you think! 😀 kathykorevec.substack.com/p/…
1
2
616
having designated mask-only hours seems like a really good idea for our current times! surprised we haven't seen this more
This Sun, Dec 4, we'll be offering our monthly mask-required hours from 11–1. These hours are offered to better accommodate immunocompromised individuals & those who prefer to visit when everyone is masked. If that's you, we hope you'll stop by! fryemuseum.org/programs/mask…
1
the best remote teams are highly effective async regardless of whether or not they’re fully async
Loom is to Zoom as a memo is to text chat. The former is a method of summarization, redaction, rational analysis. You shouldn't write a memo when you need to quickly send an idea, but you shouldn't expect a collection of chat messages to approximate the analysis of a memo.
1
2
is it necessary to understand the “how” if we see the causation and have a solution (mrna vaccines)?
This quoted post is unavailable.