
Academic and Data Scientist. AI / ML / Forecasting / Economics. PhD Economics from @LancasterUni. Views my own.
United States
Joined November 2015
- Tweets353
- Following617
- Followers141
- Likes2.2K
Simon Spavound retweeted
I wouldn't characterize the state of enterprise AI adoption this way. Broadly, I would say there are two variables in measuring the ROI of AI in an organization:
1. The task you are automating or augmenting
2. Does this business have any fucking idea why they are successful?
#1 The task.
Let's take a very simple example. Closing out a support ticket from a customer. Most companies know how much this costs by channel - say, $40 to pick up the phone, $15 to answer it by email, $5 by chat. This is calculating using things like the agent salaries, the headcount that supports them, the technology they use, the facilities, and any travel and training expenses
Now imagine you can use AI to *successfully* answer a question and the AI company you use charges you 70 cents for that. OK, you really clearly know the ROI of AI in this specific, bounded, use case.
Consider other tasks. The velocity with which you ship new products. A bit fuzzier but not a crazy task to determine whether 1) Developer productivity has increased 2) Whether that is bending the slope of your revenue curve upward.
Farther out on the curve. Being able to now generate 21 different versions of a marketing campaign's creative using your brand design assets. Making 40% of analytics requests self-service instead of being stuck in a queue for humans to answer. Quite a bit messier to measure direct ROI.
#2 Does the company have it's shit together?
Do they know what is actually driving the business forward? I think that the answer for many businesses is actually, No. Do they reliably know who their top performers are? Do they know the actual 2-3 inputs that drive revenue growth? Not only do they know them but can they accelerate them? If you asked 10 people at the company what "the most important thing" was for the business, would they give the same answer? Again, I think the answer is mostly No.
If you do not deeply understand your business, what is driving it, and who is driving it, it is quite hard to know whether AI is helping you.
----------
TL;DR. For some tasks it is quite easy to measure the ROI. For others, less so. For some businesses who have their shit together, it is quite easy to measure the ROI. For others, less so.
Knowing that its called "negation-first reveal" does not stop my involuntary shudder with the amount of times I see it now.
50
AI lacks the empathy with the intended audience unless you are super specific (and I think still requires human understanding) - the AI is writing it for you and your requests - resulting in identikit presentations which have not been created for a specific purpose
I think part of the problem is that to write well (papers, slides, it doesn't really matter) you need a really good model of the intended (human) listener and to understand the limitations of the transmission medium, and AI doesn't really have this yet.
1
43
I have this same conversation with students: I can tell when slides have been AI generated as everything is in words - good slides have a logical flow that allows you to use less words. LLMs have mostly been encouraged to be verbose - the opposite is required here
my hypothesis for why AI-generated slides for technical presentations are so awful:
people who don't have a lot of experience presenting think that to make a talk you just put all of the stuff you want to talk about on the slides, then you go through each slide and say the stuff the slide says.
how you're actually supposed to prepare a presentation: you decide what to say, write it out as a rough script, then put maybe 3% of the text on the slide, plus a bunch of visual aides (e.g. plots that only include data you're planning to discuss).
AI-generated presentations are mostly the former. they have a ton of stuff the presenter never mentions, and the stuff that is most useful for the presenter is absent or not highlighted. they include enough content for a presentation 10 times longer than the actual presentation is supposed to be.
maybe try doing it the other way: first write the script for what you actually want to say, and make sure you can say all of it in the time you have. then prompt the model to create the slides based on the script. ask it to use as little text a possible, and a minimalist visual style rather than the claude house style. you can thank me later (I accept both venmo and zelle).
34
Simon Spavound retweeted
my hypothesis for why AI-generated slides for technical presentations are so awful:
people who don't have a lot of experience presenting think that to make a talk you just put all of the stuff you want to talk about on the slides, then you go through each slide and say the stuff the slide says.
how you're actually supposed to prepare a presentation: you decide what to say, write it out as a rough script, then put maybe 3% of the text on the slide, plus a bunch of visual aides (e.g. plots that only include data you're planning to discuss).
AI-generated presentations are mostly the former. they have a ton of stuff the presenter never mentions, and the stuff that is most useful for the presenter is absent or not highlighted. they include enough content for a presentation 10 times longer than the actual presentation is supposed to be.
maybe try doing it the other way: first write the script for what you actually want to say, and make sure you can say all of it in the time you have. then prompt the model to create the slides based on the script. ask it to use as little text a possible, and a minimalist visual style rather than the claude house style. you can thank me later (I accept both venmo and zelle).
You could extend this to "a system can never be held accountable" a la @dsquareddigest. My own view is that absolute statements should be taken with a pinch of salt and this was a normative statement of what should be done, rather than what will be.
I hate this slide every time it comes up, and now from CSAIL
This has been false since before the slide was made. Automated systems make management decisions all the time (trading, logistics, customer service, insurance) & pretending they don't actually launders responsibility.
1
27
Simon Spavound retweeted
I hate this slide every time it comes up, and now from CSAIL
This has been false since before the slide was made. Automated systems make management decisions all the time (trading, logistics, customer service, insurance) & pretending they don't actually launders responsibility.
As an example of code moats dropping - AnyPS5 was started in August and has now got ~88% coverage of all PS5 system calls. This would have been a multi year undertaking not so long ago
1
3
163
I think this says a larger amount about what value companies measure from AI usage. Agree on the comparison with salary costs, but consider from a CFO perspective $1000 * number of employees is a significant expense if you cannot show ROI
37
Where is the thinking about REA (Reverse Engineer Anything), or the sudden ability to replicate existing software. A few of my own thoughts: (1/9)
1
46
Media industry radically changed with streaming and the collapse in physical sales. I am not so naive to believe that piracy was the only cause of these changes. (8/9)
1
1
7











