Fullstack Developer, Journalist, Entreprenuer and Co-founder of @blockfeedapp, Also Cooking, Tech, Startups, and Video Games
Joined November 2012
- Tweets395
- Following527
- Followers261
- Likes764
Adam May retweeted
I always get asked what my favorite **Open Source** eval tool is. The most impressive one thus far to me is Phoenix
Very permissive license as well.
github.com/Arize-ai/phoenix
Adam May retweeted
I asked here on Twitter who's the best IC product manager you've ever worked with, and @amankhan shot to the top of the list.
Aman Khan is a long-time AI product manager. He's currently Director of Product at @ArizeAI, and was previously at @Spotify, @Cruise, @Zipline, and @Apple.
In our conversation, we discuss:
🔸 What exactly is an “AI product manager”?
🔸 How to break into AI PM
🔸 What separates top 5% AI PMs
🔸 How to thrive as an individual-contributor PM
🔸 Common pitfalls to avoid when building AI products
🔸 The importance of energy and curiosity in product roles
🔸 Much more
Listen now 👇
- YouTube: youtu.be/E_rNotqs--I
- Spotify: open.spotify.com/episode/1FN…
- Apple: podcasts.apple.com/us/podcas…
Thank you to our wonderful sponsors for supporting the podcast:
🏆 @Pendoio — The only all-in-one product experience platform for any type of application: pendo.io/lenny
🏆 @trustVanta — Automate compliance. Simplify security: vanta.com/lenny
🏆 @useParagon — Ship every SaaS integration your customers want: useparagon.com/lenny
Adam May retweeted
(1/7) o1-Preview Time Series ⌚Eval Challenge 🥊
Time Series anomaly evaluations trip up most LLMs. We decided to put the new @OpenAI O1 models to the test.
We actually use LLMs for time series analysis in our @arizeai Co-pilot so this test is important to us. This skill is incredibly useful for debugging time series patterns, correlations and problems in data.
TLDR:
✅o1-preview did actually perform a lot better
❌o1-preview is just too slow for our product right now
❌o1-mini was horrible on this Eval
If you take a look at the data - here's a set of traces you can check out on @ArizePhoenix:
phoenix-demo.arize.com/proje…
If this gets 2-3x faster, we would use it immediately for our time series debugging skill. There is a very real future where we swap in and out stronger models for the “tougher” cognitive tasks.
Results
o1-Preview:
85% of anomalies detected - 110k context window
80% of anomalies detected - 80k context window
95% of anomalies detected - 56k context window
100% of anomalies detected - 30k context window
Claude-Sonnet 3.5:
55% of anomalies detected - 110k context window
75% of anomalies detected - 80k context window
85% of anomalies detected - 56k context window
60% of anomalies detected - 30k context window
o1-mini:
20% of anomalies detected - 110k context window
20% of anomalies detected - 80k context window
25% of anomalies detected - 56k context window
45% of anomalies detected - 30k context window
Tagging relevant LLM Evals folks!
@rown @universeinanegg @ybisk @YejinChoinka
@allen_ai @haileysch__ @lintangsutawika
@hendrycks @markchen90 @MillionInt @HenriquePonde @Shahules786 @karlcobbe @jerryjliu0 @mobav0 @lukaszkaiser @gdb @HamelHusain @sh_reya @eugeneyan @DimitrisPapail
@_akhaliq @JeffDean @demishassabis @jxnlco
@OpenAI @AnthropicAI @GregKamradt @MiqJ
Adam May retweeted
Phoenix is now integrated with Ragas to help in evaluating and analyzing your RAG pipeline. 🤩
It may seem silly, but this is as important as it gets to having a day where the conflict ends. The problem with the tourist discourse on the Israel/Gaza conflict is the unavoidable urge to cast this as good vs evil. It's not, we're all people - we have to remember that.
Adam May retweeted
1/3 Resigning, a Hanukkah Message: As of today I have resigned from the antisemitism advisory committee at Harvard. Without rehashing all of the obvious reasons that have been endlessly adumbrated online, and with great respect for the members of the committee, the short explanation is that both events on campus and the painfully inadequate testimony reinforced the idea that I cannot make the sort of difference I had hoped. Still, there are several points worth making. I believe Claudine Gay to be both a kind and thoughtful person. Most of the students here wish only to get an education and a job, not prosecute ideological agendas, and there are many, many honorable, thoughtful and good people at the institution. Harvard is still a repository of extraordinary minds and important research. – at Newton, MA
Adam May retweeted
So here's a story of our time.
1. In 2016, an Israeli soldier called Elor Azaria executes a wounded Palestinian militant. Point blank. On camera.
2. He becomes a hero to the far-right and the execution of wounded suspects briefly became a wedge issue in Israeli society.
Adam May retweeted
You don’t get it.
It’s not about an UNRWA teacher who held an Israeli kid hostage in his house.
It’s all about how for 75 years you have destroyed the future of generations of Palestinians, including my family.
My cousins in Arab countries are still not citizens - not even the grandkids of the real refugees.
My cousins in Lebanon are not even allowed to work in certain jobs.
And my friends in Gaza and the West Bank continue to be victims of incitement in their own schools.
And it’s all your fault.
Shut UNRWA down.
🛑 Spreading unsubstantiated claims about @UNRWA must stop immediately.
Making serious allegations in the public domain, unsupported by any evidence or verifiable facts in support thereof may amount to misinformation.
FULL Statement ⬇️
unrwa.org/newsroom/official-…
Readers added context they thought people might want to know
Claims against UNRWA have been documented for a long time.
The headmaster of an UNRWA school was a terrorist.
reuters.com/article/middle…
Film by David Bedein in Jenin, UNRWA policies and practices
vimeo.com/856467890
UNRWA teachers celebrated Oct 7 massacre
unwatch.org/report-u-n-tea…
UNRWA teacher holds hostage in attic
washingtonexaminer.com/policy/defense…
Adam May retweeted
It seems odd that there’s one UN agency for Palestinian refugees from the 1948 Arab-Israeli war and their descendants and another UN agency for all other refugees from all other conflicts. – at Manhattan, NY
Adam May retweeted
A Hamas video from a kibbutz shows terrorists torturing a pregnant woman and removing her fetus. Our forensic scientists have found bodies of women and girls raped with such violence that their pelvic bones were broken, writes Michal Herzog.
newsweek.com/silence-interna…
While I don't agree with every word - this is a good well measured call for a ceasefire. You can both be horrified and heartbroken at the loss of civilian life AND realize that Netanyahu != Israel, and the Hamas != Palestinians. Both have to go.
Adam May retweeted
"Can't make any criticism of Israel without being called antisemitic"
Oh, did you call for Netanyahu and his far right cabinet to resign?
"No, the other thing"
1/
Adam May retweeted
The problem is Tik Tok.
Anyone who thinks it’s a coincidence that the CCP app is making anti-American takes go viral isn’t paying attention. American kids are being manipulated and misinformed and we refuse to do anything about it.
The Guardian deleted Osama Bin Laden's "Letter to America" because it went viral on TikTok:
404media.co/guardian-deletes…
By FAR the best take I’ve seen so far on what a ceasefire and solution could actually look like. The biggest roadblocks are Netanyahu and the complete unwillingness of the world to see that the Palestinians greatest enemy for peace is Hamas.
nytimes.com/2023/11/14/opini…
Adam May retweeted
Vivian Silver actually led efforts to get better pay and more work permits for Gaza residents. She founded several groups specifically dedicated to building relationships between Israelis and Palestinians. Even after retiring, she volunteered for a group that helps Gaza resident get medical treatment in Israel. She spent her entire life trying to help Palestinians.
I don’t think it gets mentioned enough is that the people targeted on 10/7 tended to be some of the biggest advocates for Palestinians within Israel. They are people who lived on the border w Gaza and interacted with Palestinians constantly.
It’s an area filled with people who self-identified as peace activists. Many of those communities actively hired Palestinian workers.
They were rewarded for those efforts by being targeted with a type of barbaric slaughter that the world has rarely seen.
The Body of Vivian Silver a 74 y/o Canadian-Israeli Peace Activist and Founder of the Women Wage Peace who was believed to have been taken as a Hostage by Hamas into the Gaza Strip during their Surprise Attack against her Hometown of Be'eri in Southern Israel on October 7th has now been Identified in the Ruins of the Town; Vivian’s Family has been Notified of the Discovery.
The idea that being pro-Israel means you aren’t sympathetic, or even devastated by what’s happening in Gaza is straight up nonsense.
Adam May retweeted
Hi Kyle,
Since you’re citing Gaza health ministry numbers, let me ask a few questions:
1) How many Hamas militants have been killed? You seem to assume 0 since you counted every casualty as a civilian.
2) How many Palestinians have been killed by the 600-700 rockets that have fallen short in Gaza? You seem to assume 0 since you blamed every death on Israel
3) How many Palestinians have been killed by Hamas via bombs and shooting? You seem to assume 0
4) Does Hamas have any responsibility for the deaths of people they use as human shields? It seems you think the answer is no.
Adam May retweeted
#Assad joined the #ArabLeague "emergency" meeting on #Gaza today, hosted by #Saudi.
Little mention of his year-long starvation siege of the Palestinian Yarmouk Camp; the 5,000+ Palestinians killed by regime shelling or the ~4,000 Palestinians still in #Assad's torture prisons.
Adam May retweeted
Gaza Kindergarten Graduation Ceremony: Kids Stage Mock Military Attack and Hostage-Taking (Archival) #Hamas #Gaza #Palestinians #child_indoctrination
