Engineering Manager @Wooshii - Building the next generation of AI-driven enterprise content management platforms 🤖⚡️🎬 Tweets are my own.
United Kingdom
Joined March 2014
- Tweets2.6K
- Following109
- Followers667
- Likes3.4K
FullStackFool retweeted
This is because in 2013 the *definition* of autism was expanded to include Asperger's, CDD, PDD-NOS, and the Autistic Disorder.
Numbers went up because the definition changed to include more variants. And testing and awareness of Autism just got better over time. This is insane.
FullStackFool retweeted
New Glenn safely reached its intended orbit during today's NG-1 mission, accomplishing our primary objective. The second stage is in its final orbit following two successful burns of the BE-3U engines. The Blue Ring Pathfinder is receiving data and performing well.
We lost the booster during descent. We knew landing the first stage on the first try was ambitious. We'll learn, refine, and apply that knowledge to our next launch in the spring. We're thrilled with today's outcome. More: bit.ly/4hj8q0o
FullStackFool retweeted
BEWARE OF LIVE AI DEEPFAKES!
95K people are watching this “LIVE event” and the creator appears to be SpaceX…
If your loved ones are late to know about this, they could get scammed. Smart people get scammed all the time.
And congrats @SpaceX !
FullStackFool retweeted
Two weeks ago we released Reflect on Apple's App Store - it’s been a disaster.
Initially, we used TestFlight to distribute our iOS app, which worked well and allowed us to quickly iterate and ship updates. However, there was a cap on the number of users we could have.
Eventually, we decided to move to the App Store. Anyone familiar with this process knows it involves some back-and-forth before getting approval, but finally, the Apple reviewers gave us their blessing.
We announced our new iOS app and transitioned our customers off TestFlight. Job done! Or so we thought...
A week later, we prepared an update to the app with some editor fixes. The update was denied. Apple stated that our sign-in process violated their Terms of Service.
We use end-to-end encryption to protect our customers' notes so that nobody else, not even us, can read them. Our encryption process works in two stages: first, Apple or Google Auth is used to log into your Reflect account, and then you provide an encryption password which is used to end-to-end encrypt your notes. This password never leaves your device, ensuring your notes remain secure.
Apple's issue was with displaying a password field after the "Sign in with Apple" button, which they claimed was against their terms of service. It's important to note that this functionality had not changed since the initial approval of the app. They had approved the app, we announced it, moved everyone to it, and now they were blocking our update. We've tried appealing several times but have never received a response.
We are now in limbo. I don't know what to do.
This story is not unique. Every day developers are running into these app-store issues, but many are afraid to speak up in case of reprisals. Apple, if you're wondering why there are no apps for the Apple Vision, maybe take a good look in the mirror and think about how you treat us.
FullStackFool retweeted
This is regional prompting for image generation.
Wait till the end to see the result 🤯
FullStackFool retweeted
Of all the apps I made:
- Only 7 generated at least $1 (40%) 💸
- Only 2 allowed make a living (11%) ✅
🔪💀 Shipping fast is nice but killing fast is underrated.
👉 If one app generate $100/month but you have no idea how to grow it, and you already exhausted all your options to grow it, move on.
You're going to waste lots of time otherwise. Some products are just not made for you.
Bonus: Each new app you make will be faster because you can reuse code etc
I shipped 17 apps in one year.
😩 Me before:
- Don't know how to code
- No revenues
- Stress because only 1 year of savings
😎 Me after:
- Can code almost anything I want
- $5000/month from my apps
- Good savings after selling one app for $65,000
👉 If you're just starting with 0 knowledge, the best way to learn to ship is to ship a LOT and figure things out on the go
Flashbacks to a very old project 🤩
youtube.com/watch?v=pHKqhuP_…
FullStackFool retweeted
“Lost in the middle” again! Great analysis by Greg on GPT-4’s true long context recall.
Context length means nothing without actual performance benchmark. LSTMs have literally *infinite* context, so what. Eval on your actual application is all that matters.
Pressure Testing GPT-4-128K With Long Context Recall
128K tokens of context is awesome - but what's performance like?
I wanted to find out so I did a “needle in a haystack” analysis
Some expected (and unexpected) results
Here's what I found:
Findings:
* GPT-4’s recall performance started to degrade above 73K tokens
* Low recall performance was correlated when the fact to be recalled was placed between at 7%-50% document depth
* If the fact was at the beginning of the document, it was recalled regardless of context length
So what:
* No Guarantees - Your facts are not guaranteed to be retrieved. Don’t bake the assumption they will into your applications
* Less context = more accuracy - This is well know, but when possible reduce the amount of context you send to GPT-4 to increase its ability to recall
* Position matters - Also well know, but facts placed at the very beginning and 2nd half of the document seem to be recalled better
Overview of the process:
* Use Paul Graham essays as ‘background’ tokens. With 218 essays it’s easy to get up to 128K tokens
* Place a random statement within the document at various depths. Fact used: “The best thing to do in San Francisco is eat a sandwich and sit in Dolores Park on a sunny day.”
* Ask GPT-4 to answer this question only using the context provided
* Evaluate GPT-4s answer with another model (gpt-4 again) using @langchain evals
* Rinse and repeat for 15x document depths between 0% (top of document) and 100% (bottom of document) and 15x context lengths (1K Tokens > 128K Tokens)
Next Steps To Take This Further:
* Iterations of this analysis were evenly distributed, it’s been suggested that doing a sigmoid distribution would be better (it would tease out more nuanced at the start and end of the document)
* For rigor, one should do a key:value retrieval step. However for relatability I did a San Francisco line within PGs essays.
Notes:
* While I think this will be directionally correct, more testing is needed to get a firmer grip on GPT4s abilities
* Switching up prompt with vary results
* 2x tests were run at large context lengths to tease out more performance
* This test cost ~$200 for API calls (a single call at 128K input tokens costs $1.28)
* Thank you to @charles_irl for being a sounding board and providing great next steps
In a couple of months, when the editing is near-perfect and near-zero-cost, this kind of shit is going to ruin so many lives...
facebook.com/nhacaiuytinnapr…
@MartinSLewis
FullStackFool retweeted
See, the thing is @CepheidNews makes this amazing TB test that can tell doctors whether a patient has TB and, if so, which antibiotics that TB is resistant to. Which is amazing…when doctors can afford to use it.
They sell it through a printer-ink business model, where the machines are cheap, but the cartridges that do the test are sold at 300% to 500% margins.
There are tens of millions of tests done per year. But many places cannot afford them. Please tweet to @CepheidNews and their parent company @DanaherCorp to tell them that it will be great for their business (which has $8B in profit, and TB tests are not a huge part of it!!) if they lowered the cost!!
I want to celebrate science, not feel like access to something that could save hundreds of thousands of lives is being denied to patients because they cannot afford the price.
FullStackFool retweeted
Today we open source the training code for our audio generation and compression research in AudioCraft and share new models.
With this release, we aim at giving people the full recipe to play with our models and develop their own models!
FullStackFool retweeted
I keep being asked what I make of the UFO thing in Congress yesterday, so here it is: I watched a few clips and saw some people who seemed to believe stuff saying extraordinary things without presenting extraordinary evidence. Therefore I have nothing more to say, other than: It would be great if true - it would take a bit of the pressure off our civilisation if we weren’t the only means within the Milky Way by which the Universe understands itself. Sadly, as of today, I still feel that pressure, so can we perhaps focus on not messing our world up rather than hoping that, to paraphrase Sagan, someone will float down to save us from ourselves.
If you've been paying attention to their work on ANE over the years - ESPECIALLY the space dedicated to it on the M1&M2 SoCs - then you will know they have not "been quiet".
I don't like "hot takes" but...
@Apple has the hardware. It has first-party support. It will dominate.
This quoted post is unavailable.
Apple is going to dominate here. There are several "bars" of expectation for LLMs. There are the code-generation and academic expectations which require an almost-perfect accuracy - for instance in medicine.