Tweets on Technology and Art. Views my own @munhitsu@fosstodon.org
London, UK
Joined September 2007
- Tweets4.9K
- Following1.6K
- Followers645
- Likes3.4K
I loved gevent. It was bringing all the benefits of event loop I needed and leaving me with a straightforward API on monkey patched threads. I never could understand why it was treated as an ugly child
“I'm now convinced that async/await is, in fact, a bad abstraction for most languages, and we should be aiming for something better instead and that I believe to be thread.”
lucumr.pocoo.org/2024/11/18/…
Nice, I might be eventually able to use letter “m” in passwords for some, ancient services. But then again if they are already ancient, will their CISO actually care about the new NIST guidance?
mastodon.social/@LukaszOlejn…
Lapsa-Malawski retweeted
Explains why I found myself forced to not just block Musk, but also mute the terms “Elon”, “Musk”, “Elonmusk” to get a Twitter experience where I wouldn’t have every second tweet of his on my timeline.
Case study worthy on how you degrade a social network long-term
This Post is from an account that no longer exists. Learn more
Lapsa-Malawski retweeted
As Apple Intelligence is rolling out to our beta users today, we are proud to present a technical report on our Foundation Language Models that power these features on devices and cloud: machinelearning.apple.com/re…. 🧵
Long awaited - standardised way to run web app in lambda github.com/awslabs/aws-lambd…
We live in the future; it's just not distributed evenly: hakaimagazine.com/videos-vis…
Intellectual obesity - I love the term:
gurwinder.blog/p/the-intelle…
Next time you estimate development think about leaky pipelines: hiandrewquinn.github.io/til-…
Lapsa-Malawski retweeted
Yann LeCun says he is working to develop an entirely new generation of AI systems that he hopes will power machines with human-level intelligence. It could take up to 10 years to achieve, he tells the @FT in an interview on.ft.com/3KbShLF
Lapsa-Malawski retweeted
How to be as "smart" as Auto-Regressive LLMs:
- memorize lots of problem statements together with recipes on how to solve them.
- to solve a new problem, retrieve the recipe whose problem statement superficially matches the new problem.
- apply the recipe blindly and declare victory.
- do not use basic logic.
- do not use common sense to check your solution.
- do not use a mental model of the situation as a sanity check.
- do not simulate the scenario in your mind using your world model.
- when someone tells you your solution is wrong, reply "I'm sorry, you are right" and apply another irrelevant recipe.
Knowledge accumulation is not a substitute for actual understanding.
Lapsa-Malawski retweeted
At this point I feel like we understand pretty well what's going on with LLMs:
- Outputs are roughly equivalent to kernel smoothing over positional embeddings (arxiv.org/pdf/1908.11775.pdf)
- The learned computation model is *probably* bounded by RASP-L (arxiv.org/pdf/2310.16028.pdf)
- LLMs learn structure primarily from human generated content (text, images) which is far more structured and predictable than the universe.
- LLaMa3 shows us that the higher quality the annotations on the human generated content, the better the LLMs do (10million messages is a lot!)
- Multi-turn labeling is currently very expensive and so likely driving the costs of the models.
- Right now we're likely bottlenecked not on CPU, or size of data, but number and quality of annotations.
So tl;dr. Great at predicting what a human would do or say by averaging in distribution data in the corpus. No emergent generality. Currently bottlenecked by high quality annotated data.
@hausdorff_space did I miss anything?