AI & Software. Comedy, Voice, Acting & Improv. Film Analysis & Reviews @FilmTagger_com. @Google @DWAnimation @Cinesite @StHughsCollege alum. Love #Golang.

Joined June 2009
Andrew Bonello retweeted
This essay is one of the most readable criticisms of US housing finance and policy I’ve ever seen: medium.com/@byrnehobart/the-…
6
53
9
301
Need help developing your cpp cplusplus backend application ? fiverr.com/s/38EgpmL
1
1
102
Do you need a comedic or humorous audio voiceover? Then look no further! fiverr.com/s/D7DyeA
1
1
40
Do you need a comedic or humorous audio voiceover? Then look no further! fiverr.com/s/D7DyeA
1
1
40
Need to record a hilarious comedy Donald Trump voiceover? fiverr.com/bitofenglish/reco…
1
1
83
Andrew Bonello retweeted
Replying to @jeanqasaur @hrlomax
I never heard of the term developer influencers until now.
1
1
13
But! You may (correctly) say: people don't like to write specifications OR types. One of my other favorite papers is by Ras Bodik (again), about using machine learning to automatically mine specifications from program behavior. dl.acm.org/doi/10.1145/50327… 6/
1
1
5
1,594
Andrew Bonello retweeted
I'm joining forces with @mikeknoop to start Ndea (@ndeainc), a new AI lab. Our focus: deep learning-guided program synthesis. We're betting on a different path to build AI capable of true invention, adaptation, and innovation.
125
214
75
2,388
193,388
The Kalman filter is a widely used algorithm for estimating the hidden states of a dynamic system from a series of noisy measurements. It works by recursively predicting the system's state using a dynamic model, and then updating this prediction with new measurement data. Some key points about the Kalman filter: - It is an optimal estimator for linear systems with Gaussian noise, minimizing the mean squared error of the estimated state. [2] - It consists of two main steps: prediction and update. In the prediction step, it estimates the current state based on the previous state and the system dynamics. In the update step, it incorporates a new measurement to correct the prediction. [3] - It accounts for both process noise (uncertainty in the system dynamics) and measurement noise (errors in the sensor data). [1] - It requires a mathematical model of the system dynamics (state transition matrix) and the measurement process (measurement matrix). [3] - The filter is recursive, meaning it only needs the current measurement and the previous state estimate to compute the new state estimate, without requiring storage of the entire measurement history. [2] - It has found widespread applications in areas like navigation, object tracking, signal processing, and control systems due to its effectiveness and computational efficiency. [2] The Kalman filter provides an elegant and powerful solution for state estimation problems involving noisy sensor data and uncertain system dynamics, making it a fundamental tool in many engineering and scientific fields. [1][2][3]
9
62
10
690
130,822
Andrew Bonello retweeted
The problem with this popular tweet is that DeepSeek is NOT actually a capability improvement per se. It’s an efficiency improvement; those two are not the same.
The most unnerving part of the DeepSeek reaction online has been seeing folks take it as a sign that AI capability growth is not real It signals the opposite, large improvements are possible, and is almost certain to kick off an acceleration in AI development through competition
27
27
4
324
36,344
Andrew Bonello retweeted
here are my notes on @fchollet's neat explanation of the differences between deep learning and program synthesis, and the advantages and disadvantages of each, and how they'd fit together to build AGI. in deep learning, your underlying model is a differentiable curve; in program synthesis, your model is a discrete graph of operators – you’re picking from a set of operators and structuring that into a program. this has implications for the amount of compute and data needed for each: - in deep learning your learning engine is gradient descent, which is very compute efficient – you have a very informative feedback signal about where the solution is. but it's very data inefficient — you need a dense sampling of the data distribution. - in program synthesis, your learning engine is combinatorial search. this is extremely data efficient (I believe because the problem space is inherently more constrained?), but it’s extremely compute inefficient (because the search space is massive). how does this apply to AGI? deep learning is great for system 1 thinking; discrete program search is great for system 2 thinking. AGI will likely require a combination of both approaches. Chollet expects that an AGI system would have an outer program that does program synthesis and it will use deep learning to assist it.
24
113
14
805
113,401
Andrew Bonello retweeted
Why AI needs to ease up on scaling and learn how to code: deoxyribose.github.io/No-Sho… @GaryMarcus @fchollet @yudapearl
9
33
Andrew Bonello retweeted
I believe that program synthesis will solve reasoning. And I believe that deep learning will solve program synthesis (by guiding a discrete program search process). But I don't think you can go all that far with just prompting a LLM to generate end-to-end Python programs (even with a verification step and many samples). That won't scale to very long programs.
45
73
21
822
170,972
Andrew Bonello retweeted
We have to take the LLMs to school. When you open any textbook, you'll see three major types of information: 1. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it, your brain is training on that data. This is equivalent to pretraining, where the model is reading the internet and accumulating background knowledge. 2. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning, where the model is finetuning on "ideal responses" for an Assistant, written by humans. 3. Practice problems. These are prompts to the student, usually without the solution, but always with the final answer. There are usually many, many of these at the end of each chapter. They are prompting the student to learn by trial & error - they have to try a bunch of stuff to get to the right answer. This is equivalent to reinforcement learning. We've subjected LLMs to a ton of 1 and 2, but 3 is a nascent, emerging frontier. When we're creating datasets for LLMs, it's no different from writing textbooks for them, with these 3 types of data. They have to read, and they have to practice.
378
1,732
203
11,751
722,971
Replying to @brentdavis @paulg
This might help xitter.cf/newstem61/status/18712… Also this aiguide.substack.com/p/can-l… Researchers don't actually know. I'm feeding a number of research papers on the topic into notebooklm, to see if it can understand how it behaves. I have a feeling a user interaction with say Grok, doesn't have the same weight of trained data. So may not force it into new thought processes. Maybe to avoid brainwashing?
Replying to @DaniDeli3 @paulg
I think the AIs are reasonably good at debate and modify their position based on logical reasoning. The AI models are not perfect machines. So you need to fine tune results to fill in logical gaps. They first became emergent (able to perform unexpected tasks) with the Transformer architecture and the first LLM (ChatGTP). A LLM Large Language Model has 6 to 10 billion parameters. They are growing ten fold per year. The importance of carefully designed prompts implies that reasoning in LLMs isn't just a matter of retrieving the "right" data point. The prompt guides the LLM's reasoning process by establishing context and desired outcomes. The research community is still playing catchup. As no one fully understands why they work so well, even with an accuracy of 72%.
1
153
Andrew Bonello retweeted
How to train a State-of-the-art reasoner. Let's talk about the DeepSeek-R1 paper and how DeepSeek trained a model that is at frontier Sonnet/o1 level.
15
243
9
2,491
290,249
This is fascinating: Rich Sutton on the "bitter lesson" of AI research: incompleteideas.net/IncIdeas…
24
98
26
557
want to get into program synthesis but don't know how to started? I wrote a minimalist intro to modern program synthesis that can help you -- from problem formulation to generating code by fine-tuning llm on huggingface. evanthebouncy.github.io/prog…
2
17
6
133
Andrew Bonello retweeted
Replying to @ID_AA_Carmack
The question is will top AIs get better at gui faster than all apps add text. I think I have a guess
67
35
14
1,388
114,434