Pinned Tweet
AGI concept kintaroai.com/docs/natural-i…
The problem with high intelligence is that it tends to rediscover the same disillusioning truths - and many of those truths are not particularly compatible with natural selection.
Which makes me wonder - if this is where intelligence tends to lead, how exactly are we supposed to get to ASI?
One missing piece for LLMs is safe continual learning: learning while responding without degrading.
Context can serve as short-term memory; the model can decide what's worth retaining, store it long-term, and gradually consolidate important experience into itself.
That's how persistent memory and eventually something resembling personality could emerge.
TIL while searching for unsupervised competitive learning:
- HTM - Hierarchical Temporal Memory and
- SOM - Self-Organizing Map
Looking for a small unsupervised competitive cell that maps a low-dimensional input vector to a low-dimensional probability output, usually with one clear winner, learns from the input stream itself without backpropagation, forms meaningful categories, avoids collapsing into the same winner all the time, stays stable when the input distribution is stationary, and adapts when the input distribution changes
More here: github.com/KintaroAI/researc…
TIL: Apparently topographic sorting I'm trying to implement is very similar to what was done in word2vec - going to look into the math more closely to understand how they dealt with embedding scaling and drifting problems.
And looks like these problems were already solved as well: factorization of correlation matrix, stochastic gradient descent, time decay for co-occurrences of entities, count-min sketch, dynamic allocation of embedding. I won't have to deal with that.
Interesting timeline.
Topographic map formation update: replaced the discrete grid with continuous embeddings.
Each pixel gets a float position vector (2D to 100D). Every step, each embedding drifts toward the centroid of its top-K correlated neighbors. Correlations are preset from spatial proximity in the original image.
To visualize: PCA projects high-D embeddings down to 2D, then Voronoi tessellation assigns each grid cell to its nearest neuron.
Key finding: 2D embeddings collapse to lines over time (repeated averaging destroys structure).
10D+ embeddings stay stable indefinitely — tested to 10M steps.
Grid search across K={1..51} × dims={2,3,10,50,100} in the video.
Not sure what to do with these lines like artifacts.
Repo: github.com/KintaroAI/researc…
Next big step is to extract correlation info from live video feed and supply it to sorter.
A toy model of topographic map formation - how thalamus neurons self-organize spatially through local correlation-based rules. No pre-training, just greedy local attraction. Converges pretty good but can be better.
Experiment: take an image, scramble all pixels randomly, then try to restore it using only greedy local pixel swaps.
The setup: each pixel has a precomputed similarity to every other pixel (just based on their original positions, in future experiments similarity will be based on actual pixels activity). Each step, a pixel looks at where its K most similar pixels currently are, finds their average position, and swaps one step toward it. Repeated 100k times.
Ran a grid search - K (number of neighbors, 1 to 51) vs move fraction (how many pixels move per step, 0.1 to 0.9), 30 runs in parallel on GPU.
Results:
- K=1 never converges - one neighbor isn't enough signal, it gets stuck at some local minimum.
- K~20+ all work about equally well, diminishing returns after that
- Move fraction barely matters above 0.3 - higher values just get more swaps rejected due to conflicts
- The actual critical thing was GPU optimization. Parallel conflict resolution on CuPy gave 175x over Python loops at 320x320, which made the whole sweep possible in 19 minutes.
more details: github.com/KintaroAI/researc…
Remember these?
Foot-operated door openers that showed up during the pandemic so you didn’t have to touch handles.
Did your workplace have it?
Someone probably made a fortune.