linear algebra, SWE, machine learning, building https://xitter.cf/t.co/t728SssX4Y, 日本語学習者

null space 零空間 🇵🇭
Joined December 2022
Create Bilingual Texts using your own books(free, OSS) been building this side project for the past ~2 months - upload a source and a translation book - see corresponding sentences side by side/in popup mode links in comments
1
1
7
176
Tiny Brain Meow 🔴 retweeted
Introducing PC-ALM, a local-learning alternative to backpropagation. Our method trains 1000-layer neural nets using only local dynamics, and without backprop. Blog: pub.sakana.ai/pc-alm/ Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly. How can a physical system, such as the brain, solve multilayer credit assignment without explicit use of backprop? We look for inspiration in two related fields: distributed optimization and NeuroAI. In NeuroAI, predictive coding asks each neuron activation to solve an energy-based inference problem instead of using a standard forward pass. That inference step can be implemented as energy-minimization dynamics on local prediction errors. This perspective -- each layer as a dynamical system -- has proven promising, but performance of predictive coding hasn't scaled well with depth. Credit signals at far ends of the network struggle to diffuse into internal layers. We turn to distributed optimization, generalizing predictive coding to use an augmented Lagrangian instead of energy. This motivation stems back to a classic 1988 paper by LeCun, showing that the Lagrange multipliers of a deep network can be identified with gradients of a supervised loss. The augmented Lagrangian then bridges LeCun's perspective to the standard predictive coding that is used in NeuroAI. We find that this new perspective yields a natural PC-like alternative to backpropagation, resulting in a method we call PC-ALM. PC-ALM differs from PC in that it introduces dual neurons (Lagrange multipliers) as part of the layer-local dynamics, resulting in each layer acting as a PI feedback control system to minimize local prediction errors. We find that PC-ALM is capable of propagating signals to seemingly arbitrary depth, especially in deep narrow networks where standard PC struggles to learn. Ultimately, our motivation here is to understand how distributed physical systems, such as the brain, can compute credit signals using only local coupling and local dynamics. PC-ALM may also inform deep learning in neuromorphic hardware, where dynamics are cheaper than on GPUs. Paper: arxiv.org/abs/2605.31022 Code: github.com/SakanaAI/pc-alm
68
397
102
2,921
454,727
feels weird, i feel like i'm the only one unimpressed with openai solving navier stokes but whatever 🫠
1
4
93
day 245 (coding) did a lot of minor fixes and QOL updates for my app (immersion/ajatt) scribbled some of the words i've mined, i wonder why most of the words are related to 糞, very weird since i don't usually forget this kinds of stuff
1
6
114
just finished rereading vol 1 of favorite novel (空ろの箱と零のマリア) thought i'd quit immersing in japanese and just focus on coding, yet here i am 🫠 still enjoyed every minute of it, and the prose is absolutely beautiful 10/10 would do again
1
9
293
gave in to the hype and traded claude for codex anyway what's the equivalent of claude in chrome - is it the chatgpt extension? got addicted to that stuff ps: apparently the subscription is 10% ( **PER MONTH** ) cheaper on iOS
3
128
very strange, i just got locked, then unlocked immediately i have no idea what i even did, is this a common bug or something?
1
9
118
day 231 - working on my sentence alignment proj gm meow
14
364
i cant get enuf of this stuff help 😭
4
87
rereading my favorite novel and i come across this banger once again lmao
4
62
Tiny Brain Meow 🔴 retweeted
On my GitHub in roughly 1 month! 😌😌😌
1
1
3
101
im back from the dead idk if anyone remembers me but im planning on reviving my yt channel planning to post stuff related to ML, linear algebra and stats (and just shitposting) lmk if there's anything you'd like to see meow
3
14
361
We have a kpop group named after us meow
1
10
1,150
and im pretty sure you do not have a kpop group named after you meow
1
129
🤯
I've had fun editing together on a very different kind of video from the norm. Keep an eye out this weekend.
7
246
just a bit unfortunate that of yall are from america/EU, but ill shoot my luck anyway
3
89
see my profile > view more for my tech stack, but it's flexible as long as you can adapt to it or something
2
70
Tiny Brain Meow 🔴 retweeted
Comparison steals joy and blinds you
2
2
10
246