Maths / Cryptography / AI

🌌
Joined March 2021
Include

Only show posts containing:

Exclude

Hide posts containing:

Time range
-
Minimum likes
Session Recap | Bangalore, India 🇮🇳 | 26th Sept | Topic: Beyond Autoregression and Backprop Another amazing session just completed in Bangalore by @ocdbytes & @BLRxZo! 📖 Summary - drive.google.com/file/d/1zEk… @typesafeai's Jev answers typed questions in a single forward pass and returns probabilities, so the whole session came down to whether those probabilities can be trusted. 📝 Topic Overview: A chatbot generates an answer token by token and then sounds confident about it. Jev does neither. You give it state plus typed questions: Noul for yes/no, Choice for one option from your list, Score for ordered levels. It returns a probability over only the options you allowed. That makes the output type-safe, not correct. The "0% hallucination" figure comes from guaranteed schema matching, so Jev can still pick the wrong option from your list with high confidence. ✍🏻 @typesafeai has not published the architecture, weights or training method. So we rebuilt the idea from public papers and labeled every guess as a guess. ✍🏻 Better models got worse at knowing when they are wrong. In Guo et al., ResNet beat LeNet on CIFAR-100 error (30.6% vs 44.9%) but kept saying 0.9 when it was right far less often. ✍🏻 Calibration means that of all answers given at 0.8, about 80% should be right. GPT-4's pre-trained model scored an ECE of 0.007 on an MMLU subset, and post-training pushed it to 0.074. 💡 ECE is the average gap between stated confidence and actual accuracy. ✍🏻 Most "calibration" in the open Jev clones is a single temperature fitted after training, which is a 2017 trick - What does RLCD add beyond a strong base model plus one number? ✍🏻 This is the backprop problem. Jev outputs full probability distributions, which are differentiable, so plain supervised training with a Brier loss already rewards calibration. 💡 So why is RLCD reinforcement learning at all? The candidates are a reward based on downstream outcomes instead of labels, or decisions chained over steps as in the Doom demo?
Upcoming Event | Bangalore, India 🇮🇳 | 26th Sept | Topic: Beyond Autoregression and Backprop 🎟️ RSVP - luma.com/wprc-blr Thank you for our Session Guide @ocdbytes & Session Partner @BLRxZo - India’s first permanent hacker house in Bangalore🤝🏻 📝 Topics and TL;DR 1) JEV by @typesafeai - A model that doesn't generate text at all, user just needs to send it context plus typed questions, and it returns a decision with a confidence score in one pass! 2) PC-ALM by Seely and Gould (@SakanaAILabs) Training without backprop, meaning no global backward pass sending errors from the output through every layer. Each layer learns locally, and a per-layer correction term pulls its updates close to what backprop would have computed. 💡 What we'll argue about: > Is JEV a new model class or a well-packaged classifier? > Can local learning hold up past small benchmarks? Summary will be shared on Luma soon!
2
3
391
Replying to @nitanshu
Search about butlerian jihad
1
30
Replying to @nitanshu
We are entering the Dune universe
1
42
ocdbytes retweeted
Whitepaper Reading Club at @BLRxZo What's on agenda today: - @typesafeai 's Jev architecture discussion - PC-ALM paper by @SakanaAILabs Might even make notes and post them after the session
2
5
44
1,116
Upcoming Event | Bangalore, India 🇮🇳 | 26th Sept | Topic: Beyond Autoregression and Backprop 🎟️ RSVP - luma.com/wprc-blr Thank you for our Session Guide @ocdbytes & Session Partner @BLRxZo - India’s first permanent hacker house in Bangalore🤝🏻 📝 Topics and TL;DR 1) JEV by @typesafeai - A model that doesn't generate text at all, user just needs to send it context plus typed questions, and it returns a decision with a confidence score in one pass! 2) PC-ALM by Seely and Gould (@SakanaAILabs) Training without backprop, meaning no global backward pass sending errors from the output through every layer. Each layer learns locally, and a per-layer correction term pulls its updates close to what backprop would have computed. 💡 What we'll argue about: > Is JEV a new model class or a well-packaged classifier? > Can local learning hold up past small benchmarks? Summary will be shared on Luma soon!
1
3
1
4
699
Replying to @Timeroot
I was literally training something similar. Looks cools man !!!
80
Replying to @shivamhwp
Sure let me ask chat gpt to how to open my whole network and enable public access to all my devices to make things easy for you also remind me to disable auth. Thanks
1
1
42
Replying to @shivamhwp
I will try 🫡
1
1
32
ocdbytes retweeted
word2vec embeddings study + implementation Code : github.com/ocdbytes-ai/embed… Blog : ocdbyt.es/blogs/word2vec Check out the thread 🧵 :
1
1
5
111
Replying to @sama @huggingface
OpenAI executives were literally stroking themselves while they typed out this report. If y'all can't understand that this was written to purely brag about the model then I don't know what to tell you.
27
24
5
1,901
94,901
Replying to @shivamhwp
Less go 🚀
1
19
ocdbytes retweeted
Whitepaper Reading Club at @BLRxZo
16
4
103
3,860
in hindsight it'll be so obvious that throwing LLMs at every ML problem was stupid
39
18
1
527
26,067
Replying to @0xAbix
We can utilise pow chains for this. Check tg dm.
1
1
61
‼️EVENTS THIS WEEK‼️ If you have friends or colleagues in Bengaluru, Kuala Lumpur, Ankara, or Davao City, feel free to share! 🔥 WANT TO ATTEND? luma.com/whitepaper?locale=e… 1) 🇲🇾 KUALA LUMPUR, MALAYSIA | 27 Jun: Register: luma.com/whitep-q5rq 🧠 Topic: Agentic Trading: How do markets change when AI agents can observe, decide, and execute trades on-chain? ——————- 2) 🇮🇳 BENGALURU, INDIA | 27 Jun, 10:30AM IST Register: luma.com/whitep-phkn?locale=… 🧠 Topic: The Quantum Era & Proof-of-Work ——————- 3) 🇹🇷 ANKARA, TURKEY | 27 Jun Register: luma.com/g3ml5o0g 🧠 Discussion: Round table discussion at the Monad Blitz event in Ankara with Emire who translated our papers to Turkish ——————- 4) 🇵🇭 DAVAO CITY, PHILIPPINES | 27 Jun, 1:00PM PHT Register: luma.com/whitep-eq3i?locale=… 🧠 Topic: RWAs, DeFi & Institutional Blockchain Infrastructure
1
4
259
ocdbytes retweeted
Hash-based IOPs are great at proximity testing but weren't built for privacy. Bolting on zero-knowledge usually adds overhead, that is, until ZOOK. We implemented and benchmarked ZOOK, the zero-knowledge hash-based PCS from Alessandro Chiesa, @GiacomoFenzi and @gwberg95.
3
2
2
15
3,433
Replying to @sylvechv
Sad to hear that, best of luck man 🫡 I know next thing would be awesome
1
2
49
ocdbytes retweeted
Today we're launching early access to Zwap. Trustless, shielded cross-chain swaps for Zcash. In and out of Orchard, trusting no one. Live now: app.zwap.exchange
1
4
4
18
4,645