4dimcube (@4dimcube) header
4dimcube (@4dimcube) profile picture

Professional circle packer. Does math sometimes.

Joined July 2018
Photo by 4dimcube (@4dimcube)Photo by 4dimcube (@4dimcube)
I've known Justin for the better part of 4 years through some pretty awesome chatrooms for discussing frontier ML/AI research, and I think his philosophy and approach are going to make waves. Super excited to see him come out of stealth here!
This quoted post is unavailable.
3
263
I agree with @ylecun on a lot of things but this seems off: by the data processing inequality, if a system "understands" (captures) an input then it *must* contain the information required to generate that input. Perhaps the point is that you don't need a generative architecture?
Replying to @ylecun
Video generation systems will get better with time, no doubt. But learning systems that actually understand physics will not be generative. All birds and mammals understand physics better than any video generation system. Yet none of them can generate detailed videos.
1
2
374
To be clear: I think you can build effective models that do not explicitly express a posterior over the input space, but I contend that internally the model must capture this information if it is complete. It *is* a generative model, just without a decoder in the input space.
1
1
188
Working on some new writing for the first time in a while. Theme is broadly how constructing narratives to explain the world is critical for compressing information but can backfire in complex contemporary environments. Please respond with any related research or anecdotes!
2
159
New glasses acquired with a substantial prescription change. On one hand, everything is now in HD again, but on the other, HOLY SHIT when did my floors get this dirty? 🙃
123
This reads to me like "vocal critic of cars voluntarily rides the train." How dare he be . . . consistent in word and action?
Not sure why one of the most vocal AI critics would publicly volunteer the fact he doesn't even use LLMs 🤔
1
1
198
For what it's worth I obviously disagree with the idea that you can't build on the current generation of LLMs, but that's not really the point here imho. Shameless plug: go check out qdeck.ai if you want to see some impressive RAG for fintech stuff ;)
130
This is it, right here. The answer to most of your questions about LLMs. The language box cares not about factuality; you need to impose factuality as a constraint on the language box.
Replying to @gerardsans
Because it’s not trying to be right
2
123
I can't believe this site is still free
Feeling super generous so, somewhat reluctantly, here are 5 industry secret holy grail books that have immensely assisted me in my quantitative trading endeavours (as an ex manual HFT mm) that I highly recommend new and experienced traders pick up if they can. No specific order.
3
203
Easily one of the most exciting new ML textbooks since I got into the field. Hats off to @sirbayes and the rest of the team he put together to make this happen!
I am delighted to announce that the camera-ready version of my new book, "Machine Learning: Advanced Topics", is finally available online for free at probml.github.io/book2 (@mitpress will publish the hard copy in 2023.)
Look, I disagree with Yud on a lot of things but this one's a work of art.
Welp, looks like all the advanced language and image models are being trained not to read or output sexy things. Do you WANT the human resistance movement fighting in sexy scanty armor so that the robots can't see them? This is how anime ends up real, people.
1
My lack of commentary regarding the phenomenally exciting ML announcements over the last few weeks is not a product of apathy or inattention, but of awe. I suspect we already live in the future: our dreams are striving to catch up with our capacity, not the other way around.
With multiple foundation models “talking to each other”, we can combine commonsense across domains, to do multimodal tasks like zero-shot video Q&A or image captioning, no finetuning needed. Socratic Models: website + code: socraticmodels.github.io paper: arxiv.org/abs/2204.00598
3
Contrary to popular belief, this is not the case because summoning a demon and compiling C code are fundamentally different processes but because Latin has far more expressive syntax. The verbosity is a feature not a bug!
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more
2
Quant finance has taught me that specification of your problem requires careful thought. Early stopping has been replaced by techniques like dropout but the principle is the same: limit the space of models you consider to avoid bad solutions. 🧵
"Early stopping" used to be important for deep learning classifiers. But nowadays we just train for as long as possible. What changed (if anything)? And when?
1
2
In finance, the best solutions often involve regularization tailored to the problem at hand. This means carefully considering the objective you're trying to optimize and modifying it to discourage poor solutions (e.g. penalizing sharpe if a model rarely trades).
1
1
Ultimately this is one of the factors that requires creativity from ML researchers and practitioners! Exceptional models are often differentiated by the careful thought put into precisely what is being optimized. ◾️
1