Training with large minibatches is bad for your health. More importantly, it's bad for your test error. Friends dont let friends use minibatches larger than 32. arxiv.org/abs/1804.07612

Excellent blog post from Turing Post on JEPA (Joint Embedding Predictive Architecture), my favorite meta-architecture for Self-Supervised Learning of continuous data, such as images, video, and audio. The post includes a list of relevant papers from my collaborators and me, as well as part of the growing list of papers from other groups that use JEPA for various things: audio, EEG, SAR, LIDAR, etc. A small clarification: JEPA is not an alternative to transformers. I'm fact, many JEPA systems use transformer modules. It is an alternative to Auto-Regressive Generative Architectures (such as LLMs), regardless of whether they use transformers. turingpost.com/p/jepa