Ana Paula Mofarrej

Today we’re releasing V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. This work is another important step towards @ylecun’s outlined vision of AI models that use a learned understanding of the world to plan, reason and accomplish complex tasks. Details ➡️ bit.ly/49fCeaM We're releasing a collection of V-JEPA vision models trained with a feature prediction objective using self-supervised learning. The models are able to understand and predict what is going on in a video, even with limited information. It learns by predicting missing or obscured parts of a video in its internal feature space. Unlike generative approaches that fill in missing pixels, this flexible approach enables up to 6x improvements in training and sample efficiency. The models were pre-trained on entirely unlabeled data, and a small amount of labeled data can be used to train a task-specific prediction head on top after pre-training. Our results show that, using a frozen backbone, our top V-JEPA models achieve 82.0% on Kinetics-400, 72.2% on Something-Something-v2 and 77.9% on ImageNet1K — competitive with or exceeding previous leading video models. We believe that this work is an important milestone on the path to advancing machine intelligence.

Last week we released FACET, a new comprehensive benchmark dataset for evaluating the fairness of models across a number of different vision tasks, constructed of 32K images from SA-1B, labeled by expert annotators. Read the paper ➡️ bit.ly/3EotZvg

We believe that AI models benefit from an open approach, both in terms of innovation and safety. Releasing models like Code Llama means the entire community can evaluate their capabilities, identify issues & fix vulnerabilities. github.com/facebookresearch/…