We have been working on PyTorch native float8 and FSDP2 for distributed training. Check out TorchTitan and TorchAO/float8 dev-discuss.pytorch.org/t/en… with Andrew Gu, @wanchao_ , @drisspg , @vkuzo , @brian_hirsh