We also find that MT systems/LLMs produce better translations than the human translators (at least according to automatic metrics), a result that should be confirmed with a follow up study (hint: future work coming soon!)

We collect machine translations from a variety of different MT systems/LLMs and rank them with automatic metrics. Takeaway: LLMs get the best scores across all languages! Data coming to MTME soon github.com/google-research/m…

🚨New machine translation dataset alert! 🚨We expanded the language coverage of WMT24 from 9 to 55 en->xx language pairs by collecting new reference translations for 46 languages in a dataset called WMT24++ Paper: arxiv.org/abs/2502.12404v1 Data: huggingface.co/datasets/goog…

Running large-scale experiments for building SOTA MT models arxiv.org/pdf/2309.10966

Researching how to improve the quality of human evaluations of text generation arxiv.org/pdf/2404.01474

Using SOTA LLMs for machine translation arxiv.org/pdf/2409.06790

Researching better ways to do metric meta-evaluation arxiv.org/abs/2305.14324

Developing SOTA automatic evaluation metrics arxiv.org/pdf/2410.03983

🌐Exciting News in Machine Translation! 🚀MetricX-23, our SOTA evaluation metric, is now OPEN-SOURCE in PyTorch/Transformers! 🎉There are three model sizes available, all trained on 1m+ human judgments of MT quality! 🔗Code github.com/google-research/m… 🔗Paper www2.statmt.org/wmt23/pdf/20…

@emnlpmeeting I'm running into an error when I try to submit the camera ready version of my paper. All the boxes are checked, but when I hit the submit button, this error message appears. Has anybody else had this problem?