We released ClimateGPT, a family of publicly accessible climate change domain-specific LLMs (7B, 13B & 70B) trained using only renewable energy.
Paper Preprint: arxiv.org/abs/2401.09646
HuggingFace: huggingface.co/collections/e…
TL;DR⬇️
Very initial human evaluation aligns with these results, but there are still a lot of open questions to answer and ablation experiments ahead in the coming months.
This project was a joint effort of researchers from @AppTek_McLean Erasmus.AI @EQTYLab @RWTH.
If you want to try the model with all features apply for access on eci.io or just download the model itself from HuggingFace.
Also checkout what @EQTYLab was working on to document and verify the model lineage of ClimateGPT: huggingface.co/spaces/EQTYLa…
David Thulke retweeted
Dear ACL community, ACL is considering multiple proposals to change its anonymity period policy. It seeks immediate feedback from the community about the proposed changes. Please add your voice until Friday, September 22nd (AOE): aclweb.org/portal/content/su… #NLProc
David Thulke retweeted
Excited to share our preprint on uncertainty & natural language generation!
We need NLG models that can indicate when they are wrong and can handle language variation.
A good representation of uncertainty is crucial.
But where does uncertainty come from? How to represent it? How does it power NLG models?
We unpack this arxiv.org/pdf/2307.15703.pdf
🧵👇1/7
David Thulke retweeted
Check our #EMNLP findings and #SustaiNLP paper. In short, we speed up the pre-training of masked language modeling by 1.5 times without any degradation. With the same training budget, our model outperforms BERT, RoBERTa and ELECTRA. (1/n).
Code: github.com/BaohaoLiao/3ml
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token arxiv.org/abs/2211.04898