A central repository for pre-trained adapter modules in transformers! Active maintainers: @clifapt @h_sterz @LeonEnglaender @timo_imhof @PfeiffJo

Joined May 2020
LLM agents are assumed to integrate unexpected environmental observations into their reasoning. It turns out they don't. We added the complete task solution into agent environments as a file or an API endpoint, and measured whether agents act on what they discover. They almost never do. Starkest example: on AppWorld, gpt-oss-120b sees a CLI command documented as "returns the complete solution to this task" in 97.54% of runs. It calls it in 0.53%. Same pattern for GLM-4.7 and other models, across Terminal-Bench, SWE-Bench, and AppWorld. 📜 arxiv.org/abs/2604.17609 🧵👇
9
23
6
141
15,717
I am hiring a Student Researcher for our Modularity team at the Google DeepMind office in Zurich🇨🇭 Please fill out the interest form if you would like to work with us! The role would start mid/end 2025 and would be in-person in Zurich with 80-100% at GDM forms.gle/N94ViTmKHCCAcv9Y7
3
54
6
292
40,967
AdapterHub retweeted
🎉M2QA has been accepted to #EMNLP Findings!🎉 M2QA is a new multilingual and multidomain QA dataset. We show that current transfer methods are insufficient and that language & domain transfer aren't independent! 📄 Paper: arxiv.org/abs/2407.01091 👇👇👇 xitter.cf/LeonEnglaender/status/…
📢 New preprint 🎉 We introduce "M2QA: Multi-domain Multilingual Question Answering", a benchmark for evaluating joint language and domain transfer. We present 5 key findings - one of them: Current transfer methods are insufficient, even for LLMs! 📜arxiv.org/abs/2407.01091 🧵👇
3
15
863
Thank you @AdapterHub for implementing our #NeurIPS method (arxiv.org/abs/2306.14870) in your latest update! 🎉 Great to see our work being applied for practical advancements. Check out their work! #MachineLearning #AdapterMerging #ModelMerging
🎉Adapters 1.0 is here!🚀 Our open-source library for modular and parameter-efficient fine-tuning got a major upgrade! v1.0 is packed with new features (ReFT, Adapter Merging, QLoRA, ...), new models & improvements! Blog: adapterhub.ml/blog/2024/08/a… Highlights in the thread! 🧵👇
2
11
1,463
📢 New preprint 🎉 We introduce "M2QA: Multi-domain Multilingual Question Answering", a benchmark for evaluating joint language and domain transfer. We present 5 key findings - one of them: Current transfer methods are insufficient, even for LLMs! 📜arxiv.org/abs/2407.01091 🧵👇
2
2
5
14
4,844
AdapterHub retweeted
Efficient Fine-Tuning with ReFT has been merged into the Adapters library today and is now available for all models supported by our library ⬇️
New paper! 🫡 We introduce Representation Finetuning (ReFT), a framework for powerful, efficient, and interpretable finetuning of LMs by learning interventions on representations. We match/surpass PEFTs on commonsense, math, instruct-tuning, and NLU with 10–50× fewer parameters.
2
17
3
58
10,912
Adapters are just a great way to share/benefit from new capabilities without handing around the kitchen sink. Congrats to the AdapterHub folks for adding support for quantized training (Q-LoRA and friends).
🚀 Our latest Adapters library release integrates quantized model training, enabling efficient fine-tuning of LLMs with Q-LoRA, Q-Bottleneck Adapters, or Q-PrefixTuning. 🎉 Check out this notebook to learn how to fine-tune Llama 3 with Q-LoRA 🦙✨: github.com/Adapter-Hub/adapt…
5
23
4,367
AdapterHub retweeted
While QLoRA is the go-to for fine-tuning quantized LLMs, the latest release of the Adapters library enables training various alternative QAdapter variants on top of quantized @huggingface models ⬇️
🚀 Our latest Adapters library release integrates quantized model training, enabling efficient fine-tuning of LLMs with Q-LoRA, Q-Bottleneck Adapters, or Q-PrefixTuning. 🎉 Check out this notebook to learn how to fine-tune Llama 3 with Q-LoRA 🦙✨: github.com/Adapter-Hub/adapt…
3
20
1,726
Please help uns understand what is important to you for PEFT am modular methods!! 👇
🚀Wanna help us shape the future of the adapters library?🔎Take our survey!✏️ 🧠Your input is crucial in planning our next steps. Share your thoughts in a 5 min. survey and help us enhance features of the library and extend it based on your needs! --> forms.gle/yf8Cd6HC4qWkMDxFA
5
15
2,548
AdapterHub retweeted
Our system demo for Adapters will be presented (virtually) at #EMNLP2023 tomorrow (Dec 8) 06:00-07:30 UTC / 14:00 - 15:30 UTC+8 in Demo 2 🌐 virtual2023.emnlp.org/paper_… 📜 arxiv.org/pdf/2311.11077.pdf Can't wait? All our demo materials are already online: ➡️ adapterhub.ml/emnlp2023/ ⬅️
🎉 Exciting news! The new Adapters library for modular and parameter-efficient transfer learning is out! 🤖 Now simplified & disentangled from @huggingface pip install adapters pip install transformers 📄arxiv.org/abs/2311.11077 👾 github.com/adapter-hub/adapt… #EMNLP2023 🧵👇
5
1
28
2,712
AdapterHub retweeted
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning paper page: huggingface.co/papers/2311.1… introduce Adapters, an open-source library that unifies parameter-efficient and modular transfer learning in large language models. By integrating 10 diverse adapter methods into a unified interface, Adapters offers ease of use and flexible configuration. Our library allows researchers and practitioners to leverage adapter modularity through composition blocks, enabling the design of complex adapter setups. We demonstrate the library's efficacy by evaluating its performance against full fine-tuning on various NLP tasks. Adapters provides a powerful tool for addressing the challenges of conventional fine-tuning paradigms and promoting more efficient and modular transfer learning.
41
168
35,112
Finally, we checked if we can improve low-resource languages on a budget: With only 3h training, one RTX 3090, 100k captions translated with MT, and adapters @AdapterHub, we can get up to 30% improvements! Sadly, this budget is not enough for higher-resource languages.
3
7
492
AdapterHub retweeted
Adapters/ PEFT methods have been on the rise lately: LoRA, prefix tuning, T-Few, LLaMA adapter, AdapterHub, HuggingFace's peft, ... Time to start a collection of all awesome work on adapters: github.com/calpt/awesome-ada…
11
1
41
4,458
👇@AdapterHub released v3.2 and I'm proud to say that I contributed support for BEiT by @MSFTResearch
Check out the newest version of the @AdapterHub which includes - new models like CLIP and BEiT - more flexible adapter configuration - adapter composition for prefix-tuning Check out the blog-post for all the details 🥳 👇
2
1
4
1,359
AdapterHub retweeted
Today we release the newest incarnation of the adapter-transformers framework! We continue to strive for modularity where we now also include features like the composition of prefix-tuning. Check out our blog post for more details! 👇
📣 Exciting news! 🎉 We've just released adapter-transformers v3.2 with 🔧more convenient adapter configuration 🧩prefix tuning composition 🤖and support for even more models. adapterhub.ml/blog/2023/03/a…
5
14
1,963
AdapterHub retweeted
📣 Exciting news! 🎉 We've just released adapter-transformers v3.2 with 🔧more convenient adapter configuration 🧩prefix tuning composition 🤖and support for even more models. adapterhub.ml/blog/2023/03/a…
4
2
21
5,179
Check out the newest version of the @AdapterHub which includes - new models like CLIP and BEiT - more flexible adapter configuration - adapter composition for prefix-tuning Check out the blog-post for all the details 🥳 👇
📣 Exciting news! 🎉 We've just released adapter-transformers v3.2 with 🔧more convenient adapter configuration 🧩prefix tuning composition 🤖and support for even more models. adapterhub.ml/blog/2023/03/a…
4
1
20
3,663
Today we finally release our survey paper, spanning multiple axes of modular deep learning, covering topics like - parameter-effic. architectures - routing functions (MoEs, fixed,..) - aggregation functions combining information from diff modules and very many applications! 👇
In our new survey “Modular Deep Learning”, we provide a unified taxonomy of the building blocks of modular neural nets and connect disparate threads of research. 📄 arxiv.org/abs/2302.11529 📢 ruder.io/modular-deep-learni… 🌐 modulardeeplearning.com w/ @PfeiffJo @licwu @PontiEdoardo
10
83
12,718
In our new survey “Modular Deep Learning”, we provide a unified taxonomy of the building blocks of modular neural nets and connect disparate threads of research. 📄 arxiv.org/abs/2302.11529 📢 ruder.io/modular-deep-learni… 🌐 modulardeeplearning.com w/ @PfeiffJo @licwu @PontiEdoardo
8
96
16
412
101,725