
A central repository for pre-trained adapter modules in transformers! Active maintainers: @clifapt @h_sterz @LeonEnglaender @timo_imhof @PfeiffJo
Joined May 2020
- Tweets169
- Following1.3K
- Followers1.2K
- Likes228
AdapterHub retweeted
LLM agents are assumed to integrate unexpected environmental observations into their reasoning. It turns out they don't.
We added the complete task solution into agent environments as a file or an API endpoint, and measured whether agents act on what they discover. They almost never do.
Starkest example: on AppWorld, gpt-oss-120b sees a CLI command documented as "returns the complete solution to this task" in 97.54% of runs. It calls it in 0.53%. Same pattern for GLM-4.7 and other models, across Terminal-Bench, SWE-Bench, and AppWorld.
📜 arxiv.org/abs/2604.17609
🧵👇
AdapterHub retweeted
Took Claude up for a spin on the weekend and started a quick open-source self-hosted re-implementation Thinking Machines' Tinker API: github.com/calpt/open-tinker
AdapterHub retweeted
I am hiring a Student Researcher for our Modularity team at the Google DeepMind office in Zurich🇨🇭
Please fill out the interest form if you would like to work with us! The role would start mid/end 2025 and would be in-person in Zurich with 80-100% at GDM
forms.gle/N94ViTmKHCCAcv9Y7
AdapterHub retweeted
🎉M2QA has been accepted to #EMNLP Findings!🎉
M2QA is a new multilingual and multidomain QA dataset. We show that current transfer methods are insufficient and that language & domain transfer aren't independent!
📄 Paper: arxiv.org/abs/2407.01091
👇👇👇
xitter.cf/LeonEnglaender/status/…
📢 New preprint 🎉
We introduce "M2QA: Multi-domain Multilingual Question Answering", a benchmark for evaluating joint language and domain transfer.
We present 5 key findings - one of them: Current transfer methods are insufficient, even for LLMs!
📜arxiv.org/abs/2407.01091
🧵👇
AdapterHub retweeted
Thank you @AdapterHub for implementing our #NeurIPS method (arxiv.org/abs/2306.14870) in your latest update! 🎉 Great to see our work being applied for practical advancements. Check out their work! #MachineLearning #AdapterMerging #ModelMerging
🎉Adapters 1.0 is here!🚀
Our open-source library for modular and parameter-efficient fine-tuning got a major upgrade! v1.0 is packed with new features (ReFT, Adapter Merging, QLoRA, ...), new models & improvements!
Blog: adapterhub.ml/blog/2024/08/a…
Highlights in the thread! 🧵👇
AdapterHub retweeted
📢 New preprint 🎉
We introduce "M2QA: Multi-domain Multilingual Question Answering", a benchmark for evaluating joint language and domain transfer.
We present 5 key findings - one of them: Current transfer methods are insufficient, even for LLMs!
📜arxiv.org/abs/2407.01091
🧵👇
AdapterHub retweeted
Efficient Fine-Tuning with ReFT has been merged into the Adapters library today and is now available for all models supported by our library ⬇️
New paper! 🫡
We introduce Representation Finetuning (ReFT), a framework for powerful, efficient, and interpretable finetuning of LMs by learning interventions on representations. We match/surpass PEFTs on commonsense, math, instruct-tuning, and NLU with 10–50× fewer parameters.

ALT Figure 1: Parameter count vs. performance for LoReFT and other PEFTs across four benchmarks when applied to LLaMA, Llama-2, and RoBERTa models. Despite training much fewer parameters than existing PEFTs, LoReFT achieves competitive or even state-of-the-art performance on all tasks. Its value is most apparent for the largest models in our evaluations. Note: FT refers to full-parameter finetuning, which is not a PEFT or ReFT method.
AdapterHub retweeted
Adapters are just a great way to share/benefit from new capabilities without handing around the kitchen sink.
Congrats to the AdapterHub folks for adding support for quantized training (Q-LoRA and friends).
🚀 Our latest Adapters library release integrates quantized model training, enabling efficient fine-tuning of LLMs with Q-LoRA, Q-Bottleneck Adapters, or Q-PrefixTuning. 🎉
Check out this notebook to learn how to fine-tune Llama 3 with Q-LoRA 🦙✨:
github.com/Adapter-Hub/adapt…
AdapterHub retweeted
While QLoRA is the go-to for fine-tuning quantized LLMs, the latest release of the Adapters library enables training various alternative QAdapter variants on top of quantized @huggingface models ⬇️
🚀 Our latest Adapters library release integrates quantized model training, enabling efficient fine-tuning of LLMs with Q-LoRA, Q-Bottleneck Adapters, or Q-PrefixTuning. 🎉
Check out this notebook to learn how to fine-tune Llama 3 with Q-LoRA 🦙✨:
github.com/Adapter-Hub/adapt…
AdapterHub retweeted
Please help uns understand what is important to you for PEFT am modular methods!!
👇
🚀Wanna help us shape the future of the adapters library?🔎Take our survey!✏️
🧠Your input is crucial in planning our next steps. Share your thoughts in a 5 min. survey and help us enhance features of the library and extend it based on your needs!
--> forms.gle/yf8Cd6HC4qWkMDxFA
AdapterHub retweeted
Our system demo for Adapters will be presented (virtually) at #EMNLP2023 tomorrow (Dec 8) 06:00-07:30 UTC / 14:00 - 15:30 UTC+8 in Demo 2
🌐 virtual2023.emnlp.org/paper_…
📜 arxiv.org/pdf/2311.11077.pdf
Can't wait? All our demo materials are already online: ➡️ adapterhub.ml/emnlp2023/ ⬅️
🎉 Exciting news! The new Adapters library for modular and parameter-efficient transfer learning is out! 🤖
Now simplified & disentangled from @huggingface
pip install adapters
pip install transformers
📄arxiv.org/abs/2311.11077
👾 github.com/adapter-hub/adapt…
#EMNLP2023
🧵👇
AdapterHub retweeted
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning
paper page: huggingface.co/papers/2311.1…
introduce Adapters, an open-source library that unifies parameter-efficient and modular transfer learning in large language models. By integrating 10 diverse adapter methods into a unified interface, Adapters offers ease of use and flexible configuration. Our library allows researchers and practitioners to leverage adapter modularity through composition blocks, enabling the design of complex adapter setups. We demonstrate the library's efficacy by evaluating its performance against full fine-tuning on various NLP tasks. Adapters provides a powerful tool for addressing the challenges of conventional fine-tuning paradigms and promoting more efficient and modular transfer learning.
AdapterHub retweeted
Finally, we checked if we can improve low-resource languages on a budget:
With only 3h training, one RTX 3090, 100k captions translated with MT, and adapters @AdapterHub, we can get up to 30% improvements!
Sadly, this budget is not enough for higher-resource languages.
AdapterHub retweeted
Adapters/ PEFT methods have been on the rise lately:
LoRA, prefix tuning, T-Few, LLaMA adapter, AdapterHub, HuggingFace's peft, ...
Time to start a collection of all awesome work on adapters:
github.com/calpt/awesome-ada…
AdapterHub retweeted
👇@AdapterHub released v3.2 and I'm proud to say that I contributed support for BEiT by @MSFTResearch
Check out the newest version of the @AdapterHub which includes
- new models like CLIP and BEiT
- more flexible adapter configuration
- adapter composition for prefix-tuning
Check out the blog-post for all the details 🥳
👇
AdapterHub retweeted
Today we release the newest incarnation of the adapter-transformers framework!
We continue to strive for modularity where we now also include features like the composition of prefix-tuning.
Check out our blog post for more details!
👇
📣 Exciting news! 🎉
We've just released adapter-transformers v3.2 with
🔧more convenient adapter configuration
🧩prefix tuning composition
🤖and support for even more models.
adapterhub.ml/blog/2023/03/a…
AdapterHub retweeted
📣 Exciting news! 🎉
We've just released adapter-transformers v3.2 with
🔧more convenient adapter configuration
🧩prefix tuning composition
🤖and support for even more models.
adapterhub.ml/blog/2023/03/a…
AdapterHub retweeted
Check out the newest version of the @AdapterHub which includes
- new models like CLIP and BEiT
- more flexible adapter configuration
- adapter composition for prefix-tuning
Check out the blog-post for all the details 🥳
👇
📣 Exciting news! 🎉
We've just released adapter-transformers v3.2 with
🔧more convenient adapter configuration
🧩prefix tuning composition
🤖and support for even more models.
adapterhub.ml/blog/2023/03/a…
AdapterHub retweeted
Today we finally release our survey paper, spanning multiple axes of modular deep learning, covering topics like
- parameter-effic. architectures
- routing functions (MoEs, fixed,..)
- aggregation functions combining information from diff modules
and very many applications!
👇
In our new survey “Modular Deep Learning”, we provide a unified taxonomy of the building blocks of modular neural nets and connect disparate threads of research.
📄 arxiv.org/abs/2302.11529
📢 ruder.io/modular-deep-learni…
🌐 modulardeeplearning.com
w/ @PfeiffJo @licwu @PontiEdoardo
AdapterHub retweeted
In our new survey “Modular Deep Learning”, we provide a unified taxonomy of the building blocks of modular neural nets and connect disparate threads of research.
📄 arxiv.org/abs/2302.11529
📢 ruder.io/modular-deep-learni…
🌐 modulardeeplearning.com
w/ @PfeiffJo @licwu @PontiEdoardo























