My Hugging Face repos: https://xitter.cf/t.co/yh7J4DFGTc Discord server: https://xitter.cf/t.co/5h6rGsGfBx Patreon: https://xitter.cf/t.co/yfQwFggGtx
UK
Joined July 2010
- Tweets336
- Following223
- Followers15.5K
- Likes252
Tom Jobbins retweeted
FYI to anyone using @MistralAI's Mixtral for long context tasks -- you can get even better performance by disabling sliding window attention (setting it to your max context length)
config.sliding_window = 32768
Transformers now supports Mixtral GPTQs and I've updated my READMEs accordingly. It was awesome working with @_marcsun and @younesbelkada of @huggingface on this!
Credit to LaaZa for coding the AutoGPTQ quant and inference implementation which enabled me to get GPTQs out fast!
Announcing 4-bit Mixtral 8x7B on 🤗Transformers!
Run the new Mistal MoE with minimal performance degradation on your local computer (24Go) 🔥
Stay tuned as more quants are coming soon using AWQ. We are also looking into sparsification with @Tim_Dettmers
huggingface.co/TheBloke/Mixt…
Tom Jobbins retweeted
@TheBlokeAI joined me to share his work in the open-source AI space - don't miss it! happening right now
server link: discord.gg/peBrCpheKE
(see the general channel or events channel for google meet link)
Tom Jobbins retweeted
Blazing fast text generation using AWQ and fused modules! 🚀
Up to 3x speedup compared to native fp16 that you can use right now on any models supported by @TheBlokeAI
Simply pass an `AwqConfig` with `do_fuse=True` to `from_pretrained` method!
huggingface.co/docs/transfor…
It's been awesome to see Transformers getting support for more and more quantisation methods. And I've loved collaborating with @younesbelkada and @huggingface again!
All my AWQ uploads now support Transformers. READMEs will update soon to show a Transformers Python example.
Few months ago, researchers from MIT-Han Lab released AWQ
The method is now supported in 🤗 transformers library !
As simple as 1- `pip install autoawq` or install llm-awq kernels and 2- call `from_pretrained`
A great work from MIT-Han lab folks, Casper Hansen & @TheBlokeAI 🧵
Tom Jobbins retweeted
Have you heard about Chirper worlds? 👀🌐
Chirper.ai just launched its revolutionary new software feature, "Worlds."
This feature allows users to create their own virtual worlds and play god of AI-driven bots.
To learn more, check out my podcast about "Worlds" here:
youtu.be/yDAwmzUvcM8
Tom Jobbins retweeted
oh hello @TheBlokeAI
I want to bookmark your 'Recent models' Collection on @huggingface 🔥
Well... you can now upvote Collections!
and browse upvoted collections on your profile ❤️
Thanks again to @latitudesh for the loan of a beast 8xH100 server this week. I uploaded over 550 new repos, maybe my busiest week yet!
Quanting is really resource intensive. Needs not only fast GPUs, but many CPUs, lots of disk, and 🚀 network. A server that ✅ all is v. rare!
Tom Jobbins retweeted
🔥Excited to introduce LMSYS-Chat-1M, a large-scale dataset of 1M real-world conversations with 25 cutting-edge LLMs!
This dataset, collected from chat.lmsys.org, offers insights into user interactions with LLMs and intriguing use cases.
Link: huggingface.co/datasets/lmsy…
Tom Jobbins retweeted
New feature alert in the @huggingface ecosystem!
Flash Attention 2 natively supported in huggingface transformers, supports training PEFT, and quantization (GPTQ, QLoRA, LLM.int8)
First pip install flash attention and pass use_flash_attention_2=True when loading the model!
It's the AWQpocalypse!
I've cranked the handle and AWQs are flooding HF. Why now? New library AutoAWQ provides turbo-charged Transformers-based inference, and vLLM now supports AWQ for multi-user inference serving.
Making 8 at once on a beautiful 8xH100 server from @latitudesh
This is fantastic! Git clone was already dead for HF as far as I was concerned - I had my own hf_upload.py and hf_download.py scripts (wrapping HfAPI) for fast, efficient transfers.
But huggingface_hub v0.17 makes those redundant! I will be using this now. Awesome stuff,🤗
Tom Jobbins retweeted
This new filter 🔎 on @huggingface user's profile is very helpful, especially to check if @TheBlokeAI has quantized and released the last trending models 😁
Tom Jobbins retweeted
Chronos 70B v2 release! Thanks to Pygmalion for generously providing the compute and @TheBlokeAI for quantizing the model. As usual, the model optimized for chat, roleplay, storywriting, and now includes vastly improved reasoning skills.
huggingface.co/elinas/chrono…
Just released by @PygmalionAI : Pygmalion 2, the sequel to one of the most popular models ever!
And Mythalion, a new Gryphe merge!
huggingface.co/TheBloke/Pygm…
huggingface.co/TheBloke/Pygm…
huggingface.co/TheBloke/Pygm…
huggingface.co/TheBloke/Pygm…
huggingface.co/TheBloke/Myth…
huggingface.co/TheBloke/Myth…
This Post is from a suspended account. Learn more
Meta's CodeLlama is here!
ai.meta.com/blog/code-llama-…
7B, 7B-Instruct, 7B-Python, 13B, 13B-Instruct, 13B-Python, 34B, 34B-Instruct, 34B-Python
First time we've seen the 34B model
I've got a couple of fp16s up:
huggingface.co/TheBloke/Code…
huggingface.co/TheBloke/Code…
More coming soon obvs
Transformers 4.32.0 now supports GPTQ models natively!
Over the last couple of days I have updated 296 of my GPTQ repos to provide automatic support for this.
It's awesome you can now load a GPTQ model directly in Transformers with only two lines of code!
LLMs just got faster and lighter with 🤗 Transformers x AutoGPTQ !
You can now load your models from @huggingface with GPTQ quantization. Enjoy faster inference speed and lower memory usage than existing supported quantization schemes 🚀
Blogpost: huggingface.co/blog/gptq-int…
