@mztwo

Writing on the world of AI at @ArtisanaAI

 New York
artisana.ai
Joined April 2009
  • Tweets215
  • Following129
  • Followers249
32Media
Michael Zhang retweeted
Hacker News 50@betterhn5016 Jun 2023
Meta's plan to offer free commercial AI models puts pressure on Google, OpenAI artisana.ai/articles/metas-p… (news.ycombinator.com/item?id…)

Meta's Plan to Offer Free Commercial AI Models Puts Pressure on Google and OpenAI

Meta's latest strategy to offer its AI models for free commercial use turns up the heat on OpenAI and Google and their closed-source AI business model.

artisana.ai
1
2
263
I am loving the AI model wars! artisana.ai/articles/metas-p…

Meta's Plan to Offer Free Commercial AI Models Puts Pressure on Google and OpenAI

Meta's latest strategy to offer its AI models for free commercial use turns up the heat on OpenAI and Google and their closed-source AI business model.

artisana.ai
2
1
12
2,270
Mobile devices could start to fine-tune LLMs soon. This opens up so many options for data privacy, personalized LLMs, and more.
1
2
210
Open-source is emerging as an even bigger threat to closed-source. Many of these closed-source models haven't even considered using LoRA fine-tuning, and instead prefer to train from scratch. The pace is real.
1
1
205
For more on this, see my full breakdown: artisana.ai/articles/qlora-e… Other links: Paper: arxiv.org/abs/2305.14314 Code+Demo: github.com/artidoro/qlora Samples: colab.research.google.com/dr… Colab: colab.research.google.com/dr…

Novel QLoRA Approach Unlocks AI Fine-Tuning on Consumer GPUs

Researchers have unveiled QLoRA, a novel and highly efficient 4-bit method for fine-tuning AI models that is able to run on single professional and consumer GPUs. This dramatic increase in efficiency...

artisana.ai
176
What does this mean for the future of AI? Here's where things get interesting.
1
71
more replies
Frequent fine-tuning enables models to incorporate real-time info. By bringing cost down, this is more possible.
1
89
Producing highly capable, state of the art models no longer requires expensive compute for fine-tuning. You can do it with minimal commercial resources or on a RTX 3090 now. Everyone can be their own mad scientist.
1
85
They quantize the quantization constants. This is akin to compressing their compression formula as well. And memory spikes typical in fine-tuning are optimized, which reduces max memory load required.
1
58
The results are noteworthy: The 65B and 33B Guanaco variants consistently matched ChatGPT-3.5's performance. While the Vicuna benchmarking is imperfect (the researchers note that extensively), it's nonetheless significant and newsworthy.
1
115
How does the science work here? Three things to know:
1
45
A special 4-bit NormalFloat data type is efficient at being precise, versus the 16-bit floats and integers which are memory-intensive. Best way to think about this is that it's like compression (but not exactly the same).
1
48
This is open-sourced and available now. Huggingface already enables you to use it. Things are moving at 1000 mph here.
1
44
A commercial GPU with 48GB of memory is now able to produce the same fine-tuned results as the same 16-bit tuning requiring 780GB of memory. This is a massive decrease in resources.
1
37
QLoRA is a novel approach that enables 4-bit finetuning of LLMs. Let's talk about why this is a milestone moment for open-source LLMs. 👇
1
1
1
367
more replies
It's so efficient that researchers were able to fine-tune a 33B parameter model on a 24GB consumer GPU (RTX 3090, etc.) in 12 hours, which scored 97.8% in a benchmark against GPT-3.5.
1
51
QLoRA is an even more efficient way of fine-tuning which truly democratizes access to fine-tuning (no longer requiring expensive GPU power). It's a 4-bit approach that generates performance similar to 16-bit finetuning methods.
1
53
Fine-tuning an existing model is already a popular and cost-effective way to enhance an existing LLMs capabilities. It's cheaper than training from scratch, but it can still cost $$$.
1
43
OpenAI CEO Sam Altman testified for three hours today before the US Senate. Unlike typical hearings, this one had substance. Here's what you need to know:
1
1
1
4
367
more replies
P.S. If you like this kind of analysis, I write a free newsletter that tracks the biggest issues and implications of generative AI tech. It's sent once a week and helps you stay up-to-date in the time it takes to have your Sunday morning coffee. artisana.beehiiv.com/subscri…

Artisana: Generative AI News

High-impact AI news in easy bites. No fluff, no BS.

artisana.beehiiv.com
1
143
That's a wrap! If you enjoyed this thread: 1. Follow me @mztwo for more of these 2. RT the tweet below to share this thread with your audience
Michael Zhang@mztwo16 May 2023
OpenAI CEO Sam Altman testified for three hours today before the US Senate. Unlike typical hearings, this one had substance. Here's what you need to know:
117
Load more