Sovolt (@EricDelobel) header
Sovolt (@EricDelobel) profile picture

We are building things together.

Marseille, France
Joined March 2011
Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)Photo by Sovolt (@EricDelobel)
Creating is hard and also the best fulfillment source ever. 😎Watch : imagine a desirable future 🎯Catch : design the experience 🦸Make : create your playground 🤩Scale : push what's growing 🧵(1/4) Here is full framework to navigate through the creation waves 👇
1
Looking for something to talk to your family about while you’re home for the holidays? Why not give them a clear, accessible explanation of ChatGPT? Here’s a simple, three-part framework that you can use to explain generative language models to (almost) anyone… TL;DR: We can explain ChatGPT pretty easily by focusing on three core ideas. 1. Transformer architecture: the neural network architecture used by LLMs. 2. Language model pretraining: the (initial) training process used by LLMs. 3. The alignment process: how we teach LLMs to behave to our liking. Although AI researchers might know these techniques well, it is important that we know how to explain them in simple terms as well! AI is no longer just a research topic, but rather a topic of public interest. Why is this important? Generative AI has now become a popular topic among both researchers and the general public. Now more than ever before, it is important that researchers and engineers (i.e., those building the technology) develop an ability to communicate the nuances of their creations to others. A failure to communicate the technical aspects of AI in an understandable and accessible manner could lead to widespread public skepticism (e.g., research on nuclear energy went down a comparable path) or the enactment of overly-restrictive legislation that hinders forward progress in our field. (1) Transformers: Most recent generative language models are based upon the transformer architecture. Although the transformer was originally proposed with two modules (i.e., an encoder and a decoder), generative LLMs use a decoder-only variant of this architecture. This architecture takes as input a sequence of tokens (i.e., words or subwords) that have been embedded into a corresponding vector representation and transforms them via two repeated operations: - Masked self-attention: looks at other tokens in the sequence (i..e, those that precede the current token). - Feed-forward transformation: transforms each token representation individually. These two operations each play a distinct and crucial role. By stacking several blocks of masked self-attention and feed-forward transformations on top of each other, we get the neural network architecture that is used by most generative LLMs today. (2) Pretraining: Self-supervised learning refers to the idea of using signals that are already present in raw data to train a machine learning model. In the case of generative language models, the most commonly-used objective for self-supervised learning is next token prediction, also known as the standard language modeling objective. Interestingly, this objective—despite being quite simple to understand—is the core of all generative language models. To pretrain a generative language model, we first curate a large corpus of raw text (e.g., from books, the web, scientific publications, and much more) to use as a dataset. Starting from a randomly initialized model, we then pretrain the LLM by iteratively performing the following steps: 1. Sample a sequence of raw text from the dataset. 2. Pass this textual sequence through the decoder-only transformer. 3. Train the model to accurately predict the next token at each position within the sequence. “Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users.” - from InstructGPT (3) Alignment: After pretraining, the LLM can accurately perform next token prediction, but its output is oftentimes repetitive and uninteresting. The alignment process teaches a language model how to generate text that aligns with the desires of a human user. To align a language model, we first define a set of alignment criteria (e.g., helpful and harmless). To instill each of these alignment criteria within the model, we perform finetuning via supervised finetuning (SFT) and reinforcement learning from human feedback (RLHF), which together form the three-step technique for alignment proposed by InstructGPT. For more details on each of these three components, see the links in the replies.
8
48
7
343
60,452
🧪 I'm excited to share that I'll be learning something new about AI for 100 days as part of the #100DaysOfAI challenge! Starting January 1st! Join me via @100DaysAI! 100daysai.com/
100
Free AI (closed) vs Free AI (open source)
88
252
29
3,627
444,232
Access Mixtral with the fastest inference performance anywhere! Up to 100 token/s for $0.0006/1K tokens — to our knowledge the fastest performance at the lowest price! Mixtral-8x7b-32kseqlen @MistralAI & DiscoLM-mixtral-8x7b-v2 are live on Together API! together.ai/blog/mixtral
44
137
55
1,174
925,077
Replying to @MistralAI
Mistral is quickly becoming my most favorite AI company
1
5
1
326
64,686
Sovolt retweeted
Why do you hate the boat so much?
235
1,674
263
10,319
1,334,810
"In the intricate labyrinths of tomorrow’s manors, where gears hum in harmony with the whispers of dreams, humanity must tread lightly; for the marvels we craft hold the keys to both our greatest triumphs and our deepest sorrows."
56
"In the aether-woven tapestry of time and mechanics, man's thoughts find roots in the gears of yesterday, bloom amid the mysteries of the present, and send seeds into the infinite jungles of possibilities yet to be explored." - ChatGPT
44
Probably the best thing you'll see today. In 2017, a group of developers hilariously competed for who could create worst volume control interface in the world. The results 🧵 1/22
1,377
40,281
5,849
163,868
22,884,228
Playing with an @OpenAI fine-tuned model trained on openbuild library to generate @Bubble components. All in all, the challenge might not be technical but actually human. How non-tech people will ask the machine what they want? The future of no-code will be exciting! #nocode #ai
7
5
5
57
12,253
Week-end exploration : a plugin offering an animations engine/editor for @bubble apps. Quite some work to do still but definitely happy with the first results! Curious about Bubblers thoughts about such a plugin? #nocode #design #bubble
9
4
57
5,758
Schrödinger's Plates: They are both broken and not broken until you open the door.
1,872
21,412
1,745
203,702
We are thinking about creating a PRO Bubblers cohort and community @bubble @nocodebackend 😎 Who wants to be in? 💥 What do you think that a collective of pro bubblers could leverage for you? 👀 What existing format are you thinking of?
1
🎨The artist [Watcher] Where do you see beauty? What makes your energy vibration rate rise? What can you sketch right now? “Everything has its beauty, but not everyone sees it.” – Andy Warhol.
1
✨The Visionary [Watcher] What could happen? How could our future be? How things could be really different? “The difference between a dreamer and a visionary is that a dreamer has his eyes closed and a visionary has his eyes open” - Martin Luther King, Jr.
✌️I would like to connect with product builders who use @bubble to build their next project, company, side project. To connect and share tips / challenges on day to day progress #buildinpublic #NoCode
1
1
3