Philippe MARTIN (@feloy2) header
Philippe MARTIN (@feloy2) profile picture

Software engineer @ Red Hat Apress author: https://xitter.cf/t.co/vuzXzAxhmd

Bordeaux, France
Joined March 2011
Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)Photo by Philippe MARTIN (@feloy2)
Viendez me voir au salon du fantastique à Paris ! Plus que demain et après demain pour les collectors ! #salonfantastique #litterature
3
13
654
Me cracking the Fizz-Buzz interview 😅🤪 These machines become really human 🚀
1
2
103
#LAmourOuf c'est vraiment ouf 🤪 on prend une grosse vague d'amour, ça fait du bien 😍
1
2
313
Je viens de lire tout le thread, ben maintenant j'ai mal à la tête... A ne pas relire avant 6 heures 😅
« Le paracétamol, c’est 1g toutes les 6h » Vous avez tous sûrement déjà entendu cette phrase. Mais vous êtes-vous déjà demandé comment on a décidé de cette dose et de ce délai entre 2 prises ? Petit zoom sur : la PHARMACOCINÉTIQUE 💊✨ PARTIE 1 ‼️
92
"You are multiplying matrices and this thing starts talking to you" 😅 #devoxxBe
72
Replying to @Duralex
@Duralex Je suis de tout coeur avec vous dans votre nouvelle aventure de scop, mais il semblerait que vous ayez des problèmes de logistique 🥺 j'avais demandé une expédition à un point relais, qui était prévue aujourd'hui, et le livreur m'indique qu'ils n'ont pas trouvé l'adresse
2
32
D'après l'adresse que me donne le livreur, il n'y a aucune mention de point relais, comme si le point relais était mon adresse perso
23
J'aimerais pouvoir vous contacter via le site, mais mon cas ne semble pas prévu, et je n'ai trouvé aucun champ libre pour m'expliquer
16
#stubb @UBBrugby vient de remettre St André au centre de la ville 🏈🎉🥳
261
🎉 Devoxx Belgium is BACK. This year’s event will showcase some of the most innovative and inspiring speakers in the industry, presenting on everything from #Java, Cloud, and #AI to Security and Architecture. Are you going? devoxx.be
2
10
998
In October 7 or 8, I'll be presenting Podman AI Lab (podman-desktop.io/extensions…) at the Devoxx BE conference, in a Tools-in-action session. Registration is opening in a few hours: devoxx.be/
1
3
128
Philippe MARTIN retweeted
Replying to @rhatdan
@rhatdan has dropped two videos on his Red Hat Dan on Tech channel. The first is about his thoughts on Red Hat AI, the second, a talk about Podman AI Lab for App Dev with Generative AI. The last talk features @stevanLM Find them here on this playlist: youtube.com/playlist?list=PL…
5
4
508
J'ai beau me repasser les bérus en boucle, j'ai mal 😭
1
1
68
New language models get released every day (Gemini-1.5, Gemma, Claude 3, potentially GPT-5 etc. etc.), but one component of LLMs has remained constant over the last few years—the decoder-only transformer architecture. This architecture has five components… Why should we care? Research on LLMs moves fast. Shockingly, however, the architecture used by most modern LLMs is pretty similar to that of the original GPT model. We just make the model much larger, modify it slightly, and use a more extensive training (and alignment) process. For this reason, the decoder-only transformer architecture is one of the most fundamental/important ideas in AI research, so investing into understanding it deeply is a wise idea. (1) Input layer: Decoder-only transformers receive a textual prompt as input. We use a tokenizer—based upon an algorithm like Byte-Pair Encoding (BPE)—to break this text into discrete tokens (i.e., words or sub-words). Then, we map each of these tokens to a corresponding vector stored in an embedding layer. This process forms a sequence of token vectors that are passed to the model as input. Optionally, we can augment these token vectors with additive positional embeddings. (2) Causal self-attention is the core of the decoder-only transformer and allows the model to learn from relationships between tokens in the input. The vanilla self-attention operation transforms each token’s representation by taking a weighted combination of other token representations, where weights are given by pairwise attention/importance scores between tokens. Causal self-attention follows a similar strategy but only computes attention scores for preceding tokens in the sequence. Attention is performed in parallel across several heads (i.e., multi-head attention), each of which can focus upon different parts of the input sequence. (3) Feed-forward transformations are performed within each block of the decoder-only transformer, allowing us to individually transform each token’s representation. This feed-forward component is a small neural network that is applied in a pointwise manner to each token vector. Given a token vector as input, we pass this vector through a linear projection that increases its size by ~4X, apply a non-linear activation function (e.g., SwiGLU or GeLU), then perform another linear projection that restores the original size of the token vector. (4) Classification head: The decoder-only transformer has one final classification head that takes token vectors from the transformer’s final output layer as input and outputs a vector with the same size as the vocabulary of the model’s tokenizer. This vector can be used to either train the LLM via next token prediction or generate text at inference time via sampling strategies like nucleus sampling and beam search. (5) Transformer blocks form the body of the decoder-only transformer architecture. The exact layout of the decoder-only transformer block may change depending upon the implementation, but two primary sub-layers are always present: 1. Causal self-attention 2. Feed-forward transformation Additionally, these sub-layers are surrounded by a layer normalization module—either before or after the sub-layer (or both!)—as well as a residual connection.
12
150
5
662
74,112
Philippe MARTIN retweeted
If you are looking for a resource to understand the instruction fine-tuning process in LLMs, I've uploaded a notebook to implement the fine-tuning process from scratch: github.com/rasbt/LLMs-from-s… It explains 1. how to format the data into 1100 instruction-response pairs 2. how to apply a prompt-style template 3. and how to use masking. Of course, this also includes a section on implementing an LLM-based automated process for evaluation. Happy coding!
35
397
12
2,081
208,461
😅
Tous les racistes veulent quitter la France pour fuir la gauche mais aucun d’entre eux ne veut accueillir les étrangers qui fuient la guerre et les violences. Je vais finir par penser qu’être d’extrême droite c’est être extrêmement con.
97
I am thrilled to share that my new #textbook, Go Programming - From Beginner to Professional, is now available! 📘✨ This book serves as your #go to guide to master Go for real-world software dev success covering fundamentals to advanced topics. Linked: a.co/d/1RIEXDo
33
51
4
441
28,055