Crafting Language Models
France
Joined April 2013
- Tweets262
- Following433
- Followers312
- Likes502
Pinned Tweet
A(nother) new name, a new chapter, and even greater ambitions.
We are proud to announce that Dragon LLM is now OVHai LLM.😃
We wanted to share this important news ! linkedin.com/posts/dragonllm…
If you are in Paris, don't miss these talks!
📅 our wonderful team will be participating at 2 unmissable events tomorrow on Feb 3, go meet them.
@JG_Barthelemy and @AlexandreTL2 are presenting at the @datacraft "Mamba and AI Factories" event from 16:30 , 55 Rue La Boëtie, 75008.
👉 luma.com/u66szusv
Gaëtan Caillaud will be joining an AI-in-Finance round table organised by @LouisBachelier in collaboration with @Genci_fr and AI Factory France at Palais Brongniart from 16:30.
👉 lnkd.in/eeB_m4Be
Sign up for these events, see you there!
OVHaiLLM retweeted
Well, works really great compared to Diff Attn v1! 0.01 loss gap with a faster model
The chosen model is a fine tuned Qwen3 Reasoning 8B, we can evaluate its ability to call tools and produce structured outputs. Our Open Finance models are all there :
huggingface.co/collections/D…
OVHaiLLM retweeted
This is the best I got for Muon on a hybrid architecture compared to AdamW. Anyone knows what could be happening here ?
We can only agree with @Lagarde latest speech on EU risk of lagging on #AI.
As @AndreeaNiculcea puts it (c) ,
"We become users of someone else’s models, someone else’s compute, someone else’s innovation — with our margins squeezed and our clients drifting to whoever offers real intelligence, not generic chatbots."
#Hint 👉 Dragon LLM is not generic.
The full transcript is worth reading :
ecb.europa.eu/press/key/date…
🥳 #Adopt_AI - Grand Palais is rapidly approaching!
Iconic event at an iconic venue! 😍
We'll be at booth F6—we'd love to see you there! 🤗
📅 25-26 November
🗺️ The Grand Palais, Paris 🥖
👉 adoptai.artefact.com/partner…
Introducing LLM Pro Finance models with @AgefiFrance today.
‣ Gemma Pro Finance 12B
‣ Qwen Pro Finance 32B
‣ Llama Pro Finance 70B
They are fine tuned for FR/EN finance and show significantly better performance vs base models. Find them here 👇
One more 🍪: try or train the two Open Finance models:
‣ Llama Open Finance 8B
‣ Qwen Open Finance R 8B
huggingface.co/collections/D…
If you would enjoy some ready made cookbooks or recipes to use them with @langchain @LLamaIndex @Pydantic_ai or @Mastra, just ❤️ this post.
Today is the day. Parlez-vous finance en français dans le texte?
@DragonLLM releases its fine tuned models for french finance. Stay tuned.
OVHaiLLM retweeted
RoPE vs NoPE in hybrid linear attention models like Kimi Linear / Qwen3Next is tricky
for example, when using NoPE, we found that slowly expanding the window size of the attention layers during training (64->4k) greatly helps convergence:
You see:
- a new arch that is better and faster than full attention verified with Kimi-style solidness.
I see:
- Starting with inferior performance even on short contexts. Nothing works and nobody knows why.
- Tweaking every possible hyper-parameter to grasp what is wrong.
- Trying to find the efficient chunkwise parallelizable form to squeeze juice out of the GPU
- RoPE or NoPE, a question haunting for nights.
- Fighting buggy implementation that causes one of the long-context benchmarks drops ~20 pts.
- RL diverging. Aligning training-inference numerics.
- Dedicated efforts to make sure comparisons are solid and fair.
- Going back-and-forth in a pool of adversarial gate-keeping tests, and finally it survives.
Great teamwork!