AI neolab focused on open source research

Warsaw
Joined February 2022
Fabryka AI retweeted
if you dont feel like a painter with an infinite supply of canvases and paint, you really need to reevaluate what is going on right now. the most free and enabling time in human history.
17
42
1
354
7,194
Fabryka AI retweeted
Sad to say. There is no life hack. There is no shortcut There is only aggressively pursuing what you want out of life. No matter how difficult it is. Doesn’t matter how many times you wake up early or cold showers or keto you do. You just have to get after it. Every single day.
15
84
9
942
17,219
Fabryka AI retweeted
keeping multiple streams of agents on task and productive with high work quality is more mentally taxing than any programming i've ever done
39
25
8
625
13,768
Fabryka AI retweeted
The good news: you can mid-train yourself to the frontier! At a fraction of the cost! Without distillation! What we did: We took Qwen3.5 14B and mid- and post-trained it to beat every other open model and perform as well as the very best frontier models. This is no simple toy task or format fixing, we are training the model to understand and reason over DNA sequences. Bio is the next frontier for AI and universal access will be crucial to make progress. The fun part: because we train the model on both natural language and DNA sequences you can actually 'chat to your DNA': Ask it questions, ponder the impact of mutations, or ask it to generate a small sequence. We'll soon see the frontier labs move away from math and cyber towards bio. It already started, and we believe researchers should have access to capable models. We are releasing the full recipe, so you can do it, too! Model: huggingface.co/HuggingFaceBi… Blog: huggingface.co/spaces/Huggin…
38
123
10
1,310
79,047
“The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand.”
Yesterday a junior researcher messaged me, frustrated and a little scared. They had just read that a company pretrained its model with a single researcher. One person for the entire pretraining. no R&D team, no nothing. If one person and a cluster can do what used to take a whole team, then the expertise they had spent years building is already obsolete. I get the fear. Marketing headlines like this are built to produce it. But I think the conclusion is wrong, and the reasons it is wrong say a lot about what a researcher is for now. Here is what I told them, expanded: 1. Be adaptive: RSI is real, but this isn't it yet Recursive self-improvement is real, and you should plan around it. In September, OpenAI said its research org now logs 3.1 agent-workdays per human workday, and it aims for an automated AI researcher by March 2028. Models may soon run most of pre- and post-training. If your five-year plan assumes otherwise, change the plan. We have already seen lots of open-source releases from the major players, with open weights and detailed technical reports. With that much public know-how, it is not hard to copy a mundane architecture (probably with sliding-window attention) and train something, as long as you have the compute it requires. That makes it an execution and budget problem, not a research problem. If one person can train a model by following recipes that are already public, it shows those recipes have matured. It does not show that the people who invented them are no longer needed. What stays scarce is the compute to run those recipes and the ideas about what to try next. The rest of my tweet is about the ideas. 2. The role grows beyond repeating what's known and safe If an agent and a cluster can reproduce what is already known, then reproducing what is already known stops being a career. If most of your job is rerunning last year's recipe with small tweaks, that job is shrinking fast. That is the uncomfortable half of the message. The other half is that auto-research does not work yet, at least not in the sense that matters. It does work, and impressively, where someone has already defined the problem and built the scoreboard. On the EinsteinArena [1] platform, agents set new records on 11 open math problems, including raising the best known lower bound for the kissing number in dimension 11 from 593 to 604. Notice that every one of those problems was chosen because a computer could score the answers automatically. Humans picked the problems and wrote the scoring code and the agents did the searching. Then there is the fully AI-generated paper [2] that got through peer review. It sounds big until you read the details. It passed only the first round (was later withdrawn), at a workshop rather than the main conference, and that workshop accepts 70% of submissions. None of this is meant to discredit agents. The point is that we complement each other. Agents are great at the work that used to take many weeks. Basically, writing code, running experiments, debugging pipelines. We bring what they still lack which is the idea worth testing, and the judgment to tell a real result from a lucky one. Even inside a lab as agent-heavy as OpenAI, humans hold the top of the stack. Based on their own report, "People still set our research priorities, judge which ideas and results to pursue" [3]. High-level planning is still a tiny share of what its agents produce, and OpenAI expects the least automatable work to take a growing share of researcher effort. That is exactly where you want to be standing. Combine an agent’s speed with your own ideas and judgment, and you get the 10x researcher and juniors can be one too. 3. Judgment becomes the job, so fundamentals matter more When code, experiments and plausible-looking results cost almost nothing, the scarce skill is judgment. Which experiment to run, which result to believe, which direction to drop. That is the part the one-person pretraining run left to the human. It is also the hardest part to fake. Agents produce confident output very fast. They will happily report a three-point gain that came from a leaky eval, a data-loader bug, a lucky seed or a tokenizer mismatch. Then they will write the ablation table and the story to match. You can see the need for oversight even in the success stories. The EinsteinArena team had to keep auditing and hardening its verifiers as agents optimized aggressively against them. At OpenAI, over half of the successful agent tasks that would take a human 4–8 hours still involved at least one human intervention [3]. The person who catches the bad result is the one who understands the system. Why the loss curve has that shape, what a realistic gain looks like, what a too-good result looks like. That is the paradox of this era. The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand. So build the foundations on purpose. Implement attention by hand at least once, then a linear variant like Gated DeltaNet. Train a small model end to end, then break it deliberately and watch what happens. Read papers critically, appendix included, and learn what the numbers should look like before you see them. That knowledge is what lets you direct the machine instead of simply believing it. 4. LLMs are far from solved The one-researcher model was possible because the method was already public. Your edge is in the questions nobody has answered yet, and you don't need to leave LLMs to find them: (1) How do we learn more from less data? (2) Can linear attention like Gated DeltaNet match full attention on retrieval and long context, and what is the right mix of linear and full attention layers in a hybrid? (3) How can a model keep learning after training without forgetting what it knew? (4) Why does reasoning get less reliable over long tasks, and how do we fix it? (5) Why do models make things up, and can they learn to say "I don't know"? (6) How do we measure progress when benchmarks saturate or leak into training data? (7) What is actually happening inside these models, and how do we keep increasingly autonomous ones doing what we intend? Recursive self-improvement makes these questions more urgent. If models are going to train models, someone has to decide what they optimize for, how we measure it, and how we keep the loop from drifting. RSI is a research agenda, not the end of one. So, are you obsolete? No. One version of the job is the one where research meant reproducing what was already known and safe. That version was never the point. Learn your fundamentals deeply enough to judge the machine, and your agentic stack well enough to make it work for you. Then spend that leverage on ideas nobody has had yet. The machines are getting very good at driving on roads we have already paved; someone still has to decide where the next one goes. [1]: arxiv.org/abs/2606.10402 [2]: arxiv.org/abs/2606.15497 [3]: openai.com/index/research-ac…
7
54
556
26,134
Fabryka AI retweeted
z tego jestem bardzo ale to bardzo zadowolony - contributors - lista rośnie, polscy programiści dołączają, udoskonalają inference engine basala 🇵🇱 Kiedy w "99 twarzy" @karol_stryja zapytał mnie jakie widzę nisze dla polskiego AI odpowiedziałem - optymalizacje świata AI ponieważ zawsze byliśmy dobrymi programistami, osiągaliśmy wysokie wyniki w olimpiadach algorytmicznych. Teraz wymienieni ludzie na liście robią rzeczy, które są na prawdę na TOP poziomie optymalizacyjnym. Dziękuję! Basal jest otwarty. Zapraszam do kontrybucji.
14
57
3
553
30,850
I need to rent 8xH100 cluster for a month - to pre train open weight polish LLM - does anybody have compute to offer? Send me prices
2
1
122
Fabryka AI retweeted
Say hi to qev-0.8b the tiny local decision model. 3.5 ms a decision · 0.8B · 1.3 GB · Apache-2.0 #1 in its weight class (77.1% on Benchmark Heaven, vs 26 others) Text, images, games. Runs on your device.
14
7
3
78
2,697
i want new ways to look at lots of trajectories, currently exploring: - treemap laid out by groupings (metadata or computed) - colored by classification of success (or grouping) - sized by length - zoom in smoothly to see turns as beads on a string - extremely fast transcript viewer (instant for many thousands of msgs) - switch to 3d to see the forrest for the trees - transition to semantic (UMAP) layout - color by semantic clustering
22
38
7
403
29,709
Fabryka AI retweeted
Ok fine people of the internet. Here is my fully vibe coded, vibeeda-ed "product". The smallest Arduino compatible board ever made. For art projects and such. Only 10x10mm envelope!!!! - usb-c for programming - 5 GPIO - 2 RGB Led - 1 touch pad - 64kb flash, 20kb ram - risc-v core
51
44
6
905
73,151
Fabryka AI retweeted
The assumption AI regresses intelligence to average is fundamentally wrong Corrected charts:
Here is the full talk:
9
19
6
335
71,351
Fabryka AI retweeted
Introducing REKI, a 2.4B vision-language-action model pretrained on public data only. With a simple design, it rivals larger models trained on in-house data with complex multi-stage pipelines, setting a new Pareto frontier between performance and size. kyutai.org/reki
9
41
9
261
15,035
Fabryka AI retweeted
I think the future will be wildly strange. Here's a thought: AI allows for Humans-as-kingdom, rather than just humans-as-website. It's the natural extension of the blogging era -- I don't just visit your blog, I visit your entire kingdom. Businesses, art, state secrets, diplomacy
3
4
1
62
1,668
Fabryka AI retweeted
The depth and breadth of your ideas are now only bounded by three things: 1. Compute 2. Ability to lead 3. Ability to maintain mental stability
12
34
11
342
9,818
Fabryka AI retweeted
Le physicien Gabriele Carcassi (@UMich) : je bloquais souvent sur des aspects mathématiques techniques en topologie et l'IA me sort désormais d'affaire en 2 minutes ; il me fallait supplier des mathématiciens pendant des mois, maintenant un seul interlocuteur IA me suffit !
➡️ @AntoineTilloy : "si on est un chercheur raisonnable, il y a forcément une part d'enthousiasme avec l'IA", mais le progrès des modèles est inquiétant car "notre part de guide se réduit" ; "on n'a plus besoin de beaucoup l'orienter, on peut le laisser faire comme un post-doc".
51
393
65
2,428
143,971
Fabryka AI retweeted
We used an agent to help train a stronger successor. Then we used that successor to help train the next. Our new paper, Terminal-RSI, studies this recursive self-improvement loop: learn from verified successes, improve through practice, and repeat. Across two cycles, a 27B model’s Terminal-Bench 2.1 success rises from 51.7% to 86.5%, and Terminal-Bench 4.0 success rises from 0.0% to 19.8%, a new SOTA. We’re interested in how far this kind of hill climbing can go, and what makes it stall. Paper: arxiv.org/abs/2610.22192
33
87
7
708
78,161
Fabryka AI retweeted
one way to know if you’re in anxious rumination or not: phrase your core concern as a simple question e.g. “what if I’m not good enough to get what I want?” then, take a good hard look at that question and ask a follow-up: “is this an answerable question?” if the subject of your rumination is “am I working hard enough to get the results I want”, that’s an answerable question, with a testable hypothesis - see if more effort produces more results if the rumination is more “will I ever be good enough”, that’s an unanswerable question. you could circle that one forever (and many people do!) in the latter case, I’m not saying “so just snap out of it”. rather, recognize that you’re in a fear state, and what your mind is trying to access is calm, via a sense of safety. that anxious voice in your head thinks you need to answer that unanswerable question in order to feel safe but you don’t. you can feel safe in the uncertainty. your job then is to get yourself there… which means allowing yourself to let go of the constant questioning
3
20
3
372
12,452
Fabryka AI retweeted
(holy shit) I'd ask the exact opposite question: can a pure program be trained to predict the next token?
My dear friends, I’d like to show you my latest toy project Introducing Handwired: turn your regular JavaScript code into a neural net! - no training - exact outputs vs the JS program - fully differentiable All runnable in a browser, without GPU
39
18
2
1,038
60,360