@minishlab

Building Model2Vec, Semble, SemHash, and the Potion embedding models. GitHub: https://xitter.cf/t.co/WAoFTUFnWm. HuggingFace: https://xitter.cf/t.co/1khDD6YCO9

minish.ai/
Joined January 2025
  • Tweets60
  • Following14
  • Followers119
13Media
We’ve just released Semble v0.3.0! Biggest changes: - Automatic disk caching: First search builds the index; later searches are (much) faster. - Search more than code: new `--content` flag for searching docs, config, code, or all of them together. 🧵
1
1
4
147
- JSON output: CLI and MCP results are now structured JSON, making Semble easier to use from agents and scripts. - Better file control: use `.sembleignore` to exclude files or force-include custom extensions. Release notes: github.com/MinishLab/semble/…

Release v0.3.0 · MinishLab/semble

What's Changed feat: Add --content flag for docs, config, and code search by @Pringled in #142 feat: add index persistence by @stephantul in #140 feat: add JSON output by @stephantul in #147 c...

github.com
1
76
Today we're releasing Semble, a fast and accurate code search library built for agents 🤖! We're also releasing potion-code-16M, a small code-specialized static embedding model that powers Semble. 🧵
1
1
2
11
853
Main features: - Fast: indexes a full codebase in ~250 ms and answers queries in ~1.5 ms, all on CPU - Accurate: on par with transformers - MCP server: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent - Zero setup: no API keys, no GPU 🧵
2
1
147
Semble: github.com/MinishLab/semble Benchmarks: github.com/MinishLab/semble/… How it works: github.com/MinishLab/semble#… Model: huggingface.co/minishlab/pot…

GitHub - MinishLab/semble: Fast and Accurate Code Search for Agents. Uses 99% fewer tokens than...

Fast and Accurate Code Search for Agents. Uses 99% fewer tokens than grep+read - MinishLab/semble

github.com
122
Model2Vec 0.7.0 is out now, as well as a blogpost on model size reduction techniques! This release features a large number of ways to improve the distillation process. - Vocabulary quantization - Configurable pooling - A number of small improvements and bugfixes 🧵
2
1
1
8
438
We also have a new blogpost on model size reduction where we showcase how to reduce model size by a factor of 15, creating a 6MB model (!) without impacting performance. Links: Release notes: github.com/MinishLab/model2v… Blogpost: minish.ai/blog/2025-10-05-si…
1
126
semhash is so so so convenient and fast
2
1
2
16
2,031
We have a new website (and name): minish.ai We’ve been working on an improved website for a while, and it’s finally here. It has documentation for all our packages as well as our blog. More things coming soon! 🚀

Minish

Fast open-source NLP models and packages

minish.ai
1
9
252
Some guy forked our "model2vec-rs" crate, and put it under the "model2vec" name on crates io and then didn't tell us about it. See here: crates.io/crates/model2vec Like what's the goal here except name squatting.
1
1
5
300
We just released model2vec 0.6.0! This is a big release, containing many big improvements 🔥 GitHub release: github.com/MinishLab/model2v… PyPi release: pypi.org/project/model2vec/0… 🧵
1
1
3
134
more replies
- Model improvements: nearly all distilled models will perform better, especially in STS and clustering tasks. A while ago we published a blog post on modernbert not working, but we now found out why, and fixed it! 🧵
1
1
56
- Smaller tokenizers: all tokenizers are now 40% smaller, at no cost to anyone. A blog post with experimental results is coming in the next couple of days.
53
The deduplication Space by @minishlab just got a fresh update, allowing you to remove near duplicates in (training) datasets. Details in 🧵
2
8
82
2,632
Do not sleep on deduplication! Use this FREE app for semantic deduplication of multiple massive datasets. This is how it works: - You pick one all more datasets from the Hub - It make a semantic embedding of each row - It remove removes near duplicates based on a threshold like 0.9 - You can push the deduplicated dataset back to a new repo, and get to work.
3
4
2
19
3,109
We just released tokenlearn 0.2.0! Tokenlearn is our post-distill distillation framework, and is what we use to train our potion models. 🧵
1
1
2
69
All you need is ~100k-1M documents, a sentence transformer, and some time, and your static model will be a lot better than before. See the snippet below for the recipe that will reproduce potion-base-8m (you should use more than 500 docs though). 🧵
1
1
36
Release notes: github.com/MinishLab/tokenle…

Release v0.2.0 · MinishLab/tokenlearn

What's Changed fix: typing in featurize by @stephantul in #20 feat: new distillation by @stephantul in #23 fix: add license by @stephantul in #24 Full Changelog: v0.1.2...v0.2.0

github.com
30
Load more