Rough edges are still there - I'd rather ship it and hear where it breaks than sit on it. If you're on a Mac and this sounds useful to you, I'll drop the link below.
What's the thing you'd actually want autocomplete to get right that nothing currently does?
What it's not: it's not trying to write the email for you. No "generate a paragraph" button. It finishes the sentence you started, at the speed you're already typing, and gets out of the way.
Another one: it would swap in a wrong word mid-suggestion and never self-correct. You'd accept the completion and only catch the wrong word after the fact. Small-model failure modes are boring and specific in a way that never shows up in a demo.
The failure modes were not what I expected going in. The model would silently drop the subject of a sentence past a certain context length. Grammatically fine, semantically empty. Took a while to even notice, because it doesn't look broken - it just doesn't mean anything.
The model is small on purpose - under 1B params - because it has to run inference in tens of milliseconds, locally, without melting your battery. That constraint shaped almost every decision.
Every autocomplete tool I tried worked the same way - send what you're typing to a server, get a suggestion back. Fine for a lot of use cases. Not fine for the client emails, passwords, drafts, and half-formed thoughts most of us type all day. So I built the alternative.
Best Account-Based Marketing Software in 2020 zcu.io/juXI
Shout out to @vasanthdevelop for all the incredible work he has been doing for the tech community.
