Announcing Surya OCR 2! It uses a new architecture and improves on v1 in every way:
- OCR with automatic language detection for 93 languages (no more specifying languages!)
- More accurate on old/noisy documents
- 20% faster
- Basic English handwriting support
very cool open-source project built with the ai sdk, nextjs, and shadcn
great use case and love that it supports multiple models and providers, configurable with just a dropdown!
๐ Today we're launching a free, open-source tool for AI dataset management:
๐ Quickly view, filter, and edit datasets with keyboard shortcuts
๐ง Generate or rewrite samples using AI, powered by @vercel AI SDK
๐พ Download datasets for fine-tuning anywhere
๐ Everything is on-device to preserve privacy
๐ Fully open-source project
โฌ๏ธ Check out the links in the thread below
Tokenizers for OpenAI models are everywhere, not the case for Mistral.
I always have to dig up a python notebook when I need to count tokens.
No longer the case...
Mistral tokenizer is now live at forefront.ai/tokenizer