FW retweeted
Microsoft just released VibeVoice-ASR on Hugging Face
A unified speech-to-text model that transcribes hour-long audio in one pass
With built-in speaker diarization, timestamps, and customizable user context
FW retweeted
🚨 Microsoft Research just launched something that might define the next era of AI systems.
They call it 'Agentic Organization' and it’s not just a new model. It’s a new way for intelligence itself to organize.
Here’s what’s wild:
Most large language models still “think” like a single brain.
Step-by-step. Linear. Slow. Even “parallel thinking” just runs the same process twice and merges answers later.
Agentic Organization changes the entire game.
They built a new reasoning protocol called AsyncThink, where a model plays both roles an Organizer that breaks a complex problem into sub-queries, and Workers that solve those sub-parts at the same time.
Think of it like this:
Instead of one mind grinding through steps, AsyncThink forms a mini civilization of minds delegating, merging, adapting in real time.
And it learns this behavior through reinforcement learning literally learning how to organize its own thoughts.
The results are insane:
→ 28% lower inference latency than parallel thinking
→ Higher accuracy on math reasoning tasks
→ Zero-shot generalization to unseen problems like Sudoku
→ Learned organizational policies that evolve dynamically during reasoning
It’s like scaling from “an intelligent agent” → to “an intelligent organization.”
AsyncThink models don’t just reason faster they reason like teams do.
Fork. Think. Join. Verify. Iterate.
This is a glimpse of post-LLM intelligence systems that don’t just think, they coordinate thought.
And if that holds, the future of AI might look less like a single brain… and more like a company of minds.
Paper: The Era of Agentic Organization: Learning to Organize with Language Models
FW retweeted
On-policy + Reverse KLD = MiniLLM (arxiv.org/abs/2306.08543). Really nice blog by @thinkymachines. Exciting to see it being offered as a service!
Our latest post explores on-policy distillation, a training approach that unites the error-correcting relevance of RL with the reward density of SFT. When training it for math reasoning and as an internal chat assistant, we find that on-policy distillation can outperform other approaches for a fraction of the cost.
thinkingmachines.ai/blog/on-…
FW retweeted
Long paper thread!
I've held a skeptical opinion of DiffTransformer after skimming it, but after some recent interaction thought that's unfair and I'd give it a proper read. Did so on a recent flight.
Paper TL;DR: pair two attention heads, and do:
(sm(Q1K1) - λ sm(Q2K2)) V
FW retweeted
Top 3 Hacker News from 2024-10-08:
1. Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]; tinyurl.com/yogv9fpt
2. Differential Transformer; tinyurl.com/ykpfvf38
3. Show HN: Kotlin Money; tinyurl.com/yu99z775
FW retweeted
About 250 years ago, Wolfgang von Kempelen made an early—maybe the first— “speaking machine.” We’ve come a long way since.
After releasing my book Impromptu, I wondered if AI could serve as the “speaking machine” for the book and help create a compelling audio version.
🧵👇
FW retweeted
Are Multimodal LLMs the future for Computer Vision?
Kosmos-2 is a new model from Microsoft Research
It has quite a broad range of tricks up its sleeve (including grounding)
An overview of the work 👇
youtu.be/K3K5BoN2b5k
FW retweeted

Microsoft Research@MSFTResearch18 May 2023
United States
United StatesConnected via WebAccount-level information, not a live location or per-post device.
Learn how Microsoft researchers leveraged cutting-edge innovations in Microsoft Editor to optimize the server neural grammar checker model and achieve the ultimate zero-cost-of-goods-sold (COGS) goal with the client neural grammar checker model. msft.it/6012gZBhG
FW retweeted
GPT-3/LLMs' Achilles heel is short context length - how many "in-context" examples they can consume to learn a new task.
Enter "Structured Prompting": scale your examples from dozens => 1,000+
Here's how👇
arxiv.org/pdf/2212.06713.pdf
FW retweeted
Language Models are General-Purpose Interfaces
abs: arxiv.org/abs/2206.06336
Experimental results show that METALM exhibits strong finetuning and in-context learning performance across a wide range of language-only and vision-language tasks
FW retweeted
🤩 LayoutLMv3 by @Microsoft is now available @huggingface!
🔥 The model replaces the CNN backbone of its predecessor with (much simpler) patch embeddings à la ViT.
🚀 SOTA performance on all document AI benchmarks, both image-only and text+image! (1/2)
FW retweeted
On the Representation Collapse of Sparse Mixture of Experts
abs: arxiv.org/abs/2204.09179
method alleviates the representation collapse issue and achieves more consistent routing than the baseline mixture-of-experts methods
FW retweeted
.@Gradio Demo for BEiT: BERT Pre-Training of Image Transformers on @huggingface Spaces
demo: huggingface.co/spaces/rowel/…
github: github.com/microsoft/unilm/t…
FW retweeted
😎 Models (released yesterday by @MSFTResearch) are now on the hub: huggingface.co/models?other=…
DiT: Self-supervised Pre-training for Document Image Transformer
abs: arxiv.org/abs/2203.02378
achieves sota results on downstream tasks, e.g. document image classification (91.11 → 92.69), document layout analysis (91.0 → 94.9) and table detection (94.23 → 96.55)
FW retweeted
Will this be NLP's ResNet moment? The messaging definitely feels similar, I guess time will tell if it withstands scrutiny.
I would really have liked to see a vanilla ViT experiment in there too. Part of the reason behind ViT was to bring the communities closer together.
DeepNet: Scaling Transformers to 1,000 Layers
abs: arxiv.org/abs/2203.00555
successfully scale Transformers up to 1,000 layers (i.e., 2,500 attention and feed-forward network sublayers) without difficulty, which is one order of magnitude deeper than previous deep Transformers
FW retweeted
Corrupted Image Modeling for Self-Supervised Visual Pre-Training
abs: arxiv.org/abs/2202.03382
300-epoch CIM pretrained vanilla ViT-Base/16 and ResNet-50 obtain 83.3 and 80.6 Top-1 fine-tuning accuracy on ImageNet-1K image classification respectively
FW retweeted
I tested the text similarity models on 14 datasets from different domains (emails, papers, online communities) on various tasks (clustering, retrieval, paraphrase mining).
The 175B model is actually worse than a tiny MiniLM 22M parameter model that can run in your browser.
FW retweeted
🎙️Microsoft open-sources new models at lightning-speech at the moment 🚀
We are excited to support @MicrosoftResearch's newest speech models as of🤗Transformers 4.15:
🎤WavLM
🗨️UniSpeech-SAT
💬UniSpeech
Try to get past WavLM at Speaker Verification😉
huggingface.co/spaces/micros…
FW retweeted
To the surprise of many, the leading NLP company is @OpenAtMicrosoft. Their work with distilled NLP models like MiniLM, their work on the open neural network exchange format @onnxai and most importantly @onnxruntime is what makes NLP models universally usable.
FW retweeted
VLMO: Unified Vision-Language Pre-Training with
Mixture-of-Modality-Experts
abs: arxiv.org/abs/2111.02358
introduce Mixture-of-Modality-Experts Transformer,
where each block contains a pool of modality-specific experts and a shared self attention layer































