@MercityAI

Building research-led capabilities for product and enterprise teams. Synthetic data · Custom training · Evals Solving problems generic APIs cannot solve.

mercity.ai/
Joined October 2026
  • Tweets19
  • Following8
  • Followers163
12Media
Today we introduce Mercity. A research-first AI firm building at the intersection of synthetic data, custom training, evals and safety We believe that the future of safe enterprise AI lies in enabling research led capabilities Here's our work so far : mercity.ai/

Mercity Research

Research-led capabilities for product and enterprise teams. Synthetic data, custom training, and evaluation that tells the truth.

mercity.ai
7
14
5
34
3,006
Our Latest Research: We pretrained four 1B-parameter models on 6B FineWeb tokens to test two ideas from frontier MoE labs, Kimi Delta Attention and LongCat n-gram embeddings, at small scale. We then compared grafting, pruning, 4-bit quantization, and downstream NER fine-tuning
2
1
3
13
1,399
more replies
We then operated on trained checkpoints Removing layer 0 cut the score by 1/3rd while most layers past layer 8 could go almost for free 4-bit quant cost under a point Attention could not be moved between models without collapsing Full research log: mercity.ai/research/pretrain…

Pretraining Dense Models: A Research Log | Mercity Research

We pretrained four dense 1B-parameter checkpoints on 6B FineWeb tokens to test Kimi Delta Attention and LongCat n-gram embeddings, then compared grafting, pruning, 4-bit quantization, and downstream...

mercity.ai
1
4
51
Research done by @_notapenguin Tweet authored by @freshlimesofa
3
45
We REMOVED upto 10 WHOLE layers from Qwen3-8B-Base using LaCo (Layer Collapse). Upon deleting only 6 layers, the model got 16.7% smaller, keeping 78-94% of its reasoning scores at a 1.68x perplexity ratio. Code and pruned weights are up on our Hugging Face. Here are insights from our research :
4
18
9
202
34,750
more replies
The paper's Llama settings didn't transfer to Qwen3 A 0.65 similarity threshold sent MMLU to random, 0.85 worked Merging near the output broke the model, so we stopped at layer 28 Merging LayerNorms gave gibberish, so we left them alone There is probably an element of how many tokens the model is trained on! We will explore this in the coming times!
1
7
1,225
Find the full research log here: mercity.ai/research/laco-lay…

LaCo Layer Pruning for Qwen3-8B: Our Research Log | Mercity Research

How we implemented LaCo layer pruning on Qwen3-8B-Base, compressing 36 layers to 30 with 16.7% size reduction while retaining 78–94% of reasoning capabilities — covering the paper-vs-implementation...

mercity.ai
1
2
16
981
We build custom data, models and evaluation for problems an API cannot solve. If that is the problem in front of you, our DMs are open, or start a conversation at mercity.ai

Mercity Research

Research-led capabilities for product and enterprise teams. Synthetic data, custom training, and evaluation that tells the truth.

mercity.ai
1
5
65
Find out more about our Models, Research, Datasets and work here : mercity.ai huggingface.co/Mercity github.com/Mercity-AI
5
58
Load more