
Deep into Inference Engineering SDE 3 @ UBER as EXT ex DEFI Solutions Developer building @meetdooora
Bengaluru
Joined April 2018
- Tweets1.7K
- Following1.9K
- Followers379
- Likes659
1/12
MHA vs MQA vs GQA
Watched a few videos on attention today.. then worked through the cache math.
Multi-head, multi-query and grouped-query attention change how much memory a model needs while it generates text.
#buildinpublic
1
1
38
1/4
My first pull request got merged into @vllm_project today. it is a small fix in the documentation, But somewhere we have to start.
github.com/vllm-project/vllm…
#buildinpublic
1
70


