9 techniques to generate better LLM outputs: An instruction alone, with no examples, schema, or stated constraints, is the default way to generate an LLM response. This is zero-shot prompting, and it serves as the baseline. But when outputs aren't good enough, the first thing you should try is to improve the prompt, not a model upgrade. There are 8 prompting techniques that you can use: - Few-shot prompting: Show the model a few input-output examples. It picks up the pattern and applies it to new inputs. - Chain-of-thought (CoT): Ask for step-by-step reasoning. Breaks complex problems into smaller, verifiable steps. - Prompt hierarchy: Use system, developer, and user prompts at different authority levels. Higher levels override lower ones. - Role-specific prompting: Assign a persona like "You are a security researcher." The model conditions on a different slice of its training data. - Negative prompting: Specify what the LLM should NOT do. "Do not use marketing language" works as a hard constraint. - JSON prompting: Define a JSON schema in the prompt. The LLM returns structured output matching that schema. - Attentive reasoning queries (ARQ): Replace free-form CoT with structured, domain-specific questions. In testing, it hit 90.2% instruction adherence vs. 81.5% for direct prompting. - Verbalized sampling: Explicitly prompt the LLM to generate multiple responses with probability scores. Recovers the output diversity that RLHF alignment suppresses. They stack well together. Few-shot + CoT. JSON + Negative prompting. ARQ is essentially a structured version of CoT for agentic conversations. Of course, better outputs will also come from better context, tool access, and retrieval. But these 9 techniques live entirely in the prompt and involve working with the same model, infrastructure, and setup. You are just structuring the prompt differently to get better outputs from an LLM. Once the prompt is doing its job, the next lever is how much compute the model spends before it answers. I covered that separately, walking through chain of thought, best-of-N, self-refinement, and tree search, and why each of them only keeps paying off when you have a reliable way to verify the output. Read it below. PS: This visual was inspired by ByteByteGo's post on a similar topic. I added some latest research-driven techniques like ARQ, Verbalized sampling, and JSON prompting.

I have been fine-tuning LLMs for over 2 years now! Here are the top 12 techniques I'd learn if I were to fine-tune them: Bookmark this. 1. LoRA > Freezes the base weights and trains two low-rank matrices as the update, resulting in ~95-99% fewer params to fine-tune. 2. QLoRA > LoRA on top of a 4-bit quantized base model. 3. Prefix tuning > Prepends trainable vectors to keys and values at every layer, weights frozen. 4. Adapter tuning > Inserts small trainable modules between transformer layers. 5. Instruction tuning > Supervised tuning on (instruction, response) pairs so the model follows directions instead of just continuing text. 6. P-tuning > Optimizes continuous prompt embeddings through a small encoder, mainly for NLU tasks where discrete prompts are unstable. 7. BitFit > Trains only the bias terms, ~0.08% of params, and still rivals full fine-tuning on small-to-medium datasets. 8. RLHF/RLAIF > RLHF trains a reward model on human preference rankings, then PPO against it. The pipeline behind the first ChatGPT. > RLAIF swaps the human labeler for an LLM judging. RLHF-level quality at a fraction of the cost. 9. DPO (Direct Preference Optimization) > Skips the reward model and optimizes preference pairs directly with a classification-style loss. Simpler than PPO. 10. GRPO (Group Relative Policy Optimization) > Samples a group of responses per prompt and normalizes their rewards within the group. DeepSeek R1 ran on it. 11. RLVR (Reinforcement Learning with Verifiable Rewards) > Replaces the learned reward model with a checker or compiler returning verifiable scores. The free signal behind R1's math and code. 12. Federated fine-tuning > Tunes across decentralized clients that share only weight updates, never raw data. For when data can't leave the device. GRPO needs exactly one scalar reward per response. RLVR (13) produces that for free on math and code by running the answer through a checker or compiler. But tasks like a RAG answer, a support reply, or a summary have no gold label to match against. The usual fallback is a hand-written reward function scoring faithfulness, hallucination, and completeness. It takes days to calibrate, rewards the wrong behavior when the weights are off, and breaks every time you add a tool or edit the system prompt. RULER, implemented in OpenPipe's ART (open-source), solves this. During training, it passes the N sampled trajectories to a judge LLM, which ranks them relative to each other against the agent's system prompt and returns the scores. Relative ranking is more stable than absolute scoring, and GRPO normalizes within the group anyway, so the rankings feed straight into the pipeline like with RLVR. Here's the GitHub Repo: github.com/OpenPipe/ART (don't forget to star it ⭐ ) I wrote a full breakdown recently on how exactly this works, with the training loop and code. Read it below.