Start without examples and add few-shot demonstrations only when a test shows that they address a specific error. Zero-shot is an instruction with no input-output demonstrations; few-shot adds a small number of example pairs to make the expected pattern visible. The distinction is particularly useful when a task requires labels, style, or borderline decisions that are hard to convey in one sentence. Examples do not guarantee accuracy: they also consume tokens and may lead the model to copy irrelevant features.
Zero-shot and few-shot: what changes in the prompt?
In zero-shot prompting, you describe the task, rules, format, and input to process. In few-shot prompting, you keep those instructions and add demonstrations of how earlier inputs should be answered. The OpenAI prompting guide describes few-shot as providing a few input-output examples and recommends diversity among them. Anthropic's guidance likewise emphasizes relevance, variety, and clear separation between instructions and examples.
The test is not which name sounds more advanced, but whether the examples teach something the instruction does not yet communicate. If the model already delivers the expected format and decision, a longer prompt may not be worth it.
When examples add useful information
- A format that is hard to describe: showing a short and a long output can clarify structure or detail level, provided the rules are also explicit.
- Ambiguous cases: a borderline example shows how to apply a rule when signals conflict.
- A team-specific convention: examples can demonstrate internal labels, field order, or an editorial style that the model cannot infer.
- Repeated errors found in testing: a counterexample can correct an inappropriate generalization observed during evaluation.
For simple tasks, a precise definition and explicit output format are usually a leaner starting point. If a strict JSON contract is essential, also consider schema-constrained structured outputs where available: visual examples do not replace format validation or, on their own, guarantee compliance with a schema.
A test using the same four cases
Consider classifying comments about a purchase. The choices are positive, negative, mixed, and no evaluation. Define positive as praise only; negative as complaint only; mixed as praise and complaint; and no evaluation as a factual statement without judgment. In both prompts, request only one label, with no explanation.
Zero-shot version
Instruction: “Classify the comment as positive, negative, mixed, or no evaluation using the definitions above. Reply with the label only. Comment: [input]”. No demonstration is added.
Few-shot version
Keep the same instruction and add four demonstration pairs before the new input:
- “Great service.” → positive.
- “Support did not resolve my problem.” → negative.
- “I liked the quality, but shipping was slow.” → mixed.
- “Order recorded at 2 p.m.” → no evaluation.
Then apply both versions to the test set below. The reference column states the label expected under the rule; it is not a measured model result.
| Held-out test input | Reference label | What it checks |
|---|---|---|
| “It arrived early and works well.” | positive | Praise without complaint |
| “It arrived broken.” | negative | Complaint without praise |
| “It works well, but arrived late.” | mixed | Two signals in one input |
| “I bought it on Tuesday.” | no evaluation | Fact without judgment |
What a counterexample does
If the model starts labeling every sentence containing “but” as mixed, add the rule or a corrective case: “The service was poor, but the problem remained unresolved.” → negative. These are two negative facts, not praise plus complaint. The counterexample checks the semantic rule, not the presence of a word. Test it against fresh cases instead of treating it as proof of universal improvement.
How to compare quality and cost separately
- Keep demonstrations separate from an evaluation set. Do not count examples already inside the prompt as test successes.
- Hold the model, instructions, settings, and inputs constant; change only the examples.
- Compare a zero-shot version, a compact few-shot version, and, if there is a clear hypothesis, one with a counterexample.
- Check label correctness, format compliance, and performance on borderline cases. Repeat runs if responses vary.
- Record input tokens, output tokens, latency, and total cost per call. Examine averages and error types, not just a favorable result.
The OpenAI evaluation guidance recommends measuring behavior using criteria and test data. In this example, the labels are a proposed answer key; only an actual run can establish whether examples improve, worsen, or leave results unchanged.
When examples merely consume context
Each demonstration adds text to the prompt and can therefore increase the input-token total. It occupies part of the context window that could otherwise hold documents, history, or the question itself. Redundant, lengthy, or off-task examples have less value as demonstrations. They can also teach accidental patterns: for example, if every positive example is short, the model may confuse length with sentiment.
There is no universal ideal number of examples for every model and task. Add examples to test an observable hypothesis, remove those that do not change behavior, and check the effect on the held-out set. For repeated API calls, prompt caching can lower the price of reused portions under specific conditions; it does not make examples disappear from the context window or remove the need to measure actual usage and cost.
A practical decision rule
Use zero-shot as the baseline when the rules are clear and the result can already be validated. Try few-shot when there is a consistent error in format, label interpretation, or exception handling. Choose representative, varied examples, separate them from the new input, and keep the instruction explicit. Retain the shortest version that meets quality criteria on your test set. This is an engineering decision based on local evidence, not a promise that more examples always produce better answers.
