Skip to main content

Few-Shot Prompting (In-Context Learning)

A prompt engineering methodology where the model adapts to a specific format, style, or logic during inference (In-Context Learning) by showing 2–5 reference examples.

1. Concept Overview & Systemic Problem

One of the most common challenges when working with language models is the ambiguity of natural language. When a developer writes in the instruction: “Write a concise TypeScript interface in the style of our project,” the model must rely on averaged representations from the internet:

  1. Subjectivity of Formulations: Terms like “concise,” “nice,” or “strict” are interpreted too broadly by the model, leading to instability in results from run to run.
  2. Inability to Describe All Rules Textually: Describing the conventions of a codebase can take 10 pages of rules that the model will inevitably begin to violate.
  3. “Better to See Once”: Just like a human programmer, a language model finds it significantly easier to replicate a style from 2–4 concrete examples than to parse abstract descriptions.

Few-Shot prompting eliminates ambiguity through demonstration: you show the model 2–4 contrasting pairs of “Input -> Reference Output,” establishing a clear canon of behavior.

2. Architectural Taxonomy & Mental Model

Gradation of learning methods in context (In-Context Learning):

  • 1. Zero-Shot: The model receives a purely textual instruction and a new task. Suitable for basic routine tasks but yields the highest variability in output.
  • 2. One-Shot: One ideal example is added to the prompt. This is sufficient to set a syntactic framework (e.g., the shape of the output JSON).
  • 3. Few-Shot (Classic Set of 2–5 Examples): The golden balance. Includes samples from various scenarios: a simple case, a complex case with nested data, and one edge case (Edge Case / Error Handling).
  • 4. Retrieval-Augmented (Dynamic) Few-Shot: An advanced production pattern: a library of 500 examples is indexed in a vector database. For each user query, vector search pulls the top 3 most relevant samples.

3. Technical Pipeline & Internal Mechanics

Lifecycle of preparing and executing a Few-Shot prompt:

  1. Exemplar Selection & Curation: A compact set of pairs is formed, demonstrating the desired result without errors and unnecessary verbosity.
  2. Structural Delimitation: Examples are wrapped in strict XML or Markdown tags to avoid mixing instructions with examples: <example id="1"><input>...</input><output>...</output></example>.
  3. Induction Head Pattern Activation: During the attention layer calculations, the transformer captures the transition from the <input> tag to the <output> tag as a deterministic data transformation.
  4. Target Inference Alignment: The model generates a response to a new query, subconsciously copying the syntax, length, naming conventions, and keys used in the samples.

4. Production Engineering Scenarios

01. Strict Parsing of Unstructured Data into JSON

You need to convert textual descriptions of bank transactions into valid JSON. Zero-shot often confuses date fields or copies cents. Few-shot with three samples (including refunds and multi-currency payments) ensures 99.8% schema stability.

02. Adhering to Company Code Style

The team requires the code agent to write unit tests strictly according to internal standards: using describe.concurrent, a specific user factory helper makeUser(), and prohibiting mocks via jest.spyOn. Two examples in the system prompt completely resolve the issue.

03. Canonization and Translation of Technical Terms

Translating documentation into Ukrainian while requiring the preservation of specific terms in their original form (e.g., Checkpointer, Race condition) and translating others according to the corporate glossary.

5. Pitfalls, Common Mistakes & Security

  • Majority & Recency Bias: Models are extremely sensitive to the order of examples. If 3 out of 4 examples resulted in the answer “Yes,” the model will have a strong bias towards “Yes” for any new input. Always balance classes in examples (50/50).
  • Error Mimicry: If one of the samples accidentally omitted a closing bracket or contained a spelling mistake, the model will reproduce this error in new results.
  • Inflating Input Context: Ten detailed Few-shot examples can consume 15,000 tokens. Always use Prompt Caching for system prompts with examples.
/ Frequently Asked QuestionsSchema.org FAQPage

FAQ: Few-Shot Prompting (In-Context Learning)

This phenomenon is known as In-Context Learning (ICL). Specialized attention patterns in the transformer—called Induction Heads—are trained to detect patterns and recurring motifs directly in the token sequence. The model recognizes the structure 'Input A -> Output A' and automatically extrapolates this rule to new queries.
/ Internal links

Related terms

All terms
Prompts & RAG

Prompt Engineering (Context Architecture & Prompt Engineering)

An engineering discipline focused on structuring system directives, XML markup, semantic delimiters, and examples to achieve deterministic, predictable outcomes from probabilistic models.

Read term
Prompts & RAG

System Prompt (System Instructions & Metaprompting)

The primary metacontext block of instructions passed at the zero position of the context window, defining the agent's role, safety rules, available tools, and behavioral boundaries.

Read term
Prompts & RAG

Chain of Thought (CoT)

A methodology that prompts a language model to generate sequential intermediate reasoning steps before producing a final answer, converting additional tokens (Test-Time Compute) into quality and accuracy of the output.

Read term
Prompts & RAG

Markdown AST for Agents (Abstract Syntax Tree)

A hierarchical tree-like representation of Markdown markup (mdast / Unified.js) that enables software systems and AI agents to deterministically analyze, transform, and safely edit technical content without fragile regular expressions.

Read term