Part 2 — Context engineering

Through 2023 and into 2024, the craft was prompt engineering — wording your instructions cleverly to coax better output from a model. As of 2026, that craft has been subsumed by a broader and more important one: context engineering.

Context engineering is the discipline of selecting, shaping, and delivering exactly the information and tools an agent needs at the moment it needs them. The prompt is now just one component of the context, and usually not the hard part.

The reason is mechanical. An agent loop accumulates context fast — every tool result, every sub-step, every retrieved document piles into the window. And longer context is not free or even neutral. It costs money and latency, and past a certain point it actively degrades reasoning.

Once you internalise that context is a scarce, curated resource — not a bucket you dump everything into — the core techniques become obvious. Treat the context window like a working desk, not a filing cabinet: only what you need for this decision, cleared off constantly.

The community now has a vocabulary for the failure modes. Learn these terms because you will see all of them in your own logs:

• Context poisoning— a hallucination or error gets into the context and is then referenced as if it were true, compounding.
• Context distraction— so much accumulated history that the model over-focuses on it and stops reasoning freshly about the actual task.
• Context confusion— irrelevant material in the window that the model tries to use anyway.
• Context clash— pieces of context that contradict each other, leaving the model to pick badly.
• Context rot— the general degradation of response quality as the window fills, even with relevant material.

A quick way to place a bug you’re seeing: if the model is confidently repeating something false, suspect poisoning; if it’s ignoring your latest instruction in favour of old history, suspect distraction; if quality just slid as the run got longer with nothing obviously wrong, that’s rot. The fixes in the next section map onto these directly.

Don’t keep everything in the model’s head. Give the agent a scratchpad — a “think” step where it can reason without that reasoning permanently bloating the conversation — and, increasingly, give it a filesystem. A widely-used pattern is the agent maintaining a MEMORY.mdor notes file it reads and updates, so durable state lives on disk and only the relevant slice gets pulled into context per step. “The filesystem is the context” is a real architectural stance in 2026, not a metaphor.

In Google’s ADK this “durable state lives outside the window” idea maps directly onto session state— a dict that persists across turns without being replayed verbatim into every prompt. A before_model_callback is the hook where you decide what actually reaches the model on this step:

from google.adk.agents.context import Context
from google.adk.models import LlmRequest
from google.genai import types
from typing import Optional

def before_model_callback(context: Context, request: LlmRequest) -> Optional[types.Content]:
    # Durable state persists in the session, NOT in the replayed transcript.
    step = context.state.get("model_calls", 0)
    context.state["model_calls"] = step + 1

    # Pull only the relevant slice into context for this step (offloading).
    notes = context.state.get("working_notes")
    if notes:
        request.config.system_instruction += f"\n\nWorking notes so far:\n{notes}"

    return None  # None = let the (now-shaped) model call proceed

As new information arrives, actively remove outdated or superseded material. The mature version of this is dynamic context pruning — evicting items by age and relevance, the way a cache does. Teams running this on tool-heavy agents report 50–70% token reductions with no quality loss, because most of what accumulates in a long agent run is genuinely dead weight by step ten.

The same before_model_callbackis where pruning lives — you inspect the assembled request and drop what this step doesn’t need before it’s ever sent. And at the runtime level, ADK’s RunConfig lets you cap how much history is even loaded, so old events never enter the window in the first place:

# Limit how much conversation history is fetched into context per run.
session_config = app.get_session_config(
    num_recent_events=20,          # keep only the last N events in the window
    # after_timestamp=...,         # or bound by time instead of count
)

Once a conversation or sub-task exceeds some threshold, compress the old turns into a summary and carry the summary forward instead of the raw transcript. This is the single highest-leverage move in multi-agent systems.

ADK ships this as context compaction: you point it at a cheap, fast model to do the summarising, then set how often it fires. Note the deliberate choice of a small model (gemini-flash-latest) for the summariser — you don’t burn your expensive reasoning model on compression:

from google.adk.apps.app import App, EventsCompactionConfig
from google.adk.apps.llm_event_summarizer import LlmEventSummarizer
from google.adk.models import Gemini

# Use a cheap, fast model to compress old turns — not your main reasoning model.
summarizer = LlmEventSummarizer(llm=Gemini(model="gemini-flash-latest"))

app = App(
    name="my-agent",
    root_agent=root_agent,
    events_compaction_config=EventsCompactionConfig(
        compaction_interval=3,  # summarise every 3 events
        overlap_size=1,         # carry 1 event of overlap so context isn't severed mid-thought
        summarizer=summarizer,
    ),
)

Don’t front-load every tool definition and every document. The 2026 pattern is discovery at runtime: the agent searches for the tool it needs when it needs it, rather than carrying definitions for fifty tools it will never use. Anthropic’s Tool Search Tool does exactly this and reports roughly an 85% reduction in tool-definition tokens. The same logic applies to knowledge — retrieve when relevant, don’t preload.

When you split work across multiple agents, give each one only the slice of context it needs and have it return a compressed summary, not its full transcript. Returning a full sub-agent transcript to the orchestrator is the classic way to blow up your token bill — practitioners cite something like a 15× cost difference between returning summaries and inlining transcripts.

Retrieval-Augmented Generation — fetching relevant documents and putting them in context — is still the backbone of most business agents, because verticals run on proprietary knowledge the model was never trained on. But naive RAG (embed the query, grab the top-k chunks, stuff them in the prompt) is now understood to be a weak baseline. The commonly cited figure is that naive pipelines fail at retrieval around 40% of the time.

The 2026 stack layers several improvements on top:

• Hybrid retrieval is table stakes. Combine keyword search (BM25), dense vector similarity, and structured metadata filters in a single query. Pure vector search misses exact-match terms like product codes, statute numbers, and drug names; hybrid catches them.
• Contextual retrievalprepends a short, LLM-generated description of where each chunk came from and what it is about before embedding it. Anthropic’s original work on this reported retrieval-failure reductions of up to 67% versus plain chunking. It is cheap and one of the highest-ROI improvements available.
• Agentic RAG turns retrieval into a loop instead of a single shot: the agent decomposes the query, retrieves, critiques whether the results are actually relevant, and retries with a reformulated query if not. One legal-tech deployment cut irrelevant retrievals from around 40% down to under 8% this way.
• GraphRAG and knowledge graphs matter wherever the reasoning is relational — legal cross-references, a clinical trail from trial to FDA approval, financial entity ownership chains. Use it when relationships are the problem, not by default.
• Metadata governance turns out to matter more than people expect. Well-governed metadata has been shown to substantially improve agent accuracy on structured-data tasks, because it lets retrieval filter precisely instead of guessing.

Retrieval is no longer a preprocessing step. It is part of the agent’s reasoning. That reframing is what separates a 2026 RAG system from a 2023 one.

(This series has a full pillar on retrieval — the Advanced RAG section — so the details of hybrid, contextual, and agentic retrieval live there. This article covers only what an agent builder needs to know: retrieval is reasoning, not preprocessing.)

Every major model provider now caches stable prefixes of your context so you do not pay full price to re-process them on every call. The economics are dramatic: providers charge cached reads at a large discount to the normal input rate — often around 10% — and some discount cached prefixes automatically. For a typical agent loop — where the same system prompt and tool definitions get sent on every single step — this routinely cuts the input-token bill by 30–50% with zero change to output quality.

There is an architectural catch you must design around: caching only works on a stable prefix. The cache breaks at the first byte that changes. So the ordering of your context is not cosmetic — it is a cost decision. Put stable content first and variable content last:

system prompt → tool definitions → static reference corpus → conversation history → current user message

Anything placed after a variable element cannot be cached. This is the hidden reason the before_model_callback shown earlier matters for cost, not just quality: if you inject anything variable (a timestamp, a per-step counter, a freshly retrieved chunk) into the front of system_instruction, you move the first-changed byte earlier and silently destroy your cache hit rate. Keep the callback’s mutations at the tail of the context, after everything stable:

def before_model_callback(context: Context, request: LlmRequest) -> Optional[types.Content]:
    # WRONG: prepending variable data invalidates the cached prefix every call.
    # request.config.system_instruction = f"Time: {now()}\n" + request.config.system_instruction

    # RIGHT: stable instruction stays first; variable data goes last, after the prefix.
    request.config.system_instruction += f"\n\n[dynamic] step context: {context.state.get('step_note','')}"
    return None

Teams that don’t know this rule often interleave dynamic content early in the prompt and then wonder why their cache hit rate — visible in the logs — is near zero. Check your cache hit rate. It is free money you are probably leaving on the table.

Prompt caching itself is a model-provider feature rather than an ADK feature — ADK controls what you send and in what order (which is what determines cache hits), while the discount is applied provider-side. Exact cache-read rates vary by provider and change over time, so treat the percentages above as indicative and confirm against current pricing.

Libraries and frameworks referenced on this page

• Google Agent Development Kit (ADK) — google-adk (Python) — used for all three context-management samples. Session state and before_model_callback (via google.adk.agents.context.Context and google.adk.models.LlmRequest) shape what reaches the model per step, covering offloading and pruning. RunConfig / get_session_config(num_recent_events=...) caps history loaded per run. Context compaction (App + EventsCompactionConfig + LlmEventSummarizer) handles summarisation, with gemini-flash-latest as the summarising model. The same before_model_callback is the correct place to protect a cacheable prefix by keeping variable injections at the tail of system_instruction.
• Retrieval tooling (BM25, dense-vector, and hybrid search) — referenced conceptually only; concrete choices are deferred to the Advanced RAG pillar and Part 7 (recommended stack).
• Anthropic Tool Search Tool — referenced as the just-in-time tool-loading pattern; covered in depth in Part 4 (tools and MCP).