An AI agent rarely receives a single, clean input. Before the model sees a prompt, an orchestrator may combine a user message, retrieved documents, CRM records, tool results, conversation memory, policy snippets, and metadata from several tenants. Each source can be legitimate on its own and still be wrong to place in this particular context.
That boundary is easy to miss because it is usually implemented as a few string concatenations inside a retrieval or orchestration function. When the system works, the prompt looks helpful. When it fails, the incident is described as “the model saw sensitive data,” “the RAG result crossed tenants,” or “the agent relied on stale context.” In each case, the missing abstraction is the same: context needs a firewall before it becomes model input.
The context firewall is not a prompt-injection filter, a DLP scanner bolted onto the end of a request, or another observability dashboard. It is a policy-enforced data plane that decides which fields may enter a model context, why they may enter, what transformation they require, which tenant and purpose they belong to, how long they remain valid, and how the decision can be reconstructed later.
A context firewall is the last controlled boundary before untrusted and sensitive data becomes model-visible.
This matters now because tracing is becoming common while production quality remains difficult. LangChain’s 2026 State of AI Agents reports that 89% of surveyed organizations have implemented some form of agent observability, but quality remains the largest production barrier. Seeing a bad context after the fact is useful. Preventing an unjustified field from entering it is better.
Context is a data plane, not a string
A useful mental model is to treat every context item as a typed data object with a decision attached. The firewall should never receive only text. It should receive a candidate item with an origin, owner, sensitivity, purpose, tenant, freshness, and transformation history.
| Question | Weak implementation | Context-firewall implementation |
|---|---|---|
| What is this? | A chunk of text | A field-level item with a stable source reference |
| Why is it here? | The retriever returned it | A declared purpose and policy decision |
| Who may see it? | Whoever invoked the agent | A tenant, subject, role, and scope check |
| Can it be changed? | Usually copied unchanged | Redacted, masked, tokenized, summarized, or rejected |
| Is it still valid? | Retrieval timestamp is implicit | Explicit freshness budget and expiry |
| Can we explain inclusion? | Search score and trace | Decision, rule version, lineage, and transformation record |
The distinction prevents a common category error. Retrieval relevance answers, “Does this look related?” It does not answer, “May this field be disclosed to this agent for this purpose?” A high-similarity document can still be outside the caller’s tenant, beyond the purpose of the workflow, or too sensitive to expose in raw form.
The five-stage firewall pipeline
A production pipeline can be implemented as five stages. The names are less important than the invariants: every accepted item must carry its decision context, and every rejection must be observable without leaking the rejected payload.
1. Classify before retrieving broadly
Classification should happen at ingestion and be refined at request time. A customer record might contain public account metadata, internal notes, payment identifiers, health-related information, or free-form text whose sensitivity is unknown. Treating the whole record as one sensitivity class makes the safe path either too permissive or unusably restrictive.
The minimum useful taxonomy is not a universal list of labels. It is a set of decisions the firewall can enforce: public, internal, confidential, restricted, and unknown. Unknown should not silently become public. It should follow a conservative path until a classifier, owner, or human process supplies stronger evidence.
Classification metadata should be versioned. If a record was admitted under classifier version cls_17 and the classification policy later changes, the system should be able to distinguish old decisions from new ones rather than rewriting history.
2. Minimize at field level
The firewall should ask for the smallest representation that satisfies the task. A support agent answering “Has this customer already reported the same outage?” may need an incident identifier, affected product, and timestamp. It probably does not need the customer’s full address, payment token, or internal account note.
Field-level minimization is more durable than prompt-level instructions such as “do not reveal private information.” A model cannot reliably unsee data it was given. The decision must happen before tokenization and before prompt assembly.
A practical allow decision can be expressed as:
allow(item) =
tenant_ok
AND purpose_ok
AND subject_scope_ok
AND freshness_ok
AND sensitivity <= purpose_ceiling
AND transformation_availableThe policy should be explicit about what happens when one input is unknown. For high-impact actions, unknown purpose, unknown tenant, or unknown classification should normally become a block or human-review state, not an implicit allow.
3. Transform sensitive values
Redaction, masking, tokenization, and controlled summarization serve different purposes.
| Transformation | What the model receives | When it is useful | Main failure mode |
|---|---|---|---|
| Redaction | Nothing or a placeholder | The value is not needed | Removing a value that was required to disambiguate a case |
| Masking | A partial value such as •••• 4821 |
Human-friendly comparison | Partial values can still identify a person in a small dataset |
| Tokenization | A stable surrogate such as cust_tok_91 |
Cross-step reference without raw disclosure | The detokenization service becomes a high-value target |
| Bucketing | A range or category | Numeric reasoning without exact values | Boundary effects and loss of precision |
| Controlled summary | A derived fact with provenance | The workflow needs meaning, not payload | Summary can introduce unsupported claims |
Tokenization is not anonymization. A stable token can be joined across requests, and the lookup table can restore the original value. The firewall must therefore carry token scope, purpose, expiry, and detokenization authority. A token created for fraud investigation should not automatically work in a customer-support workflow.
4. Preserve lineage and a context manifest
A model-facing prompt does not need to contain every audit detail, but the system needs a compact manifest that records what was admitted and why. The manifest is the bridge between a safe prompt and an explainable system.
{
"context_id": "ctx_01J9FIREWALL",
"workflow_id": "wf_support_triage",
"tenant": "tenant_acme",
"purpose": "duplicate_incident_detection",
"policy_version": "ctx-policy-2026.09.1",
"items": [
{
"source_ref": "incident://48291",
"field": "product_and_timestamp",
"decision": "allow",
"transform": "direct",
"classification": "internal",
"fresh_until": "2026-09-05T10:20:00Z"
},
{
"source_ref": "customer://8841",
"field": "email",
"decision": "allow",
"transform": "tokenize",
"token_scope": "support_case_48291",
"classification": "confidential",
"fresh_until": "2026-09-05T10:20:00Z"
},
{
"source_ref": "account://8841",
"field": "payment_instrument",
"decision": "deny",
"reason_code": "purpose_not_authorized"
}
]
}The manifest should be append-only or content-addressed when it is used for audit. It should not copy rejected payloads into a new log. A safe denial record can include a stable source reference, rule code, policy version, and hashed field identifier without retaining the sensitive value.
Purpose is a security boundary
Permission and purpose are related but not identical. A support employee may be allowed to view a customer’s account while still not being allowed to use payment details for a marketing recommendation. A tool may be authorized to read a ticket but not to send its private attachments to a third-party model.
Purpose should therefore be a first-class input to the firewall, not a comment in the calling code. The request should carry a purpose such as resolve_support_case, draft_internal_summary, or verify_refund_status. Policies can then express which fields are acceptable for each purpose and which model providers are approved for that data class.
This also gives teams a practical way to handle model routing. A public, low-risk context can use a broader provider pool. A restricted context may require an approved region, a private endpoint, or a local model. The firewall should produce a route constraint rather than leaving the router to infer privacy from the text itself.
Freshness belongs beside sensitivity
A field can be safe to disclose and still be unsafe to use. Inventory, entitlement, credit status, incident state, and approval status all change. A context firewall should attach a freshness budget to each item and enforce it at admission time and, for high-impact actions, again immediately before execution.
A stale item should not necessarily disappear without explanation. The firewall can return a structured state such as expired, refresh_required, or uncertain. This allows the orchestrator to refresh only the affected source instead of blindly rebuilding the entire prompt. It also avoids turning stale context into a silent correctness bug.
Fail closed, but fail usefully
“Fail closed” does not mean returning an empty prompt and leaving the user confused. It means refusing an unsafe inclusion while returning enough structured information for the workflow to recover.
A useful response envelope might look like this:
{
"status": "needs_review",
"allowed_items": 7,
"blocked_items": 2,
"refresh_items": 1,
"next_step": "request_owner_approval",
"reason_codes": ["cross_tenant", "purpose_not_authorized", "expired"]
}The user-facing message can say, “I could not use one account note because this workflow does not have the required purpose scope. I can continue with seven approved items or request review.” It is more honest and more operationally useful than silently omitting the note or exposing it because the model asked for more context.
Test the firewall as a product boundary
A context firewall needs more than unit tests for a redaction function. The important tests exercise combinations of tenant, purpose, sensitivity, freshness, transformation, and downstream provider.
| Test family | Example invariant |
|---|---|
| Tenant isolation | An item from tenant B never appears in a tenant A manifest, even when it ranks first in retrieval |
| Purpose limitation | Payment fields are denied for a marketing-summary purpose |
| Transformation | Restricted identifiers are never present in raw prompt bytes after tokenization |
| Lineage | Every admitted item resolves to a stable source and policy decision |
| Freshness | An expired entitlement triggers refresh or review before an action |
| Fail closed | Unknown classification cannot silently enter a high-impact workflow |
| Provider routing | Restricted context cannot be sent to an unapproved provider |
| Redaction safety | Denial logs contain reason codes but not the rejected payload |
| Adversarial retrieval | Prompt-like instructions inside a document do not change the firewall decision |
The last test is important for boundary clarity. A prompt-injection defense may inspect content for hostile instructions. The context firewall decides whether the content is eligible to be present at all. They complement each other, but neither replaces the other.
Rollout without breaking every workflow
Do not begin by enforcing every rule in every path. Start in shadow mode: produce manifests and decisions, measure would-be blocks, and sample the cases with a reviewer. The goal is to learn where metadata is missing and which transformations preserve task quality.
Then enforce the highest-confidence controls first: tenant boundaries, restricted fields, provider restrictions, and hard expiry for action-critical state. Keep an escape hatch with explicit owner approval and a short expiry, not a global bypass flag. Every exception should be visible in the manifest and attributable to a person or service identity.
Useful operational metrics include the rate of blocked items by reason, the percentage of contexts with unknown classification, refresh success rate, token detokenization requests, false-positive review rate, and task quality after transformation. A rising allow rate is not automatically good. The system may simply be learning to classify everything as internal. Pair policy metrics with sampled content review and downstream task metrics.
What a context firewall does not prove
A firewall can prove that a field passed a particular admission policy at a particular time. It cannot prove that the field was true, that the source was uncompromised, that the model followed instructions, or that the final action was correct. It also cannot turn a weak purpose definition into a meaningful authorization boundary.
Those limits are features of an honest design. The firewall is one control plane layer. It should connect to identity, retrieval, provider routing, observability, evals, and action approval, while keeping its own contract narrow: control what becomes model-visible, preserve why, and refuse what cannot be justified.
A practical checklist
Before calling the model, ask whether every context item has a stable source reference, classification, tenant, purpose, policy version, transformation record, and freshness deadline. Confirm that unknown values follow an explicit fail-closed path. Confirm that the provider route is compatible with the most sensitive admitted item. Confirm that the manifest is available to reviewers without copying the raw payload into another leak surface.
After deployment, sample both allowed and denied decisions. Test cross-tenant retrieval, stale state, partial outages, token-scope confusion, and exception expiry. Measure whether the system is still useful after minimization; a firewall that blocks everything is not a reliable product, and one that allows everything is only a decorative gate.
The best context firewall is not the one with the most rules. It is the one that makes the model’s input deliberate, bounded, attributable, and recoverable.
References
[1]: https://www.langchain.com/state-of-agent-engineering — LangChain, “State of AI Agents,” 2026. [2]: https://genai.owasp.org/resource/state-of-agentic-ai-security-and-governance/ — OWASP Gen AI Security Project, “State of Agentic AI Security and Governance 2.01,” June 1, 2026.
Comments (0)