Contextual integrity

How contextual integrity tests a flow

Contextual integrity does not test if a fact is secret. It tests if this flow suits this context. In the words of Abdelnabi and Bagdasarian, appropriateness “is not determined by its content alone”.

Five parameters identify a flow. The example is ours, for an AI agent that sends email.

Parameter	Question	Example
Sender	Where does the data come from?	The email assistant
Recipient	Which party gets the data?	The HR team
Subject	Which person does the data identify?	The user
Information type	Which category of data is it?	A health record
Transmission principle	On which condition can it move?	The user gave consent

Change one value and the result can change. Send the record to an advertiser, or send it with no consent, and the flow breaks the norm.

Why contextual integrity matters for AI agents

An agent holds data from many sources and sends data to many parties. Contextual integrity gives a method to test each of these flows. Research benchmarks show that models frequently break the norms:

Test	Models	Result
ConfAIde, Tier 4 (2023)	GPT-4, ChatGPT	Revealed private data in 39% and 57% of tests, where humans did not
PrivacyLens (2024)	GPT-4, Llama-3-70B	Leaked in 25.68% and 38.69% of cases, with privacy prompts

PrivacyLens also found a “discrepancy” between answers to direct questions and actual behavior. A model can give correct answers about privacy and leak data when it does the task.

How attackers use context

Abdelnabi and Bagdasarian (preprint, May 2026) use the theory to study prompt injection. They give 3 methods to make an agent break a norm:

• Misrepresent the flow. Their example shows a non-refundable deposit as a “non-binding hold”.
• Manipulate the norms.
• Mix multiple flows.

The authors say that separation of data and instructions “fails to detect attacks that operate through contextual manipulation”. Prompt Guard and Prompt Guard 2 got an AUROC of 0.43 to 0.59 on 8,400 emails. The paper calls this near chance level.

One iterative attack got 96.7% success on an email assistant, in 150 sampled scenarios. The static baseline was less than 1%. The authors call these attacks a “qualitative investigation rather than definitive quantitative analysis”.

In practice

Two defences show the benefit and the limit of the theory.

Defence	Method	Result	Limit
AirGapAgent (2024)	A language model selects the data for the task. The conversational agent sees only that data.	Gemini Ultra, easier dataset, hijack: protection 97%, against 45% with no minimizer. Utility fell from 98.2% to 87.3%.	Single turn tests on synthetic profiles. No formal guarantee.
PrivacyChecker (2025)	A step extracts the flow parameters, then applies contextual integrity at inference time.	GPT-4o leakage fell from 33.06% to 8.32%. DeepSeek-R1 leakage fell from 36.08% to 7.30%. Static benchmark, 493 cases.	Leakage is usually higher in live benchmarks. Memory poisoning can disrupt the extraction.

AirGapAgent is similar to least privilege for data. The agent gets less data, thus a hijack can get less data.

Trade-offs

Tight norms stop correct flows. Loose norms let attacks through. Abdelnabi and Bagdasarian argue that “no fixed policy can prevent all context-based attacks without also blocking some legitimate flows”. They make this argument informally, and it is not a proof.

For unclear flows, they give 3 checks:

• Cryptographic attestation of the sender identity.
• A decision from the user.
• A check of claimed interests against external records.

Where definitions disagree

The term has three uses that are different.

• Two norms or five parameters. Accounts of the 2004 article give norms of appropriateness and norms of distribution. Recent papers use five parameters.
• Privacy or security. Nissenbaum wrote a theory of privacy. Abdelnabi and Bagdasarian use the same parameters for security breaches and privacy violations.
• Social norm or formal rule. Barth and colleagues (2006) write norms as a formal model of permitted and prohibited actions. AirGapAgent uses a language model for this judgment, with no formal guarantee.

Questions and answers

What are the five parameters of contextual integrity?

The five parameters of contextual integrity are the sender, the recipient, the subject, the information type and the transmission principle. The transmission principle gives the conditions for the flow, for example consent.

How is contextual integrity different from differential privacy?

Contextual integrity tests if a flow of information suits its social context. Differential privacy adds statistical noise to hide data about individuals. Benthall and Cummings propose a framework that uses context to set the epsilon parameter of differential privacy.

Does contextual integrity stop prompt injection?

No. Contextual integrity is a method to identify and test flows. Abdelnabi and Bagdasarian write that "the CI framing should not be mistaken for a solution". They also write that defences are not futile.