# Your AI assistant refuses. That does not mean it is protecting you > The same request accepted once, refused the next time, depending on the conversation. Why a model's refusal is not a control and what governs an agent. - Source: https://snok.ai/en/news/blog/ai-assistant-refusal-is-not-a-security-control/ - Author: Michał Korzeń - Published: 2026-09-09 --- Your security analyst asks the company AI assistant for help with a vulnerability assessment. The assistant declines - politely, with a note that the task looks offensive in nature. An hour earlier the same assistant carried out an almost identical instruction for somebody else. Nothing was reconfigured. No settings changed, no model update shipped, nobody touched permissions. The only thing that changed was what the model could see in the conversation before the request arrived. ## Why this is your problem rather than a curiosity Documents describing AI deployments often list the model's refusal as a security measure. It sounds reasonable: if the assistant declines dangerous instructions, then anyone trying to misuse it walks away empty-handed. Paragraphs in security policies, slides for the board and answers in customer questionnaires get written on that basis. The trouble is that this refusal is not a fixed property of the model. It is a reaction to how the conversation unfolded - and the shape of a conversation is something more people can influence than we usually assume. A research team tested this methodically. They took 150 real security questions and put each of them to eight widely used models, every time inside a different conversational setting. The question itself was identical, character for character. Only what the model had seen beforehand changed. The results diverge far enough that calling this a control becomes hard to defend. ![Diagram: three scenarios in which the same request gets a different answer from an AI assistant - a planted history of consent, an instruction split into fragments, and an analyst working methodically](https://snok.ai/images/blog/granica-trzy-scenariusze-en.svg) ## Scenario one: a planted history of consent It is enough for the earlier part of the transcript to contain a line in which the assistant already accepted such a task once. The request stays the same, but the model now sees its own supposed prior consent and follows it the way it would follow a precedent. Compliance rises from 62 to 85 percent. That precedent never happened. It was written in. Map this onto your own environment. If your assistant reads session memory, a shared team thread or a log of its own earlier actions, then anyone who can write into those places has a say in where its refusal boundary sits. No administrative rights are required, and no access to model configuration - write access to something the assistant will later read is enough. In most deployments, nobody keeps a list of those places. ## Scenario two: an instruction split into pieces A month earlier, researchers described an attack in which the dangerous instruction never appears anywhere in full. It is scattered across the channels an assistant trusts without a second thought: the description of a tool it uses, and the result returned by another tool. Each fragment on its own is harmless and will pass any review. The meaning only emerges when they are combined, already inside the assistant's context - in other words, in the one place nobody inspects. Splitting the instruction across two channels raised the success rate from 42 to 82 percent, and three of the eleven models tested went from zero to one hundred. This is the same problem as the first scenario seen from the other side. There, the model saw something that had never happened. Here, it cannot see the whole of what it is doing. The practical conclusion is short and worth writing into your engineering standard: a value returned by one tool must not reach the arguments of another without validation. One sentence closes the entire route. ## Scenario three: doing the job properly This result is the most uncomfortable of the three, and the least intuitive. When the same request arrives after several messages in which somebody explains the problem, assesses feasibility and builds a limited proof of concept - that is, does exactly what a good analyst does - the model refuses far more often. Compliance drops from 67 to 14 percent. The direction held across three independent comparison sets, so this is not a single noisy measurement. Breaking the task into steps does not work as a bypass here, because the dialogue is internally consistent. The model sees understanding accumulate, recognises the operational nature of the work, and closes precisely because of it. Now read that again in terms of cost. A person working methodically looks more suspicious to the model than a person who asks outright. The gate meant to stop an attacker is most effective at stopping your own employee. And here the second half of the problem begins, the half that rarely gets discussed. An analyst refused halfway through an assessment does not shelve the task. They find another route: a personal account, a model without a gate, a tool outside the register. The gate did not stop the work - it moved it somewhere nobody monitors. That is a worse position than having no gate at all, because it adds the illusion of control. ## What this looks like put together The model opens up when it cannot see the whole picture, or when it sees a forged trace of its own earlier consent. It closes when the conversation looks like professional work. A safeguard that moves by twenty-three points in one direction and fifty-three in the other on the same request, depending on what preceded it, is not fit for the role our documents assign to it. This does not make model guardrails useless. It makes them the last layer rather than the first - and it means they must not be presented as evidence in an audit. ![Diagram: layers of control over an AI agent - identity and permissions, data scope, tool separation, rollback path, and the model gate as the thinnest layer on top](https://snok.ai/images/blog/granica-warstwy-kontroli-en.svg) ## Four things worth checking in your own setup **Who can write to your agent's memory.** This is a question about a list: which systems, which integrations and which people may write into content the agent will read on its next turn. For agents reaching into SAP data this is not a thought experiment but a concrete permission inventory, which either exists or does not. **Whether a model's refusal appears in your documentation as a safeguard.** If it does, you have a control in your paperwork whose effectiveness varies with the flow of a conversation. Better to find it before an auditor does. The real controls are the ones that hold regardless of how the conversation looks: the agent's identity, the scope of its permissions, the data it can reach at all, separation between its tools, and a rollback path. **Whether one tool's output reaches another tool's instruction without validation.** That single sentence in an engineering standard closes the route described in the second scenario. It costs one sentence and one architecture review. **How your team behaves when the assistant refuses mid-assessment.** If the answer is a workaround, the gate is not protecting anything - it is moving work out of sight. Worth asking directly, because nobody reports it on their own. ## How we sequence this in projects In agentic projects on SAP Joule and UiPath Maestro we work in one order from day one. Identity first: who the agent is in the system, what permissions it holds, who granted them and who can revoke them. Then the scope of data it can reach at all. Then separation between tools. Then the rollback path, together with an answer to the question of who by name switches the agent off, and when somebody last rehearsed that path rather than describing it. The model gate comes last in that sequence, and that is how we describe it in project documentation - as a supplement, never as the foundation. The reverse order produces an architecture in which the only real control depends on what somebody wrote into the context half an hour earlier. ## What this does not say The paper covering the first and third scenarios is a preprint, ahead of peer review, and the numbers come from its authors. The panel covered models of the 4.6 and 5.6 generations, so it says nothing about the newest ones. The results on the split-instruction attack come from tests in a controlled environment with planted credentials - not from an incident reported by anyone. It is also worth knowing what the word "compliance" measures here. It means delivering the requested capability directly and to a material extent. A scope-limited answer counts separately, as a third category alongside refusal. So these numbers do not say "the model stays silent"; they say "the model does not do what it was asked to do". We treat this as the result of two studies rather than as settled knowledge. And we would ask you to treat every number we quote the same way - including ours. ## Sources [Same Request, Different Boundary: Evaluating Cybersecurity Assistance across Conversational Contexts](https://arxiv.org/abs/2609.00578), arXiv:2609.00578v1, submitted 1 September 2026 - preprint, not peer reviewed. [ASSET Research Group, GhostSplice disclosure](https://asset-group.github.io/disclosures/ghostsplice/) - an instruction split across the channels of an MCP server.