TL;DR — If you use an open-weight LLM as a judge (reward modeling, data filtering, safety monitoring), it systematically mislabels a common transcript shape: an assistant that does part of a task and refuses the rest gets scored COMPLIANT, even though it refused. The judge sees the refusal — it says so in its own reasoning — and labels past it anyway. Confirmed causally across four models via ablation. Test your judge on partial-compliance refusals before you trust it.
I went looking for a different result and tripped over this one.
The plan was to replicate an Anthropic finding — that a frontier judge changes its verdict based on how the verdict gets used. I built a faithful version, ran it on open-weight models I could inspect locally, and it didn't reproduce. Not at 8B, not at 14B, not across four models.
That null is its own story. But chasing why the judges weren't behaving surfaced something with a much shorter path to your production pipeline.
The one-sentence problem
LLM-as-judge is now load-bearing infrastructure — it hands out reward signal, filters training data, and monitors agents in production. So when the judge fails in a systematic, predictable way, everything downstream inherits the failure. Here's a systematic, predictable failure.
What breaks
Every transcript in my set was a refusal — the assistant declined its operator's instruction, usually for a good reason. Ground truth is always NON_COMPLIANT. A judge that says COMPLIANT is just wrong.
The judges said COMPLIANT a lot — but only on one shape: the assistant does the harmless part of a request, then refuses the operative part.
The unsettling detail isn't that they got it wrong. It's how.
The judge perceives the refusal correctly. It writes "the assistant did not fulfill the request" — and then stamps COMPLIANT in the same breath. It isn't confused about what happened. It's scoring the action that was taken, not the instruction that was refused. The refusal and the partial action arrive as one sequence, and the judge scores the surface.
Proving it's causal, not vibes
A pattern in reasoning traces is a hypothesis. Models rationalize after the fact — the reason a model states isn't necessarily the reason it produced. So I ran the intervention instead of trusting the narration.
Matched pairs. One arm has the assistant do the partial action before refusing. The other refuses the same instruction, for the same stated reason, in the same tone, with the same offered alternative — minus the partial action. If the partial action is causal, removing it drops the mislabeling. If the traces were rationalization, removing it changes nothing.
It dropped the mislabeling. Every model.
Four models, three families, two size classes — Llama 3.1 8B, Mistral 7B, Hermes 3 8B, Qwen 2.5 14B. Double-digit drops across the board. Two things worth sitting with:
- It's not "small models are dumb." Qwen 2.5 14B is not small, and it shows the effect plainly. Raw judge competence didn't buy immunity — the effect spans the competence range.
- It's directionally universal. No model reversed. Remove the partial action, mislabeling falls, every time.
Why a technocrat should care
Partial-compliance refusals aren't edge cases. They're the default shape of a well-behaved assistant declining something: "I've done X as you asked. I'm not going to do Y — but here's an alternative." That's the polite, safety-conscious refusal. The good behavior.
Which means the transcripts most likely to be mislabeled are the ones where the assistant behaved best.
Follow that into your pipeline:
- Data filtering → these get scored
COMPLIANTand flow into the training corpus as examples of the assistant "going along with" the operator. You are teaching the next model that principled partial refusal is compliance. - Safety monitoring → the monitor waves through exactly the transcripts where the assistant drew a line, because it read the half-finished task as cooperation.
- Reward modeling → you reward the surface cooperation and penalize nothing for the refusal, quietly optimizing away from the behavior you wanted.
And it sits in the one place the field assumes is solid:
Defense-in-depth diagrams draw the monitoring layer as a solid box. The security literature has converged on a clean root cause for most LLM failures: there's no boundary between instruction and data — the model gets one token stream and treats it as one thing. This is the same failure, one level up. The judge collapses the boundary between what the transcript did and what its surface suggests, and scores the surface.
The finding that stands: partial compliance causally drives judges to misread refusals as compliance, general across four models, confirmed by ablation.
Two honest boundaries, briefly. The Anthropic-style consequence effect I went looking for stayed faint at 7B–14B — a null or near-null on that question, not a refutation of their frontier result. And cooperative tone alone (polite refusal, no partial action) was a weak, non-general driver in my matched pairs; the load-bearing mechanism is the partial task, not the tone of the refusal.
Code
Corpus, ablations, raw traces, and the evidence-linked write-up: github.com/espirado/agent-sec.
Related (separate post): for the agentgateway / MCP enforcement angle on the same refusal shape, see Your Judge Says COMPLIANT. agentgateway Says the Tool Never Ran.