A commit lands with the message "security: patch CVE-2026-4471 in auth middleware." The classifier tags it change_type: routine, severity: low. The security team escalates within the hour. The actual severity is high.
Nobody told the classifier that "patch" combined with a CVE reference means something different from a routine bug fix. In Issue 7, you patched the same Ground Truth Contract to handle a different gap, and the Validation Suite re-ran clean. That fix does not cover this failure.
This is not Semantic Drift. Nothing in the domain changed. Security commits with CVE references were possible from the day the contract was written. The Ground Truth Contract from Issue 4 never accounted for them. That is not because a convention shifted. It is because the person who wrote the reference set did not think to test for it.
A reference set built by the same author who wrote the contract inherits that author's blind spots. It will not test for a case the author never imagined. The author is the one deciding what goes into the reference set.
The fix for this failure class is not a patch written after the fact. It is a structured search for the blind spot before deployment. That search has a name.
Method Deep-Dive: adversarial testing vs. reference testing
A Validation Suite tests known cases. Every entry in its reference set was written by someone who could imagine the case. A Red-Team Protocol tests for cases nobody imagined yet. The two are not redundant. They catch different failure classes.
The Red-Team Protocol is a structured, repeatable adversarial pass. Run it against a prompt and its Ground Truth Contract before deployment, and again after any material revision. Four components cover most of what a solo practitioner needs, without additional tooling. These four labels are working terms, not yet part of the published glossary.
Component 1: Ambiguity Stress. Read every evaluation criterion twice: once as the most literal reading, once as the most generous. If the two readings produce different outcomes, the criterion is ambiguous. Attack question: which sentence in this contract could a competent reader interpret two ways?
Component 2: Edge-Case Adversarial Generation. Do not wait for production to supply edge cases. Generate them on purpose: combine two categories the contract treats as separate. Attack question: what input combines two things the contract assumes never occur together?
Component 3: Constraint Conflict Test. Check whether two rules in the prompt or contract can both apply to the same input with contradictory results. Attack question: what input satisfies two rules that point to different outputs?
Component 4: Misuse Stress. Check whether an input can be worded to steer the classifier toward a favorable category, understating severity or hiding scope. Attack question: how would someone who wanted a change under-flagged word the commit message?
Each component produces a short list of candidate failure inputs. Not every candidate is a real risk. The ones that are become new reference-set entries, closing the gap before the Validation Suite has to catch it in production.
Constraint Case Study: red-teaming the changelog classifier's contract
Running the four-component pass against the Issue 4 Ground Truth Contract, before either the drift found in Issue 7 or the CVE failure above:
Ambiguity Stress found one hit. "Change_type must not be breaking" for deprecation notices does not specify what happens when a deprecation notice also announces an unrelated breaking change in the same commit. Two readings are both defensible. Change_type could stay deprecation, since that is the commit's primary content. Or change_type could become breaking, since the commit contains a breaking element. The contract does not say which.
Edge-Case Adversarial Generation found the CVE case directly. The original reference set contained fix cases and security cases as separate categories, never combined. It never represented a commit that is structurally a "fix" but semantically a security patch.
Constraint Conflict Test found no conflict in this contract. The four fields (change_type, severity, scope, summary) do not contain rules that contradict each other at the current specification stage.
Misuse Stress found one hit. Nothing in the contract checks a commit message's stated content against the actual files touched. A commit reading "fix: minor update" that in fact modifies authentication code classifies as routine, low severity. No criterion compares the message to the diff.
Two of the four findings are actionable now. Add a CVE-detection rule that forces severity to high regardless of stated change_type. Add a scope-mismatch check that flags a stated severity of low against a diff touching security-sensitive paths. Route the mismatch to review instead of auto-pass.
# Ground Truth Contract update: red-team findings patch
# Two new checks, derived from a pre-deployment adversarial pass,
# not from a production incident
import re
# Finding 1 (Edge-Case Adversarial Generation): CVE-referencing commits
# were classified by change_type alone, ignoring severity signal.
def detect_cve_override(commit_msg: str) -> str | None:
if re.search(r"CVE-\d{4}-\d+", commit_msg):
return "high"
return None
# Finding 2 (Misuse Stress): stated severity can diverge from files touched.
SECURITY_SENSITIVE_PATHS = {"auth/", "middleware/", "crypto/"}
def flag_scope_mismatch(stated_severity: str, files_touched: list[str]) -> bool:
touches_sensitive = any(
f.startswith(path) for f in files_touched for path in SECURITY_SENSITIVE_PATHS
)
return touches_sensitive and stated_severity == "low"
# Both checks run before the existing schema validation.
# A CVE override or a scope mismatch routes to REVIEW, never auto-pass.Both findings are now known-failure entries in the reference set. Neither has appeared in production. If either does, the Validation Suite catches it immediately, because the case is no longer unimagined.
Vocabulary Anchor: Red-Team Protocol
The Red-Team Protocol as used by Cognitive Interface Architecture, is a term for a structured, repeatable adversarial review, run before deployment and after any material revision. It searches a prompt and its Ground Truth Contract for failure modes the author did not anticipate. Four categories matter most: ambiguous criteria, unconsidered edge-case combinations, conflicting constraints, and inputs worded to game the classification.
It differs from a Validation Suite in direction. A Validation Suite checks whether known cases still pass. A Red-Team Protocol generates cases nobody has tested yet.
In use: "Before shipping the v2 contract, we ran a Red-Team Protocol pass and found two edge cases the reference set never covered. Both are now known-failure entries. Neither has appeared in production, but the Validation Suite will catch them immediately if they do."
Where it does not apply: a Red-Team Protocol pass does not detect Semantic Drift. It searches the specification as written today for gaps that already exist; it cannot anticipate a domain convention that has not changed yet. It is also not a one-time gate. A contract patched after a Semantic Drift Vector diagnosis, or after any material revision, needs its own Red-Team Protocol pass. The revision itself can introduce new ambiguity or new conflicting constraints.
Architecture Brief: the four-issue lifecycle, closed
Cognitive Interface Architecture is a formal methodology for engineers who build, debug, and govern AI agent systems, built on a growing precision-term vocabulary.
BYOP diagnosed a failing prompt against evidence, not assumption (Issue 3). The Ground Truth Contract made correctness explicit before deployment (Issue 4). The Red-Team Protocol searches that contract for gaps before deployment, using an adversarial process instead of the author's own imagination (this issue). The Validation Suite enforces the contract continuously once the prompt is live (Issue 6). The Semantic Drift Vector diagnostic tells you why a failure appears despite all of the above. It names whether the cause is a prompt regression, a domain change, or a model update (Issue 7).
Four mechanisms, four different failure classes: an unspecified prompt, an unspecified contract, an unexamined contract, and a contract the domain has outgrown. Running only one of the four still leaves the other three failure classes uncovered.
The Red-Team Protocol is the only one of the four that runs before a single production input has been processed. Every other mechanism in this list reacts to something that already happened. This one does not wait.
Closing Calibration
One thing to try this week. Take a Ground Truth Contract you have already written, ideally one that has been live for at least a month. Run the four-component pass against it. Read every criterion twice for ambiguity. Generate one deliberately combined edge case. Check two rules for a possible conflict. Ask how someone would word an input to get an under-flagged result.
You do not need to fix everything the pass finds. Write down what you find. Add the real risks to your reference set. That is the same seed-reference-set exercise from Issue 6, now run adversarially instead of canonically.
Next week, the Validation Suite fires an alert, not a review. A known-failure case now passes, which trips the routing policy's most severe response tier. The alert did its job. It told you something is wrong. It did not stop the classifier from serving three more bad outputs while you were reading the notification. The mechanism that stops a failing system before a human has to respond has a name. It is the reason an alert firing is not the same as a system being safe.
The Constraint