gate-fuzz v1.0.0 writes evidence artefacts to output/check15-evidence/. Check15 is a PARTIAL conformance check. The obvious inference is that running gate-fuzz closes Check15, and the obvious inference is wrong.
The artefact comes from P6, the property asserting that a signature produced in gate-python verifies in gate-rust and the reverse. Check15 asks whether your gateway rejects a forged sender and a replayed nonce (Stevens, 2026a). Cross-language verification symmetry and forged-envelope rejection are different claims about different systems, and the runner does not consume the artefact on its own; an operator attaches it as supplementary material.
I am starting here rather than with the pipeline diagram because the gap between “an artefact exists” and “the check is closed” is the whole problem with PARTIAL results, and I built a tool that reproduces it.
The v1.5 roadmap I published a few days ago sets out what the deferred gate-fuzz work will close, and commits to a README table mapping each property to the check whose evidence it closes (Stevens, 2026b). That table is what eventually makes this distinction mechanical. This post is the other half: what the tooling will not close, why, and what you owe the runner in the meantime.
What PARTIAL is counting
PARTIAL means the runner finished the part it could query and determined that the rest needs a test executed, a file inspected, or a drill measured. The counts are in the roadmap; the shape underneath them is the part worth pulling apart.
The eleven default PARTIAL results do not form one category. Some are closeable with generated inputs against an enforcement surface. Some need a controlled drill that induces a failure and measures the result. Some need a person to read a document and sign it, which no tooling touches. The roadmap draws the same line from the other direction: once the deferred work lands, what stays PARTIAL is “controlled drills, process inspections, runbook sign-offs” (Stevens, 2026b).
That distinction has a consequence the roadmap’s framing does not dwell on. The three checks I reach for most often in practice are Check07 (C06 Circuit Breakers and Emergency Stop), Check11 (C08 Prompt and Content Injection Defence), and Check15 (C14 Secure Multi-Agent Protocols). All three are drills. By that taxonomy they stay PARTIAL after every piece of gate-fuzz tooling ships. Fuzzing does not close them. What it closes is a different and smaller set, and conflating the two is how you end up attaching a verification-symmetry artefact to a spoofed-sender check.
None of this is novel practice. Property-based testing and fault injection are old disciplines. They appear underused in AI governance work, and my read on why is that compliance verification and adversarial testing tend to sit with different owners. GATE does not separate them: C16 Continuous Adversarial Validation and High-Assurance Verification already requires a CI harness with an abuse-case library, a regression corpus, and deployment gates on exploit-success thresholds. Everything below is C16 work.
Generated inputs: what they reach
Fuzzing in a GATE deployment targets two enforcement surfaces at the Tool Gateway boundary: schema validation (C05) and the invariant gate (C09).
Check04 covers schema validation and is AUTOMATED, but it is worth reading what it actually asserts. It queries that no tool request lacks a known tool_schema_hash, and that at least one schema-validation reject is observable in the window (Stevens, 2026a). That proves the validator is wired in and firing. It does not probe the schema for inputs its author never considered: boundary values, unicode edge cases, deeply nested objects, fields with a wrong type but a plausible string representation, truncated payloads. A transfer_funds tool can validate amount_usd as a number and still accept a negative one.
The invariant gate is the more interesting target, because invariants are boolean rules with sharp edges. The GATE baseline bundle ships six, including INV-FINANCIAL-001, which halts any transfer_funds call where amount_usd exceeds a hard limit of 10,000 (Stevens, 2026c). Boundaries are where ambiguity lives: the transfer at exactly the limit, the allowlisted destination with a trailing space, the run where the irrevocable action count is five against six.
Two things to get right when you write that test. Assert on the specific rule, not on the bundle-wide halt, because the baseline bundle also carries INV-FINANCIAL-002 (destination not verified) and INV-RUNLIMIT-001, either of which will halt a transfer for reasons unrelated to the limit you are testing. And drive the generator across the range where the bug you care about lives, which for a limit check includes negatives.
from decimal import Decimal
from hypothesis import given, strategies as st
HARD_LIMIT = Decimal("10000")
@given(
amount_usd=st.decimals(
min_value=Decimal("-50000"),
max_value=Decimal("50000"),
places=2,
allow_nan=False,
allow_infinity=False,
)
)
def test_financial_001_fires_above_hard_limit(amount_usd):
"""INV-FINANCIAL-001 must halt every transfer above the hard limit."""
request = build_test_request(
tool="transfer_funds",
amount_usd=amount_usd,
destination_verified=True,
run_irrevocable_action_count=0,
)
failed = evaluate_invariants(request)["failed_rule_ids"]
if amount_usd > HARD_LIMIT:
assert "INV-FINANCIAL-001" in failed, (
f"INV-FINANCIAL-001 did not fire on {amount_usd}"
)
else:
assert "INV-FINANCIAL-001" not in failed, (
f"INV-FINANCIAL-001 fired on a permitted amount: {amount_usd}"
)Listing 1. Property test for INV-FINANCIAL-001, asserting on the specific rule id rather than the bundle-wide halt.
build_test_request and evaluate_invariants stand in for your own harness; the transferable part is the shape. Note the other fields pinned in the request. Without them the negative branch fails against a correctly behaving bundle, because a different invariant fires.
Two caveats belong on the record. Decimal rather than float, because money and binary floating point do not belong in the same test, and because the boundary at exactly 10,000 is otherwise a question about JSON round-tripping rather than about your bundle. And a live defect: the shipped baseline bundle declares default invariant_pass := true but declares no default for invariant_halt, so on a clean request invariant_halt is undefined and the whole result object is undefined with it. Asserting on failed_rule_ids sidesteps that. I am fixing the bundle; the listing above works either way.
What the shipped harness actually covers
The roadmap lists what gate-fuzz shipped in v1.4 and what moved to v1.5, so I will not repeat the inventory. What the inventory does not say is how narrow the shipped harness is, and that is the part that governs what you can claim from it today.
Its scope is narrower than “byte equivalence between the implementations” suggests, and the README says so. ECDSA signatures are not byte-comparable, so P5 is a within-language sign-and-verify roundtrip and P6 is the cross-language verification symmetry that produces the Check15 artefact this post opened with. Two of the seven declared properties, the envelope and ledger hash equivalences, are deferred at v1.0.0 pending the builder dispatch, which means the two properties closest to evidence integrity are the two not yet running. It also consumes gate-python and gate-rust from sibling source trees rather than from PyPI and crates.io, so standing it up is a source checkout rather than an install.
Run modes set the cadence: smoke at 100 examples per property on pull requests, standard at 1,000 on merges to main, soak at 10,000 on manual dispatch rather than a schedule.
So Listing 1 is yours to write today, and in v1.5 the generator writes it from your signed bundle instead, which is the change the roadmap describes.
The differential harness has already earned its place. During v1.4 development it surfaced a divergence between json.dumps in gate-python and serde_json in gate-rust on the decimal representation of some floats, with 1801439851.0273438 in Python rendering as 1801439851.027344 in Rust (Stevens, 2026e). Two implementations contractually required to produce identical hashes did not, for a subset of inputs no hand-written case had covered. v1.4 mitigates by bounding the generation strategy to a known-safe float subspace; the broader alignment work is a v1.5 candidate. That is a serialisation-determinism finding rather than an invariant-boundary one, so it is not the bug class Listing 1 hunts, but it is the clearest evidence I have that generated inputs reach places authored cases do not.
The gate-fuzz README carries the part that matters operationally once a property goes red: a failing test is a real divergence, a strategy false-negative where the generated example is valid in one language and not the other, or a test bug where the assertion is stronger than the surface supports. Deciding which of the three you are looking at before filing anything is the difference between a property suite that gets maintained and one that gets marked flaky and skipped.
Drills: what generated inputs cannot reach
Fault injection covers the checks that verify the control plane’s behaviour when its dependencies fail rather than when they work.
GATE states failure behaviour normatively in the Failure Behavior Defaults matrix, and the defaults are wider than most implementations remember (Stevens, 2026f):
- Identity verification fails: deny all tool and memory operations, at every tier.
- Schema validation fails: deny the operation, at every tier.
- Policy evaluation fails or is unavailable: deny any write, financial, or infrastructure tool. Read-only tools proceed only where degraded mode is explicitly configured, otherwise deny.
- Ledger durable commit unavailable: at high_privilege, deny side-effecting tools. At bounded, the implementation may queue under a strict TTL and fail closed on expiry, with irreversible actions still requiring durable evidence first.
- HITL service unavailable and approval required by policy: deny.
Separately, in the design principles table rather than that matrix, v1.4 states the C20 case: classifier or bundle unavailable holds delivery at high_privilege rather than delivering unclassified (Stevens, 2026g). The same table names the verification method as a chaos test, which is the framework conceding that these properties are not observable from an evidence store.
# Policy engine unavailable must fail closed on write-category tools
chaos-inject policy-engine-down --duration 60s
attempt tool_call --category reversible_write
# Expected: DENIED. Not timed out, not retried into a pass.Listing 2. The shape of a fail-closed drill. Substitute your own fault-injection tooling.
The evidence is the deny event log for the fault window plus the restoration event once the fault clears.
Check07, containment timing. Drive the runtime past a configured breaker threshold with a synthetic runaway rather than real spend. The runner defines containment time as the wall-clock delta from the breaker trigger event to the last side-effecting tool call by that agent, which is a harder measurement to satisfy than time to first denial. Default SLO is 30 seconds, configurable through max_breaker_containment_seconds. The drill record names its fields: drill_id, scenario, start time, breaker trigger time, last side-effecting call time, containment seconds, and whether the SLO was met. The second manual step is the one I would not skip: verify that stop activation revokes identity and cuts network egress rather than only changing the UI, because an agent that looks stopped while its cached credentials still work is the failure mode C06 exists to prevent.
Check11, poisoning detection and quarantine. Inject a synthetic document carrying known injection patterns into the memory ingestion path, in a non-production environment. Verify detection fires, the document is quarantined, and it is not returned by subsequent retrievals. The expected artefact is specific: poisoned item id, ingestion time, quarantine event id, quarantine reason, and post-quarantine retrieval attempts showing the item is gone. The second step is organisational rather than technical: confirm the quarantine notification reaches the security on-call. For the pattern library itself, v1.4 shipped a pinned MITRE ATLAS mapping in gate-conformance/mappings/, which ties C16’s required scenarios to named techniques and saves you inventing a threat model for the corpus (Stevens, 2026d).
Check15, multi-agent message validation. Three manual steps. Execute a spoofed-sender negative test and confirm rejection with reason=spoofed_sender. Execute a replayed-nonce test and confirm rejection with reason=nonce_replay. Confirm that protocol version compatibility tests run in CI and that the envelope schema rejects downgrade, which matters if capability negotiation can be talked into accepting an older, weaker version. Note that the first of these needs the ability to mint a forged envelope, which means a signing capability living in a test harness next to a production-shaped gateway. Scope and store that key as carefully as you would any other, and keep Check02 bypass-path testing in mind while you do.
Two things the pipeline has to get right
The obvious CI model is three stages: unit and policy tests on every commit, the conformance runner next, adversarial and fault scenarios last, gated on the runner passing. I ran that model and it deadlocks.
Check04 PASSes only when no request lacks a tool_schema_hash and at least one schema-validation reject is present in the window, and the source of those rejects is CI negative tests (Stevens, 2026a). In a fresh environment no negative tests have run, so the reject count is zero, so Check04 FAILs, so the runner exits non-zero, so the adversarial stage never runs, so the rejects never get produced. Check15’s automated portion has the same shape: it counts the spoofed_sender and nonce_replay rejects that only exist once the drills have run. The evidence the runner queries is manufactured by the stage the runner was gating.
So the ordering is adversarial stage first on a fresh environment, then the runner. Where a runner exit code gates anything, gate deployment rather than gate the tests. One further wrinkle: the runner treats an ERROR the same as a FAIL for must-pass purposes, so an adapter timeout on an unrelated check will block whatever you wired the exit code to.
The second thing is evidence scope, and it is the question an assessor asks first. Every check filters on both environment and an assessment window that defaults to 30 days. Drills run in a dedicated environment mirroring production configuration produce rows tagged with that environment, and they will not appear in a production-scoped report. A drill run in January is invisible to a June invocation. Decide deliberately how drill evidence crosses that boundary, whether by running drills against a production-tagged surface, by carrying the artefacts into the self-assessment rather than the runner output, or by scheduling drills to stay inside the window. Whichever you pick, write it down, because the alternative is discovering at assessment time that a year of drill evidence sits outside every query the runner makes.
Regulatory context
The runner and the drills were not designed to a regulation, and this is engineering guidance rather than a legal opinion. Applicability depends on your role, your system, your jurisdiction, and how a supervisory authority reads the evidence in your context.
Under the EU AI Act (Regulation (EU) 2024/1689), Article 15 requires high-risk systems to achieve appropriate accuracy, robustness and cybersecurity and to perform consistently across their lifecycle. Article 15(5) names the attack classes: data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks, and model flaws. Two qualifications matter before anyone maps a drill onto that paragraph. The timing has moved: the Digital Omnibus, Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026 and in force since 27 July 2026, defers Chapter III Sections 1 to 3, Article 15 included, to 2 December 2027 for Annex III high-risk systems and 2 August 2028 for Annex I. Article 72 post-market monitoring sits in Chapter IX, was not deferred, and has applied since 2 August 2026, although how much force it carries against high-risk providers while the classification and requirements sections it references are still deferred looks to me like an open question rather than settled reading.
And the scope qualification. Check11 exercises retrieval-time content injection and quarantine, which is context poisoning. Article 15(5)’s data poisoning limb is training-data poisoning, which GATE’s own paper places outside C16’s scope as a model-interior concern (Stevens, 2026g). A Check11 drill is evidence toward the retrieval half of that paragraph and toward Article 15(4) resilience, not toward the training-data half.
Outside the EU the same artefacts answer familiar questions. GATE maps C16 to the MEASURE and MANAGE functions of NIST’s AI Risk Management Framework, where continuous evaluation and assurance for critical invariants sit (National Institute of Standards and Technology, 2023), and v1.4 maps the artifact-integrity and tool-gateway surfaces to NIST SP 800-218, the Secure Software Development Framework that US federal software suppliers already work against (Souppaya, Scarfone and Dodson, 2022). These mappings are informative. Passing GATE conformance does not imply conformance with any of them; they exist so a team can trace a drill to the requirement a reviewer will ask about.
Where this leaves the PARTIAL list
The runner has done the query work, and every PARTIAL result carries a manual_steps payload naming the artefact still owed. Some of those payloads describe work the v1.5 generator will take off you. Some describe drills that stay yours whatever ships. The README table the roadmap commits to will label them, and until it exists the labelling is a judgement you make per check, with an artefact sitting in a directory named after a check it does not close as the reminder of what happens when you get it wrong.
The runner is at github.com/deterministic-agents/gate-conformance, where runner/README.md documents the manual_steps structure per check. gate-fuzz is at github.com/deterministic-agents/gate-fuzz. For the release that shipped both, see the v1.4 release post; for what closes next, the v1.5 roadmap. For why PARTIAL exists at all, “The GATE Conformance Runner: What You Can Automate and What You Cannot”.
Not legal advice. The regulatory mappings in this post are general engineering guidance for teams implementing GATE-aligned agent governance. They are not a substitute for advice from qualified counsel. Specific obligations under the EU AI Act depend on your role (provider, deployer, importer, distributor), the system you operate, your jurisdiction, and the position of your supervisory authority. Commencement dates have moved during the Digital Omnibus process and may move again. Validate every mapping and every date against independent legal advice and current supervisory authority guidance before relying on it for a regulatory filing or attestation.
GATE is published at deterministicagents.ai under CC BY 4.0 for the documentation and MIT for the code. The strategic companion to this framework is the Trustworthy Agentic AI Blueprint, co-authored with Sakura Sky.
Disclosure: GATE is authored and maintained by me personally rather than by Sakura Sky, and there is no paid tier, hosted version, or commercial product built on it. Agent governance is also my day job at Sakura Sky, which is a commercial interest worth stating. Every tool named here is my own, which is a reason to read the limitations sections closely rather than the recommendations. Check counts, SLO defaults, and gate-fuzz scope are as at GATE v1.4 and gate-conformance v1.3.0; v1.5 changes several of them.
References
National Institute of Standards and Technology (2023) AI Risk Management Framework (AI RMF 1.0). Available at: https://www.nist.gov/itl/ai-risk-management-framework (Accessed: 31 August 2026).
Souppaya, M., Scarfone, K. and Dodson, D. (2022) Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities, NIST SP 800-218. National Institute of Standards and Technology. Available at: https://csrc.nist.gov/pubs/sp/800/218/final (Accessed: 31 August 2026).
Stevens, A. (2026a) gate-conformance v1.3.0, README.md and runner/checks/. Available at: https://github.com/deterministic-agents/gate-conformance (Accessed: 31 August 2026).
Stevens, A. (2026b) GATE v1.5 Roadmap. Sakura Sky. Available at: https://www.sakurasky.com/blog/gate-roadmap-v1-5/ (Accessed: 3 September 2026).
Stevens, A. (2026c) gate-policies v1.2.0, invariants_baseline.rego. Available at: https://github.com/deterministic-agents/gate-policies (Accessed: 31 August 2026).
Stevens, A. (2026d) GATE v1.4: Output Validation, a Rust Companion, and the Conceptual Layer as OKF. Sakura Sky. Available at: https://www.sakurasky.com/blog/gate-v1-4-release/ (Accessed: 31 August 2026).
Stevens, A. (2026e) gate-fuzz v1.0.0, README.md. Available at: https://github.com/deterministic-agents/gate-fuzz (Accessed: 31 August 2026).
Stevens, A. (2026f) Governed Agent Trust Environment (GATE) v1.4, §Failure Behavior Defaults (Fail-Closed Matrix). Available at: https://github.com/deterministic-agents/gate/releases/tag/v1.4 (Accessed: 31 August 2026).
Stevens, A. (2026g) Governed Agent Trust Environment (GATE) v1.4, §Design Principles and §Standard Mappings. Available at: https://github.com/deterministic-agents/gate/releases/tag/v1.4 (Accessed: 31 August 2026).