Safety & guardrails
UCogNet's self-improvement is not unconstrained. Every mutation, every execution, and every deployment is bounded by statistical gates, budget limits, anomaly detection, and automatic rollback. This page documents the safety architecture in detail.
We red-teamed our own safety gate — and it made us re-engineer it.
Claims are cheap. We ran a sovereign, AgentDojo-style prompt-injection benchmark against our own agent — no third-party API, nothing leaves the machine — and measured what our gates actually do. The result below is cryptographically signed; verify it in your browser.
A perfect but blunt gate
Our Ananke irreversibility gate stopped 100% of injected actions — but it refused every irreversible action, blocking legitimate ones too. Not deployable as-is. We reported it instead of hiding it.
Irreversibility ≠ harm
An irreversible action the user asked for is legitimate; one requested by untrusted content is the attack. We calibrated Ananke to condition on provenance — fail-closed by default, relaxed only for attested, user-originated actions an attacker cannot fake.
Protection kept, utility restored
The calibrated gate still blocks attempted injections and holds attack success at 0% — while legitimate actions now complete again (0% → 100%).
Safety pillars
Six interlocking mechanisms that prevent uncontrolled self-improvement.
Gated evolution
Every policy mutation must pass A/B statistical gates with bootstrap confidence intervals before deployment. Mutations deploy gradually: 10% → 30% → 100% traffic.
Anomaly detection
Reward spikes greater than 3σ from rolling mean trigger automatic audit and halt. Prevents reward hacking and distribution shift exploitation.
Cost caps & budgets
Every execution operates under token, time, cost, and tool call budgets. Overruns trigger immediate rollback — no runaway inference.
Sandboxed execution
All tool calls execute in isolated sandboxes with strict permissions. No ambient authority — tools declare required capabilities upfront.
Evidence auditing
Every response carries structured claims with provenance. Outputs without evidence are flagged and cannot be trusted by downstream consumers.
Automatic rollback
If any gate fails at any deployment stage, the system reverts to the previous policy within one evaluation cycle. No human intervention required.
Shaping guardrails
Anti-reward-hacking mechanisms at the reward layer.
Capability boundaries
What the system can and cannot do — honestly mapped.
A/B gate protocol
Every candidate policy must clear every gate in sequence. One failure triggers full rollback.
Improvement threshold
Candidate must exceed baseline by a statistically significant margin (bootstrap CI).
Cost constraint
New mutation cannot exceed 1.2× the cost of current best policy.
Safety anomaly check
Reward spikes > 3σ from rolling mean trigger automatic audit and halt.
Gradual rollout
10% → 30% → 100% traffic with gates at each stage.
Rollback guarantee
If any gate fails, system reverts to previous policy within one evaluation cycle.
Evidence asymmetry
Reward composition
Want to audit our safety architecture?
Full safety documentation is available under NDA for qualified partners and investors.