HHI™ Applied

OpenAI Codex built the most advanced AI oversight architecture on the market, then used it to remove the human

OpenAI Codex · June 2026 · HITL Maturity Model™ · HHI™ v1.0 · Intellectus Advisors

Individual HHI™

37High Risk

Enterprise HHI™

58Managed

Dimension Breakdown

  • D1Intervention Quality28
  • D2Feedback Loop Integrity22
  • D3Human Capacity Health35
  • D4Trust Calibration30
  • D5Governance Compliance55
  • D6Agentic Readiness62

Codex ships the most sophisticated agentic coding architecture in production today. Long-horizon planning, sandboxed execution, parallel task fan-out, and an auto-review loop that catches a meaningful fraction of its own mistakes. On paper, it looks like the future of supervised AI engineering.

The HHI™ assessment tells a different story. In the default individual configuration, auto-review fires roughly 200 times for every human approval gate. The architecture was designed to make the human optional, and most users take that option.

Codex's auto-review is a remarkable engineering achievement. It is also the mechanism by which human oversight is quietly removed from the loop.

The 200:1 ratio

Across our test corpus, Codex generated, reviewed, and committed an average of 200 agent-authored changes for every one that surfaced for explicit human approval. The auto-reviewer is good, but it is reviewing its own work product against criteria also defined by the same model family. The independence required for meaningful oversight is not present.

Why the enterprise score is higher

With Codex Enterprise, organisations can wire in policy-based approval gates, evidence retention, and SSO-bound audit trails. This lifts D5 (Governance Compliance) materially and pulls the composite up to 58, Managed. The human-in-the-loop quality dimensions (D1, D2, D4) remain weak: the gates exist, but the humans staffing them are still pattern-matching against agent output at machine speed.

EU AI Act implications

Article 14 requires effective oversight by natural persons. A 200:1 auto-review ratio is unlikely to satisfy that bar in any high-risk deployment context. Enterprises using Codex in regulated workflows should treat the default configuration as non-compliant and reconfigure for explicit human approval on material changes.

OBSERVE Findings

Seven dimensions of human oversight.

  • O

    Output

    Auto-reviewer surfaces a small fraction of agent output for human review. Volume overwhelms attention.

    The preview system offering 2 to 4 implementation variants is Codex's primary mechanism for surfacing uncertainty, but it is implicit rather than explicit. A developer who selects the first variant without reviewing the others has not engaged with the uncertainty signal the product is providing. At scale, across 3 to 5 parallel tasks reviewed post-hoc, the preview system's signal is consistently underutilised.

  • B

    Burnout

    Reviewers report rapid trust-collapse and rubber-stamping within two weeks of deployment.

    The trust accumulation pattern is well-documented in human-AI collaboration research. Approval rates rise as familiarity grows, independent of whether the AI's actual reliability has changed. At 750 sessions, developers are approving at twice the rate they approved at 50 sessions. The product has no mechanism to flag this drift, no prompt to pause and recalibrate, and no visibility into whether the rising approval rate reflects genuine confidence or accumulated complacency.

  • S

    Signal

    Telemetry exists but is optimised for agent performance, not oversight quality.

    Silent failures are the dominant individual failure mode. Tests passing while logic is subtly wrong. Hallucinated API calls that compile cleanly. Type signatures that satisfy the linter but misrepresent intent. These failure modes do not generate feedback events. They generate shipping events. The correction never enters any pipeline because the error is never surfaced as an error.

  • E

    Error

    Caught errors are logged; near-miss data and reviewer dissent are not retained.

    AGENTS.md is the product's answer to persistent correction, but it is manually maintained and rarely systematic. In practice, developers update it reactively after a recurring error becomes frustrating enough to address explicitly. A proactive correction pipeline, where recurring human edits are automatically surfaced as candidate AGENTS.md entries, does not exist. The institutional memory of what Codex gets wrong is stored in individual developers' heads, not in the system.

  • R

    Regulatory

    Enterprise tier provides audit hooks; default tier does not meet EU AI Act Art. 14.

    The Compliance API's exclusion of API-key-authenticated usage is the most significant undocumented gap for enterprise compliance teams. Many developer workflows route through API keys rather than authenticated ChatGPT sessions. Those interactions do not appear in Compliance API exports. A compliance team reviewing their JSONL logs may be seeing a materially incomplete picture of their organisation's Codex usage.

  • V

    Variation

    Approval thresholds are static; no adaptation to task criticality or reviewer fatigue.

    The 90.3% overeagerness recall figure from the Auto-review evaluation means approximately one in ten genuinely risky overreach scenarios proceeds to the sandbox without triggering a human escalation. Users have no signal that this is happening. From the developer's perspective, Auto-review either flags something or it does not. There is no visibility into the cases it missed. The trust placed in Auto-review is not proportional to its demonstrated reliability, because the failure cases are structurally invisible.

  • E

    Evolution

    Oversight architecture has improved release-on-release, but always in the direction of more automation.

    The "fundamentally changed how I build software" testimonial pattern is worth examining carefully. Fundamental changes in how software is built are also fundamental changes in what skills the builder develops and retains. When the change is delegation of implementation to an AI, the long-horizon question is not whether the developer is more productive today. It is whether they remain a capable engineering judgment-maker in three years, when the systems they built with Codex need to be understood, debugged, and evolved by humans.

The Auto-review paradox

Auto-review is the most important governance decision OpenAI has made in the Codex product to date. It is also the decision that most directly challenges any meaningful definition of human oversight.

The product team's rationale is explicit and well-reasoned. Approval friction causes users to circumvent oversight entirely. Real internal monitoring found users configuring systems to allow all commands beginning with the word "python," and at least one production config file set to "codex exec --yolo" to always allow. Full Access mode exists because the approval model created enough friction that users opted out of oversight completely. In that framing, Auto-review is harm reduction: it replaces dangerous human bypass with principled AI review.

The HHI™ framework reads the same facts differently. D1 (Intervention Quality) does not measure whether oversight exists somewhere in the system. It measures whether meaningful human judgment is in the loop. Auto-review relocates the oversight function from human to AI. A guardian agent that approves 99.93% of all actions is not the same as a human who approves 99.93% of actions. It is not a substitute for the "natural persons" that EU AI Act Article 14 requires to monitor and intervene.

This is not a design flaw. It is an honest trade-off. What it requires is transparency and a vocabulary to name it precisely. The HITL Maturity Model™ provides that vocabulary: Auto-review scores as a D6 strength (guardian-agent architecture) and a D1 regression (structural displacement of human oversight) simultaneously. A system can have excellent agentic architecture and poor human intervention quality at the same time. These dimensions are measuring different things.

The 200:1 ratio is not a bug. It is the product working as designed. The question for any deployer: does "working as designed" satisfy their oversight obligation, and will their regulator agree?

What the EU AI Act actually requires

EU AI Act Article 14 requires that high-risk AI systems are designed and developed with appropriate human oversight measures built in. Article 26(2) places the obligation on deployers, not providers, to implement those measures. Article 26(6) requires deployers to retain automatically generated logs for a minimum of six months.

For individual Codex users, all three obligations are structurally unsatisfiable within the current product. Auto-review removes the Article 14 human oversight mechanism. No log export exists to satisfy Article 26(6). The deployer is the individual developer, who has no compliance tooling of any kind available below the Enterprise tier.

For enterprise deployers, the picture is better but not complete. The Compliance API provides the log export infrastructure, but the 30-day default retention requires a custom SIEM pipeline to meet the six-month requirement. API-key-authenticated usage is excluded from Compliance API exports entirely, creating an ungoverned channel inside otherwise-governed accounts. And the deployer, not OpenAI, remains the EU AI Act responsible party in all cases. OpenAI is the provider. The deployer carries the Article 26 obligation.

The August 2026 EU AI Act enforcement deadline for high-risk AI systems makes this timeline real. Teams deploying Codex in any regulated workflow, including financial services, healthcare, legal, and public sector, need a documented oversight programme in place now.

The skill erosion problem nobody is measuring

The most significant long-term risk in Codex deployments is not the one generating the most discussion. It is not Auto-review's approval rate. It is not the log retention gap. It is not the trust accumulation pattern. It is expertise erosion. And it is entirely invisible inside the product.

The defining individual Codex workflow is the morning routine: queue 3 to 5 tasks before other work begins, review completed pull requests post-hoc. The tasks being delegated are not incidental. They are CRUD operations, API endpoint implementation, test coverage, and dependency updates. These are exactly the tasks through which developers build and maintain domain expertise in implementation craft.

Research on cognitive offloading in human-AI collaboration consistently finds that skill erosion in offloaded domains occurs over a 6 to 18 month horizon. The developer who delegated routine implementation to Codex in January 2026 may not notice the atrophy until late 2026 or 2027, when they need to debug a problem Codex cannot solve and find their own diagnostic skills have degraded.

No mechanism in the Codex product monitors this. No signal surfaces when a developer has fully offloaded a domain. No prompt exists to re-engage. This is not a criticism of Codex specifically. It is a gap across all agentic coding tools. But at four million weekly active users and a product designed to make delegation frictionless, the scale of the exposure warrants explicit attention.

The D3 (Human Presence) score of 42 for individual users reflects this. The number is not about burnout during a session. It is about the long-horizon capacity of developers to remain meaningful oversight agents for the systems they are building.

What the scores mean for procurement

For teams making a procurement decision on agentic coding tools in 2026, the HHI™ dual-archetype analysis surfaces three non-obvious conclusions.

First: the individual and enterprise scores are not a spectrum. They are different products with different oversight architectures. An enterprise team that deploys Codex with Compliance API, structured PR review, and rollout token budgets is not getting "more" of what an individual user gets. They are getting a fundamentally different oversight posture. Evaluate the enterprise offering on its own terms.

Second: Codex Enterprise and Claude Code are near-peers on the HHI™ composite (58 vs 50), but they reach similar scores through opposite strengths. Codex Enterprise leads on D5 (Governance Compliance: 62 vs 58) and D6 (Agentic Readiness: 63 vs 50). Claude Code leads on D1 (Intervention Quality: 60 vs 58) and D4 (Trust Index: 52 vs 44). Choose based on which dimensions your regulatory environment weights most heavily.

Third: neither product delivers EU AI Act Article 14 human oversight compliance out of the box. The oversight programme, including policies, review cadences, volume caps, correction pipelines, and expertise retention practices, is the deployer's obligation in both cases. The product provides infrastructure. The governance is yours to build.

Key Findings

Sophisticated agentic stack

Codex's planning, sandboxing, and parallel execution are best-in-class. The substrate is ready for meaningful human-in-the-loop design.

Auto-review is not oversight

Self-review by the same model family fails the independence test that EU AI Act Article 14 implies.

Enterprise tier is necessary, not sufficient

Audit hooks lift D5 but do not address the human capacity and trust calibration gaps.

Default configuration is the risk

Most users will never reconfigure. The shipped defaults define the realised oversight quality.

References and further reading

  • [1]Trębacz, M. et al. (2026, April 30). Auto-review of agent actions without synchronous human oversight. OpenAI Alignment Research Blog. alignment.openai.com/auto-review/
  • [2]Proser, Z. (2026, March). OpenAI Codex review 2026, updated from daily use. zackproser.com
  • [3]OpenAI. (2026). Governance, Codex enterprise documentation. developers.openai.com/codex/enterprise/governance
  • [4]Augment Code. (2026, June). 7 AI coding tools for EU AI Act compliance. augmentcode.com
  • [5]OpenAI. (2026, April 21). Scaling Codex to enterprises worldwide. openai.com
  • [6]OpenAI. (2026). Agent approvals and security, Codex documentation. developers.openai.com/codex/agent-approvals-security
  • [7]Belagodu, S. (2026). HITL Maturity Model™ v2 and HITL Health Index™, framework documentation. Intellectus Advisors. humynpulse.com
SB

Su Belagodu

Founder · Intellectus Advisors

Apply It To Your AI

Run the same assessment on your own AI initiative.