AI governance

Who Is Accountable When the Agent Acts: The Value-Chain Liability Split

The agentic value chain describes who builds the system, not who pays when it errs. Across vendor terms, API mechanics, and existing law, the loss flows downhill and pools at the deployer. Here is why the confirm step is a liability firewall, not red tape.

By Siddharth Surana, Founder & CEO  /   /  9 min read

A delicate deep-green and parchment emblem: a single human hand poised above a branching chain of six small nodes, the final node glowing amber as the point where the chain settles.

When an AI agent acts on your behalf and something goes wrong, accountability does not spread evenly across the six parties who built the system. It pools at one of them. The deployer. The business that pointed the agent at its customers. The agentic value chain (model developer, tooling and MCP provider, platform provider, system provider or app developer, deployer, end user) is an accurate map of who builds the system. It is a poor map of who pays when the system errs. Read across vendor terms of service, the mechanics of API authentication, and existing liability law, and the same thing happens every time: the loss flows downhill and settles on the party that cannot disclaim it. That is the operational truth every small and mid-sized business should plan around. It is also exactly why a confirm step, bounded permissions, and an audit trail are not bureaucratic friction. They are the only liability controls the deployer actually owns.

This is Dimension 2 of the IMDA Model AI Governance Framework for Agentic AI: make humans meaningfully accountable. The framework (Version 1.5, published 20 May 2026, updated 5 June 2026) is voluntary guidance and emerging best practice. It is not law, it carries no penalties, and it is not industry specific. We treat it here as the clearest available articulation of a problem the law has not yet fully resolved, then connect it to the wider market and the regulation that does have teeth.

01

The reframe: the value chain describes who builds, not who pays

IMDA Dimension 2 asks deployers to allocate responsibility both inside the organisation (decision makers, product teams, cybersecurity teams, users) and across the agentic value chain. That allocation is the right governance instinct. The market reality is harder. Five of the six named parties have, in the standard pattern of commercial software contracts, disclaimed their way out. Damages capped at fees paid. Consequential loss disclaimed. No warranty of merchantability. Indemnification flowing down toward the deployer. The sixth party, the deployer, cannot do the same, because the obligations attach to whoever directs the agent toward customers and their data.

The cleanest mental model for why is not "defective product." It is the centuries-old common law of agency: the deployer is the principal, the agent is the principal's authorised representative, and the principal is bound by the agent's acts toward third parties. We label this clearly as an analogy, not doctrine. No court has ruled an AI agent a "digital employee," and the principal-agent framing is a proposed lens, not settled law. But it captures the logic an SME has to plan around. The API key that authenticates the agent's call is, in practice, your company's signature on every action it takes.

02

What the law actually says (the EU killed its dedicated AI-fault regime)

The most common misconception in 2026 is that a special "AI liability law" is coming to rescue deployers. The opposite happened. The European Commission formally withdrew the EU AI Liability Directive; the withdrawal was published on 6 October 2025 (OJ reference C/2025/5423), a directive first proposed back on 28 September 2022. The Commission's stated rationale is that the revised Product Liability Directive plus the AI Act already provide a liability framework, making a separate fault-based AI regime unnecessary.

That revised regime is real. The Product Liability Directive (EU) 2024/2853 of 23 October 2024) applies to products placed on the market after 8 December 2026. It explicitly brings software and AI systems into no-fault strict liability and eases the claimant's burden with a rebuttable presumption of defectiveness where the case is technically too complex to prove. That presumption pushes some exposure upstream onto manufacturers, including model and system developers. Whether it lands on the developer or the deployer in any given set of facts is an unresolved overlap, not a settled map. We flag that honestly: the allocation is a live legal question, not a consensus.

The lesson for an SME is bracing. No bespoke AI law is coming. You are judged by the boring, existing law of who-authorised-this-action, and that law has always pointed at the principal.

03

Singapore: outsourcing the work does not outsource the liability

The Singapore floor is unambiguous. Under PDPA section 4(3), the engaging organisation carries the same obligation under the Act "as if the personal data were processed by the organisation itself," even when a data intermediary (your vendor) does the processing. You can outsource the work. You cannot outsource the liability. IMDA's voluntary framework advocates allocating responsibility across the value chain; the legal floor underneath it still lands on the deployer. Both are true, and the gap between the governance ideal and the legal reality is precisely what a board should be asking about.

One hard line for the C-suite. Do not let anyone tell the market that a human-in-the-loop checkbox "satisfies the AI Act" or "solves compliance." In Singapore, holding out specialist knowledge you do not have is misrepresentation exposure, not governance.

04

The agentic value chain: who is accountable for what

Value-chain roleWhat they doWhat they typically disclaimWhat the SME deployer can actually control
Model developerTrains and serves the model weightsDamages capped at fees paid; consequential loss disclaimed; non-determinism not warrantedProvider choice; demand model cards and logs; nothing operational
Tooling / MCP providerSupplies tools and protocol integration (MCP for agent-to-tools)"As is," no merchantability warrantyWhitelist trusted MCP servers; log every tool call and response; sandbox code execution
Platform providerHosts the agent runtimeIndemnification flows down to the developerEvaluate isolation and audit-log export; cannot change their indemnities
System provider / app developerWrites the application and orchestration logicOften the SME's own vendor or in-house teamContractually mandate least-privilege scopes and approval checkpoints
Deployer (the SME)Points the agent at real customers and dataCannot disclaim; PDPA s4(3) and agency principles land hereThe confirm step, bounded permissions, transaction caps, the audit trail. This is the whole job.
End user / operatorReviews and approves agent actionsNot applicableReal review, not rubber-stamping; informed use; maintain tradecraft
05

Automation bias: human in the loop only counts if the review is real

The hardest engineering problem is not building the confirm step. It is keeping the human review real once the agent gets reliable. The literature here is settled and old. Bainbridge's "Ironies of Automation" (1983) showed that automating the easy parts leaves humans the monitoring task they perform worst. Parasuraman and Riley (1997) named overreliance as "misuse." Skitka, Mosier and Burdick (1999) showed that automation bias produces errors of both omission and commission. The paradox bites the SME specifically: the more reliable your agent, the worse your reviewer monitors it. That inverts the intuitive assumption that a high-performing agent is automatically a safer agent.

The regulators have noticed. EU AI Act Article 14(4)(b) (Regulation (EU) 2024/1689) requires human overseers of high-risk systems "to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)," and to be able to override or stop it. Even outside high-risk scope, this sets the direction of travel: a static approve button is not oversight. The Act has real consequences elsewhere too, with penalties for prohibited practices reaching up to EUR 35,000,000 or 7% of worldwide annual turnover under Article 99.

IMDA Dimension 2 gives the deployer two instruments to detect the failure: monitor the human override rate (a low rate may signal rubber-stamping) and response time (a very short median may signal review fatigue). Be honest in the design review about what those metrics can and cannot prove. Timing alone cannot prove non-review. A fast click can mean an expert, and a slow one can mean a distracted reviewer. Metrics flag the problem. Design prevents it. The defensible confirm step pairs the metrics with forced review delay proportional to blast radius, structured human-readable deltas instead of raw JSON, re-typing the transaction amount, and periodic blind second-reviews. Two architecture rules are non-negotiable: the checkpoint sits before execution (a post-hoc log is an audit note, not a control), and the approval path fails closed (deny the action by default when the approval infrastructure is down). IMDA names this last rule directly.

06

Where Origin Pi stands

Our view is direct. A confirm step is an accountability record and a commercial-liability firewall, not red tape. Its job is narrow and high-value: it stops the business being legally bound by a hallucination, and it produces the audit trail that regulators and your own lawyers will ask for when fault has to be apportioned across a value chain that otherwise pushes liability onto you.

We are deliberate about what it does not do. A confirm step does not cure an upstream data-accuracy violation, and it does not auto-satisfy AI Act Article 14(4)(b) or any compliance obligation. Anyone telling an SME that a checkbox "solves compliance" is creating misrepresentation exposure, not selling governance. The IMDA framework itself is voluntary guidance, not law, and we will not imply otherwise.

The control only earns its keep if the review is real. Rubber-stamping defeats the entire mechanism, which is why we treat override-rate and response-time monitoring, plus forced friction proportional to blast radius, as part of the control rather than decoration. In the agent-ready business layer, a deployer owns exactly three things the value chain will not own for it: the confirm step placed before execution, bounded least-privilege permissions that cap the blast radius at the token level (an agent cannot wire 10,000 if its key is scoped to 50), and an immutable audit trail that lets you apportion fault later. Those are not compliance theatre. They are the only levers that sit on your side of a liability split that otherwise leaves you holding the bag. Build them well, and the principal-agent analogy stops being a threat and becomes a defensible record of meaningful human accountability. That is the practical core of what we mean by AI governance and AI compliance for agentic systems.

Questions

Common questions.

Who is legally liable when an AI agent makes a mistake?
It depends on the facts and the jurisdiction, but the deployer (the business that points the agent at its customers) carries the obligation that cannot be disclaimed. In Singapore, PDPA section 4(3) keeps the engaging organisation accountable as if it processed the data itself, even when a vendor does the processing. In the EU, the revised Product Liability Directive (EU) 2024/2853 can push some exposure upstream onto model and system developers, but whether it lands there or on the deployer is an unresolved legal question, not a settled map.
Is there a dedicated EU law for AI liability?
No. The European Commission formally withdrew the EU AI Liability Directive, with the withdrawal published on 6 October 2025 (OJ reference C/2025/5423). The Commission's stated reason is that the revised Product Liability Directive plus the AI Act already provide a liability framework, making a separate fault-based AI regime unnecessary. SMEs are judged by existing product-liability and product-safety law, not a bespoke AI-fault regime.
Does a human-in-the-loop checkbox satisfy the AI Act or solve compliance?
No, and claiming it does is misrepresentation exposure. A confirm step stops the business being bound by a bad action and produces an audit trail, but it does not cure an upstream data-accuracy violation and does not automatically satisfy EU AI Act Article 14(4)(b). That article requires overseers to be aware of automation bias and able to override the system, which only counts if the human review is real rather than a rubber-stamp.
What is automation bias and why does it matter for agent oversight?
Automation bias is the human tendency to over-rely on automated output, producing errors of omission (missing a failure the system did not flag) and commission (following the system despite contrary evidence). The research is long established (Bainbridge 1983; Parasuraman and Riley 1997; Skitka, Mosier and Burdick 1999). The paradox is that the more reliable the agent, the worse the human monitors it, which is why a static approve button is not real oversight.
How do you tell if a confirm step is real review or just rubber-stamping?
IMDA Dimension 2 suggests monitoring the human override rate (a near-zero rate may signal rubber-stamping) and response time (a very short median may signal review fatigue). Timing alone cannot prove non-review, so metrics only flag the problem. Design prevents it: pair the metrics with forced review delay proportional to blast radius, human-readable deltas instead of raw JSON, re-typing the transaction amount, and periodic blind second-reviews. The checkpoint must sit before execution and fail closed.
What can an SME actually control in the agentic value chain?
Three things the rest of the value chain will not own for you: the confirm step placed before execution, bounded least-privilege permissions that cap the blast radius at the token level (an agent cannot wire 10,000 if its key is scoped to 50), and an immutable audit trail that lets you apportion fault later. These are the levers that sit on the deployer's side of a liability split that otherwise pools at the deployer.
Next

Work with Origin Pi.

Building the agent-ready layer for your business? Send a note. Real reply, no funnel.