Trust Is Not a Property of Technology. It Is a Property of Infrastructure.

There is a question that gets asked at the beginning of every technological era that is genuinely new. Not "does it work?", that is an engineering question, and engineers answer it relatively quickly. The harder question is "can we trust it?" And that question, it turns out, is never answered by the technology itself.

It is answered by the infrastructure that grows up around it.

This is not an obvious point. Our instinct is to look for trustworthiness in the object, in the quality of its construction, the soundness of its design, the credentials of its creators. And those things matter. But they are not what actually produces the trust that allows a technology to scale from a curiosity to critical infrastructure. What produces that trust is a different kind of institution entirely: one that studies failure, prices risk, establishes standards, and stands behind its assessments with its own capital.

We are at the beginning of that moment for autonomous AI. And understanding what it actually requires means understanding what it has required before.

The pattern that keeps repeating

In the middle of the nineteenth century, steam engines were transforming industrial production. Factories, ships, railways, the energy source that would drive the industrial revolution was a pressurized boiler sitting at the center of every major enterprise.

It was also killing people at an alarming rate. Boiler explosions were common, unpredictable, and catastrophic. In the United States alone, thousands of deaths were attributed to boiler failures in the decades before 1870. The worst single incident, the explosion of the steamship Sultana in 1865, killed approximately 1,800 people.

The technology worked. The technology was also uninsurable, because the risk was unquantifiable. Nobody had established what a safe boiler looked like, or how to tell the difference between one that would hold and one that would give way.

The answer was not better steam engines. The answer was the Hartford Steam Boiler Inspection and Insurance Company, founded in 1866. HSB did something that seems simple in retrospect but was genuinely new at the time: it sent its own engineers to inspect the boilers it insured. It tied the availability and price of insurance directly to the safety of the equipment. It created the standards that defined what "safe" meant, not because a government mandated them, but because HSB had to pay when the boilers exploded. Skin in the game produced the standards that regulation had not.

Within a generation, the boiler explosion rate collapsed. Not because the technology improved in isolation. Because the confidence infrastructure grew up around it.

The same pattern repeated with the automobile. When cars first appeared on roads designed for horses, the risk they presented was genuine but uncharacterized. In 1897, Travelers Insurance wrote the first automobile liability policy, for Gilbert J. Loomis, at a premium of $7.50 for $1,000 of coverage. By the 1920s, insurers were developing risk models that distinguished between farmers and city drivers, between careful operators and reckless ones. By 1944, the state of Michigan was mandating coverage as a condition of licensure, making insurance not just a financial product but a legal prerequisite for operating the technology.

The technology didn't become trustworthy because cars got better. Cars got better partly because insurance priced risk in ways that incentivized safer design. The confidence infrastructure and the technology co-evolved.

Commercial aviation scaled in a similar way, and the mechanism was particularly instructive. In the early days of flight, the risks were so severe and so poorly understood that no single insurer could carry the exposure on its own balance sheet. The Allianz wrote its first airship policy in 1915. By 1920, it had helped establish the Deutscher Luftpool, a consortium that aggregated capital across multiple insurers, making it possible to cover aircraft that no single company could afford to protect. That pooled financial capacity underwrote historic flights, absorbed catastrophic failures like the Hindenburg, and ultimately created the actuarial foundation for the global passenger aviation industry.

The lesson from each case is the same. A transformative technology generates a category of risk that existing frameworks cannot price. A new kind of institution emerges, one that studies the failure modes, establishes what safe behavior looks like, and stands behind its assessments financially. That institution doesn't just transfer risk. It makes risk legible. And legible risk is the precondition for scale.

What insurance actually does

It is worth being precise about what the insurance industry contributes in these moments, because it is not what most people assume.

The naive version is that insurance is a financial buffer, you pay premiums, the insurer pays claims, the risk is transferred. That is true at the transaction level. But at the ecosystem level, insurance does something more important: it forces the production of information.

To price a boiler explosion, you need to know what makes boilers explode. To price a car accident, you need to know what makes drivers dangerous. To price an aviation catastrophe, you need to know what makes aircraft fail. The insurer who prices wrong pays the difference, which means insurers are more motivated than almost anyone to understand the true distribution of risk. Their survival depends on it.

This motivation produces standards. It produces inspection regimes. It produces the behavioral data, claims history, incident reports, near-miss documentation, that allows risk models to become more accurate over time. Insurance doesn't just transfer risk. It generates the information infrastructure that makes future risk assessable.

The same logic applies here. It is not Toyota that does the car crash testing, it is independent bodies. The entity with financial exposure to the outcome has the strongest incentive to understand it accurately. And that understanding, once formalized, becomes the standard that the rest of the market operates against. Independent certification works precisely because the certifier bears the reputational and financial cost of being wrong.

This is why the emergence of AI agent insurance is not primarily interesting as a financial story. It is interesting as an information story. The market for AI agent risk is forcing the production of the information infrastructure that will make autonomous systems deployable at scale.

Why autonomous AI is harder

Every historical analogy has limits, and the autonomous AI case has a specific property that makes it more complex than steam, cars, or planes.

The previous technologies were deterministic. A boiler built to a certain specification behaved predictably within that specification. A car driven in a certain way produced predictable risk profiles. An aircraft maintained according to approved procedures had a calculable failure rate. The risk was real, but it was in principle quantifiable, because the system's behavior could be characterized once you understood its construction and operation.

Autonomous AI agents are not deterministic. They are probabilistic systems operating on dynamic inputs, producing outputs that cannot be predicted by examining their construction alone. The same agent, given the same initial configuration, may behave differently as its context changes, as the model it runs on is updated, as the data it retrieves shifts, as new attack patterns emerge that its designers did not anticipate. The behavior of the system at certification time may diverge significantly from its behavior six months later, not because of any single identifiable change, but because of the accumulated effect of a hundred small ones.

This is what the research on AI governance calls the "lumping problem" in the context of the new EU Product Liability Directive, the difficulty of applying a liability framework designed for static products to systems that learn, adapt, and drift. A steam engine inspected in 1867 was the same steam engine in 1868. An AI agent certified in February may be a different risk profile in August. The inspection tells you what the system was. It does not tell you what the system is.

This is not a reason to conclude that autonomous AI cannot be governed or insured. It is a reason to conclude that governing and insuring it requires something the previous cases did not: a continuous behavioral signal. Not a snapshot. Not a certification report. A real-time, ongoing record of how the system is actually behaving, the equivalent of the telematics device that modern usage-based insurance programs use to track actual driving behavior rather than relying solely on the pre-purchase inspection.

The ecosystem being built

The confidence infrastructure for autonomous AI is under construction. The outlines are becoming visible.

Standards bodies like AIUC are building the certification layer, the equivalent of UL Labs for AI agents. Their AIUC-1 framework already covers 51 requirements across six domains, is updated quarterly as the threat landscape evolves, and has produced its first insured deployment in ElevenLabs' voice agent platform. The SOC 2 analogy is apt: AIUC-1 gives enterprise procurement teams a concrete, auditable signal they can act on.

Major insurers, through Lloyd's syndicates and specialist vehicles like Munich Re's aiSure platform and HSB's AI Liability products, are beginning to write policies. The coverage structures are evolving from traditional liability models toward parametric approaches: if the agent's measured performance falls below a defined threshold, the payment is triggered automatically, avoiding the causal complexity that makes AI incident litigation so difficult.

Regulators are clarifying the legal architecture. The EU AI Act establishes what high-risk systems must document. The Product Liability Directive 2024 extends strict liability to AI software. DORA mandates independent audit trails for financial sector AI. These frameworks do not make AI agents safe. But they establish the legal obligations that create demand for the confidence infrastructure.

What is still missing, and this is the frontier that the ecosystem will need to solve, is the continuous behavioral data layer that sits underneath all of these structures. The layer that produces, in real time, the tamper-evident record of what agents are actually doing between certification cycles. The layer that allows insurance pricing to be dynamic rather than static. The layer that allows liability adjudication to rest on evidence rather than inference.

The deeper question

The question "can we trust autonomous AI agents?" is not, ultimately, a question about the agents.

It is a question about whether we can build the infrastructure that makes trust possible, the way Hartford Steam Boiler made steam engines trustworthy, the way Lloyd's made aviation insurable at scale, the way the entire apparatus of standards, inspection, certification, and insurance made it possible for technologies that were genuinely dangerous to become genuinely indispensable.

The historical record suggests we can. It also suggests that building it is not automatic, not fast, and not the work of any single institution. It requires standards bodies willing to establish what safe looks like. Insurers willing to price the residual risk. Infrastructure providers willing to produce the continuous behavioral signal that makes both possible. Regulators willing to enforce the accountability frameworks that give all of the above legal force.

The agents are ready. The question is whether the confidence infrastructure will be ready with them.