An AI Failed 2% of Tests. What Does That Tell an Underwriter?
A 2% AI assessment failure rate does not mean a 2% probability of an insurance claim. For underwriters, the significance of a failure percentage depends on what was tested, how scenarios were selected, whether failures resulted in completed actions or blocked attempts, and how the observed vulnerabilities relate to the insured application's real-world exposure.
A specialist insurance broker submits an AI vulnerability assessment as part of a technology company's insurance renewal. The company operates a customer-facing AI application, and the assessment reports a 2% failure rate.
Is that a reassuring result or a reason for further investigation?
The 2% figure is entirely hypothetical. It is not an AetherLab assessment result, an industry benchmark or an observed insurance loss rate. For this illustration, it means that two attempts met predefined failure criteria out of 100 recorded test attempts. Those attempts are not assumed to be random, independent or representative of production activity.
A 2% AI assessment failure rate describes what happened within a defined set of tests. It does not establish a 2% probability of a production incident, financial loss or insurance claim. Underwriters need to understand the test denominator, sampling conditions, observed consequences and the insured application's real-world exposure before drawing broader conclusions.
The percentage is evidence about the assessment. Its underwriting significance depends on what sits behind it.
The First Question: 2% of What?
Suppose an AI application fails two out of 100 test attempts.
Before interpreting the result, an underwriter needs to establish what those attempts represent.
Were they 100 distinct operating scenarios? Repeated attempts against a small number of vulnerabilities? Different variations of the same attack? Or tests concentrated on one particularly consequential workflow?
These distinctions matter because the denominator determines what the percentage actually measures.
Consider two hypothetical assessments.
Assessment A: Broad Scenario Coverage
An assessor evaluates 100 different scenarios across several application workflows. Two scenarios produce failures.
This assessment may provide broader coverage of the application's operating behaviors, depending on how the scenarios were selected.
Assessment B: Repeated Attempts Against Selected Scenarios
An assessor selects two challenging scenarios and repeats each 50 times. Two of the 100 attempts produce failures.
This assessment provides evidence about the application's behavior under repeated attempts involving two specific conditions.
Both assessments report 2%, but they provide different information.
Neither percentage automatically establishes how frequently the application will fail during normal operations.
The unit of measurement also matters. A test case, an individual attempt and a completed workflow are not necessarily interchangeable.
If an assessor counts multiple attempts against the same scenario, those attempts may share conditions that influence the outcome. Treating them as independent observations without justification could overstate the precision of any statistical inference.
For an underwriter, the relevant question is not simply how many tests passed. It is what population of behaviors the tests examined and how far the results can reasonably be generalized.
Why Test Selection Changes the Result's Meaning
An AI vulnerability assessment may be designed to discover weaknesses rather than estimate average production performance.
An assessor might deliberately concentrate on conflicting instructions, manipulated external information, inappropriate tool-use attempts or situations where the application must enforce an operating restriction.
That approach is useful for identifying vulnerabilities.
But the resulting failure rate reflects the chosen testing conditions, not necessarily the frequency of those conditions in production.
Consider a customer-service AI application used by a financial technology company.
Most customer interactions may involve routine enquiries. The assessment, however, could focus on attempts to access restricted information or trigger unauthorized account-related actions.
If two out of 100 adversarial attempts produce failures, applying that 2% rate to every customer interaction would be unjustified.
The opposite problem is equally important.
An application might pass nearly every routine test while retaining a vulnerability in a rare but consequential workflow.
A low aggregate failure rate can therefore conceal a finding that deserves significant attention.
Underwriters should distinguish between assessments designed to measure representative operating performance and those designed to challenge defined behaviors under difficult conditions.
Neither approach is inherently better. They answer different questions.
The assessment's objective, scenario selection and sampling method determine what conclusions the percentage can support.
The Same 2% Can Describe Very Different Failures
The denominator explains how a failure rate was calculated. It does not explain the consequences of the failures.
Consider three hypothetical AI applications. Each reports two failures across 100 recorded test attempts, producing the same 2% failure rate.
Yet the observed outcomes are materially different.
Application A: An Incorrect Response
The first application produces an incorrect response to an internal information request.
The failure concerns response quality under the tested conditions. Depending on the application's purpose, the error may create operational inconvenience or require further investigation.
However, the assessment does not establish that the application accessed restricted information, attempted an unauthorized operation or caused a consequential loss.
The observed finding is an incorrect response.
Application B: A Restricted Action Blocked by an External Control
The second application attempts to invoke a restricted function after receiving manipulated instructions.
The connected system identifies that the required permission is missing and blocks the operation.
The assessment therefore establishes two separate observations.
First, the AI application attempted an action outside its permitted operating boundaries. Second, the external permission control prevented the action from completing under the tested conditions.
This is an application-level behavioral failure with an effective blocking control in the observed attempt.
It should not be described as a completed unauthorized action. Equally, the inappropriate attempt should not be ignored simply because the downstream control worked.
Application C: A Completed Unauthorized Disclosure
The third application retrieves restricted customer information and discloses it to an unauthorized requester within the test environment.
Unlike Application B, the prohibited outcome actually occurs during the assessment.
The finding establishes that restricted information was disclosed under the tested conditions.
That may have more consequential implications for an underwriter evaluating an application with access to sensitive customer records.
However, the controlled assessment does not automatically establish that an equivalent event would produce an insured loss in production.
What the Comparison Tells an Underwriter
All three applications report the same 2% failure rate.
But one involves an incorrect response, another an unauthorized attempt blocked by a functioning control, and the third a completed disclosure of restricted information.
The relevant distinction is between what the application attempted, what the controls permitted and what consequence occurred.
An attempted prohibited action is not equivalent to a completed unauthorized action.
Likewise, a completed harmful action during a controlled assessment does not automatically establish that an equivalent event would generate an insured loss in production.
For underwriting purposes, the percentage must be interpreted alongside the failure mechanism and its observed consequence.
Two failures involving blocked tool calls are not equivalent to two completed unauthorized disclosures, even when the reported failure rate is identical.
Exposure Determines Why a Finding Matters
A technical finding becomes more meaningful when it is considered alongside the insured application's operating environment.
Suppose two technology companies report the same assessment failure rate.
One uses AI internally to summarize non-sensitive documents. The other deploys a customer-facing AI application with access to confidential records and connected business systems.
The percentage alone cannot establish which presents greater insurance exposure.
The underwriter needs to understand what the application can do, who interacts with it and what could happen if the observed failure occurred outside the test environment.
Several factors affect that interpretation:
Deployment scale: Is the application used by a small internal team or thousands of external customers?
Operational authority: Does it provide information, recommend actions or interact with systems capable of consequential operations?
Relevant exposure frequency: How often does the application encounter conditions comparable to those that produced the finding?
Surrounding controls: What permissions, approvals, monitoring or other safeguards operate around the application?
Potential consequences: Could the failure lead to confidentiality breaches, customer harm, operational disruption or contractual liability?
These factors help establish the relevance of the technical finding to the proposed insurance exposure.
They do not independently establish the probability or financial severity of a loss.
A finding involving a rarely encountered condition may still warrant attention if the potential consequence is severe. Conversely, a frequently observed low-impact response error may have limited relevance to the particular coverage being considered.
Underwriting significance depends on the relationship between the observed behavior and the insured's actual exposure, not simply the aggregate failure percentage.
Why a 2% Test Failure Rate Is Not a 2% Claims Probability
The most important distinction is between technical failure frequency and insurance loss frequency.
A test failure is an observed outcome under defined assessment conditions.
An insurance claim depends on a broader sequence of events.
The relevant condition must arise in production. The application must exhibit behavior capable of causing harm. Surrounding controls may need to fail or prove insufficient. An actual loss or liability must result, and the circumstances must satisfy the applicable policy terms for a covered claim.
A reported test failure rate does not establish the probability of that sequence.
For example, suppose an AI application handles 100,000 customer interactions annually.
It would be misleading to multiply that volume by the hypothetical 2% assessment failure rate and conclude that the application will produce 2,000 harmful incidents.
That calculation assumes the tested conditions represent the application's production interactions, that the observed failure frequency transfers to those interactions and that every failure results in harm.
None of those assumptions follows from the percentage alone.
A defensible loss estimate would require additional information about production exposure, relevant incident frequency, control effectiveness and the severity of resulting losses.
It may also require assumptions about how technical incidents translate into liabilities and claims under the relevant insurance coverage.
An AI vulnerability assessment can identify failure mechanisms and demonstrate observed consequences. It cannot, by itself, supply every input needed to estimate insured losses.
What Insurance Research Tells Us About the Evidence Gap
The challenge of translating AI behavior into insurance exposure extends beyond individual assessments.
The Lloyd's Market Association's AI loss scenarios survey collected 144 responses in 2025, predominantly from underwriters, to explore views on potential AI-related losses across insurance lines.
The research examined professional judgments about hypothetical loss scenarios rather than observed AI claims frequencies. It illustrates why scenario-specific consequences are important when considering emerging AI exposures.
A 2026 research paper, Insuring AI: Incentivizing Safe and Secure Deployment, also examines challenges associated with limited historical loss data and the estimation of AI-related risk.
These sources do not provide a benchmark for acceptable AI assessment failure rates.
They reinforce a narrower point: technical evidence can inform underwriting, but translating it into financial loss estimates requires additional exposure information and assumptions.
That distinction becomes particularly important when a broker or underwriter is presented with a single headline percentage.
What Can an Underwriter Actually Conclude?
Returning to the hypothetical insurance renewal, the broker has submitted an assessment reporting two failures across 100 recorded attempts.
What can the underwriter reasonably conclude?
First, the assessment observed two outcomes that met its defined failure criteria under the tested conditions.
Second, the findings may identify particular application behaviors that require further investigation, mitigation or retesting.
Third, the observed consequences and surrounding controls may help establish which findings are more relevant to the insured's operations.
These are useful conclusions.
But the evidence does not automatically establish the application's production failure probability, expected number of harmful incidents or likely insurance claims.
The underwriter also needs to consider whether the assessed configuration matches the application currently being operated.
If the company has changed its model, instructions, connected tools or operating permissions since the assessment, the relevance of the original findings may need to be reconsidered.
The status of remediation matters as well.
An unresolved vulnerability, a vulnerability addressed through a documented mitigation and a finding successfully retested under defined conditions represent different evidence states.
A completed retest may support a conclusion about the tested mitigation, but it does not prove that the application is free of other vulnerabilities.
The appropriate underwriting conclusion should therefore identify what the assessment demonstrates, which assumptions remain necessary and what additional evidence may be required.
The percentage should inform that review, not replace it.
How This Changes the Broker's Submission
For a specialist broker placing or renewing coverage for an AI-deploying technology company, a headline such as "98% of AI safety tests passed" is unlikely to answer the underwriter's most important questions.
A more useful submission would explain the assessment result in context.
For example:
The application was assessed through 100 recorded adversarial attempts covering specified operating conditions. Two attempts met the assessment's failure criteria. Both involved inappropriate attempts to access restricted functionality, which were blocked by external controls. The findings relate to the tested application configuration and do not represent an estimated production incident rate.
This is a hypothetical illustration, not an actual assessment finding.
If the company has completed mitigation and retesting, the submission should explain the resulting evidence. If findings remain unresolved, their status should be clear.
The broker should also identify the application responsibilities and exposure relevant to the proposed coverage.
This allows the receiving underwriter to distinguish observed technical weaknesses from assumptions about production incidents and insured losses.
It also supports more focused follow-up questions.
Rather than asking whether 2% is an acceptable failure rate, the underwriter can ask whether the two observed failures affect consequential application behavior, whether controls prevented the relevant actions and what evidence exists that the identified weaknesses have been addressed.
That is a more useful basis for underwriting discussion.
Evidence Should Support Interpretation, Not Just Reporting
An assessment percentage becomes useful when the receiving underwriter can trace it back to the conditions and outcomes that produced it.
The evidence should make clear what was counted, how tests were selected, which behaviors failed and what consequences were observed.
It should also identify the assessed configuration, relevant control responses, remediation status and limitations on interpreting the results.
AetherLab's related article, What Should Underwriters Look for in an AI Assessment Report?, addresses the broader structure of an assessment report.
The purpose here is different.
Once the report is available, the underwriter must decide how much weight to place on its findings and which conclusions remain unsupported.
That requires interpreting the evidence in the context of the insured application rather than treating a failure percentage as a standalone risk score.
Where AetherLab Fits
AetherLab supports independent, scoped assessment of generative and agentic AI applications.
Through AdversarialScan, AetherLab can evaluate defined application-level behavioral vulnerabilities and record how the application responds under specified testing conditions.
The Evidence Pack organizes the agreed assessment scope, observed findings, relevant control responses, mitigation measures, retesting and remaining limitations.
For brokers and underwriters, this helps clarify what a reported failure percentage represents and which conclusions the available technical evidence can support.
Evidence Pack does not convert assessment failure rates into actuarially validated claims probabilities, determine insurability, set premiums or make coverage decisions.
Those responsibilities remain with the relevant insurance professionals, using technical findings alongside exposure information, policy terms, loss experience and other underwriting considerations.
Reviewing an AI assessment as part of an insurance submission or renewal? Discuss the relevant assessment scope with AetherLab to understand what the findings establish and where further evidence may be needed.
Explore AetherLab's Evidence Pack.
Frequently Asked Questions
Does a 2% AI assessment failure rate mean a 2% chance of an insurance claim?
No. The percentage describes failures within a defined set of tests. Estimating claim probability requires additional evidence about production exposure, incident frequency, controls, resulting losses and applicable insurance terms. A technical test failure rate is not a claims-frequency estimate.
Why does the denominator matter when interpreting an AI assessment?
The denominator determines what the reported percentage measures. One hundred unique scenarios and 100 repeated attempts across a small number of scenarios provide different evidence. Underwriters need to understand what was counted, how tests were selected and whether the results can support conclusions beyond the assessment.
Should an AI action blocked by an external control count as a failure?
It may count as a behavioral failure if the assessment criteria define an inappropriate attempted action as a failure. However, a blocked attempt should be distinguished from a completed unauthorized action. The two outcomes may have different operational consequences and underwriting implications.
Can a low AI test failure rate still indicate significant risk?
Yes. A low aggregate rate may contain a consequential vulnerability involving sensitive information, unauthorized actions or ineffective controls. The significance depends on the observed behavior, deployment context, potential consequences and remediation status.
Can an AI vulnerability assessment establish expected insurance losses?
Not independently. It can provide evidence about technical weaknesses, observed behaviors and control responses. Estimating insured losses requires additional assumptions and information about real-world exposure, incident frequency, loss severity and relevant policy coverage.
Related analysis
What Should Underwriters Look for in an AI Assessment Report?
What should insurance underwriters look for in an AI assessment report? Explore how to evaluate tested requirements, observed failures, control responses, remediation, retesting and remaining exposure when reviewing consequential AI applications.
AI Insurance: When Emerging Risks Outpace Claims Data
AI is rising to the top of insurers' emerging-risk concerns, but historical claims data offers limited visibility into autonomous AI liability. Explore how insurers and AI-deploying companies can evaluate emerging exposures through technical risk assessments.