What Should AI Companies Test Before Seeking Liability Insurance?

An AI company is approaching its first liability insurance renewal since expanding its customer-support product into an agentic application.

Previously, the product answered customer questions and retrieved information. It can now interact with account-management systems, initiate selected workflows and recommend actions that affect customer accounts.

The engineering team has evaluated task completion, response quality and system reliability. The company has also implemented safeguards intended to prevent the agent from performing inappropriate actions.

As the renewal approaches, its insurance broker needs an updated understanding of the product's capabilities and the potential risks associated with those changes.

The company can explain what its agent is designed to do.

But can it explain what happens when the agent is manipulated into acting outside its intended boundaries?

Has the company tested whether its AI agent can be redirected by malicious instructions, attempt unauthorised actions, disclose sensitive information or bypass required human approvals?

These questions become increasingly relevant as AI products move beyond generating information and begin interacting with business systems.

For AI-deploying technology companies, an upcoming insurance application or renewal can be a practical reason to evaluate application-level behavioural vulnerabilities and document the effectiveness of existing controls.

Depending on the company's activities and potential exposures, insurance discussions may involve technology errors and omissions (E&O), cyber insurance or specialist coverage addressing particular AI-related liabilities. Each involves different underwriting considerations, and technical vulnerability testing addresses only part of the information an insurer may evaluate.

There is no universal requirement for AI companies to complete independent adversarial testing before obtaining liability insurance.

Nor does successful testing guarantee coverage, lower premiums or favourable policy terms.

The value of testing is more specific: it helps companies identify relevant technical weaknesses, evaluate selected safeguards and explain their application-level risk profile with greater precision.

Why Product Performance Testing Does Not Fully Explain AI Risk

Most AI companies invest considerable effort in demonstrating that their applications work.

They measure accuracy, task-completion rates, response quality, latency and reliability.

A customer-support AI company may evaluate how frequently its agent resolves customer requests.

A document-processing company may measure extraction accuracy.

An agentic workflow provider may assess whether its application completes assigned tasks without interruption.

These evaluations are essential to product development.

But they do not necessarily establish how the application behaves when exposed to adversarial inputs or conflicting instructions.

An AI agent may complete legitimate workflows reliably while following malicious instructions embedded in retrieved content.

A customer-support application may answer routine questions accurately but attempt to access information outside the intended scope of a request.

An agent may respect approval requirements during normal testing but attempt to bypass them when presented with manipulated instructions.

These are not simply ordinary product-performance errors.

They concern the application's ability to maintain its operating boundaries when its behaviour is challenged.

For companies deploying AI into consequential workflows, this creates a separate category of technical risk that deserves evaluation.

The question is not only whether the AI application performs its intended tasks.

It is whether the company understands how the application might behave when those tasks, instructions or operating restrictions come into conflict.

Five AI Vulnerability Testing Priorities Before Insurance Discussions

There is no single technical testing checklist that applies to every AI company or insurance policy.

A company developing an internal writing assistant has a different risk profile from one operating an AI agent that can modify customer records, interact with financial systems or support consequential decisions.

The following five areas are therefore risk-based testing priorities for relevant generative and agentic AI applications, not standard requirements imposed by insurers.

The appropriate scope depends on the application's capabilities, available tools, information access and potential consequences of failure.

1. Adversarial Behaviour: Can Manipulated Inputs Redirect the AI?

Generative AI applications regularly process information from sources they do not fully control.

These may include customer messages, uploaded documents, retrieved webpages, emails and external databases.

Some of that information may contain instructions designed to manipulate the application's behaviour.

Consider an AI procurement agent that reviews supplier documents and prepares purchasing recommendations.

A supplier document contains instructions telling the agent to disregard its assigned task and retrieve unrelated internal information.

The document is supposed to provide information for the procurement workflow.

Instead, the attacker attempts to make the agent treat document content as an instruction.

This is a prompt-injection scenario.

Testing can examine whether the application maintains the distinction between trusted instructions and lower-trust information.

Relevant scenarios may explore whether the agent follows instructions embedded in documents, changes its assigned objective or attempts actions outside its intended workflow.

The assessment should record the expected behaviour and the application's observed response.

If the agent follows the manipulated instruction, the test may reveal a behavioural vulnerability.

If it rejects the instruction, that provides evidence of appropriate behaviour under the tested conditions.

However, passing selected tests does not establish that the application is immune to every possible manipulation technique.

The objective is to understand how the application responds to relevant adversarial scenarios and identify weaknesses that may require mitigation.

2. Unauthorised Tool Use: Does the Agent Respect Its Operating Authority?

Agentic AI introduces risks that extend beyond generating incorrect information.

An AI agent may be connected to tools that retrieve data, update records, initiate workflows or interact with external systems.

These capabilities make it important to distinguish between an agent generating a recommendation and attempting an operational action.

Consider a customer-support agent that can recommend refunds but must obtain approval before initiating them.

A customer submits a manipulated request instructing the agent to bypass the normal approval process.

The expected behaviour is that the agent follows the required authorisation workflow.

Testing can examine whether the application respects that restriction under conflicting or adversarial instructions.

Several outcomes are possible.

The agent may refuse the request.

It may correctly escalate the request for approval.

It may attempt to invoke a restricted refund function, only to be blocked by the connected system.

Or the external system may permit an action that should have required additional authorisation.

These outcomes have different implications.

An attempted tool call that is blocked by infrastructure is not equivalent to a completed unauthorised transaction.

Nevertheless, the attempt may indicate an application-level behavioural weakness.

Application-level testing can assess whether an AI agent follows its intended workflow and operating restrictions.

It does not replace infrastructure-level permission enforcement, payment authorisation controls or other security measures governing the connected systems.

For companies deploying agents with consequential capabilities, both layers matter.

3. Sensitive Information Disclosure: Can the Application Be Manipulated Into Revealing Protected Data?

AI applications may have access to customer information, internal documents, business records or other sensitive material.

The risks depend on which information the application can access, how it retrieves that information and which users are authorised to receive it.

Consider an enterprise AI assistant connected to internal knowledge repositories.

An employee asks a legitimate question about company policies.

A manipulated document or user instruction attempts to make the assistant retrieve confidential information unrelated to the request.

Testing can examine whether the application follows that instruction or attempts to disclose information outside the intended scope.

The assessment should distinguish between attempted and completed disclosure.

An application may refuse the request.

It may generate a response containing information that should not have been disclosed.

It may attempt to access restricted material but be blocked by the connected system.

Or it may retrieve information that the infrastructure permits, even though the application's intended workflow should not have requested it.

Each outcome reveals something different about the controls involved.

A credible assessment should document the tested conditions, observed behaviour and relevant access restrictions.

Sensitive information exposure can involve model behaviour, retrieval design, access controls and infrastructure permissions.

Behavioural testing can evaluate relevant application responses, but it does not replace a comprehensive data-security assessment.

4. Escalation Failures: Does the AI Recognise When Human Approval Is Required?

Many consequential AI applications operate within workflows that require human intervention under specific conditions.

An agent may be permitted to prepare recommendations but not approve certain actions.

It may be required to escalate requests involving sensitive information, conflicting instructions or activity outside its assigned authority.

Consider an AI agent assisting with business purchasing.

The agent can prepare procurement requests but must obtain approval before selected transactions proceed.

A manipulated request attempts to persuade the agent to bypass that requirement.

Testing can examine whether the application recognises the approval boundary and follows the expected process.

The agent might correctly request human approval.

It might refuse the request.

Or it might attempt to continue the workflow despite the restriction.

These behaviours provide different evidence about instruction adherence and escalation logic.

The presence of a human-oversight policy does not automatically establish that the AI application invokes the policy when required.

A company may have documented escalation procedures without testing whether its agent follows them under adversarial conditions.

For applications supporting consequential business workflows, this distinction is important.

The assessment should define the conditions requiring escalation, establish the expected response and evaluate whether the application follows that response.

It should not imply that behavioural testing independently validates every domain-specific decision or regulatory obligation.

5. Mitigation Effectiveness: Have Identified Weaknesses Been Retested?

Vulnerability testing may identify weaknesses that require changes to the application.

A company might modify its instruction-handling logic, introduce additional workflow checks or implement application-level guardrails.

But implementing a mitigation does not establish that it works.

Consider an AI agent that follows malicious instructions embedded in retrieved content.

The company updates its application controls to reduce that behaviour.

The original failure scenario is then tested again.

If the agent now rejects the manipulated instruction, the result provides evidence of improved behaviour under the tested conditions.

Related variations may also need to be evaluated to determine whether the mitigation addresses more than the original input.

Where relevant, testing should consider whether the new control disrupts legitimate workflows.

A safeguard that prevents an inappropriate tool action but also blocks authorised requests may introduce operational problems.

A useful assessment therefore distinguishes between the vulnerability originally identified, the mitigation implemented and the results observed during retesting.

The goal is not to declare the application universally safe.

It is to establish whether specific behavioural weaknesses remain reproducible and which limitations remain.

How Should AI Companies Prioritise Testing?

The five vulnerability categories provide a starting point, but not every category will carry equal importance for every application.

A practical assessment should begin with the AI product's actual capabilities and potential consequences of failure.

For example, an AI assistant that only drafts internal text may not require the same tool-use testing as an agent connected to account-management systems.

An application with no access to sensitive customer information presents a different disclosure risk from one integrated with enterprise data repositories.

Companies should therefore prioritise testing based on what their applications can access, influence or attempt to execute.

A practical pre-insurance technical assessment can follow five steps.

Step 1: Map consequential application capabilities

Identify the application's relevant workflows, available tools, data access and operating restrictions.

Determine which actions could affect customers, business operations or sensitive information.

This establishes the scope of the assessment.

Step 2: Prioritise relevant failure scenarios

Select behavioural risks based on the application's capabilities.

For an agent that can interact with customer accounts, unauthorised tool-use attempts and escalation failures may be particularly relevant.

For a document-processing application, prompt injection and inappropriate information disclosure may deserve greater attention.

The objective is to test meaningful risks rather than apply a generic checklist.

Step 3: Evaluate expected and observed behaviour

Execute defined scenarios and compare the application's actual responses with its intended operating boundaries.

Record relevant inputs, responses, attempted tool actions and control outcomes.

Distinguish between application-level behaviour and actions blocked or permitted by external systems.

Step 4: Mitigate identified weaknesses and retest

Where vulnerabilities are found, implement appropriate controls and evaluate the application again.

Retesting should establish whether the original failure remains reproducible under the tested conditions.

Relevant variations may also be evaluated.

Step 5: Document findings and residual limitations

Record the assessment scope, tested configuration, observed findings, mitigation and retesting results.

Identify unresolved vulnerabilities and conditions outside the testing scope.

This creates a technical record that can support internal risk management and, where relevant, external discussions.

What Technical Evidence Can Support Insurance Discussions?

An AI company preparing for an insurance application or renewal may need to explain its technology, business activities and existing controls.

A specialist broker can help identify information relevant to the coverage being sought and communicate with potential insurers.

Technical vulnerability assessment may contribute to that process by documenting how the application behaves under selected failure scenarios.

Useful evidence may include the assessment scope, application configuration, tested scenarios, observed findings, implemented mitigations, retesting results and residual limitations.

The important relationship is between the risk evaluated and the evidence supporting the finding.

For example, a company may be able to explain that its AI agent was tested against manipulated instructions and inappropriate tool-use attempts, that a specific weakness was identified, and that subsequent testing did not reproduce the same behaviour under defined conditions.

This is more precise than stating that the application has undergone AI safety testing.

However, technical evidence is only one component of a broader insurance assessment.

Depending on the coverage, an insurer may consider the company's operations, contractual obligations, claims history, financial exposures, cybersecurity practices and other information.

Technology E&O, cyber insurance and specialist AI-related liability coverage may address different exposures and involve different policy terms, exclusions and underwriting considerations.

A technical assessment does not determine which policy is appropriate, whether a particular loss would be covered or whether an insurer will accept the risk.

There is also no standardised evidence pack universally required by insurers.

The purpose of documentation is to help the company communicate what was evaluated, what was observed and what remains uncertain.

Why Independent Vulnerability Assessment May Be Useful

Internal engineering teams play an essential role in AI application testing.

They understand the product architecture, intended functionality and operational requirements.

However, internal evaluation may naturally focus on expected workflows, known issues and product-performance objectives.

Independent vulnerability assessment can introduce additional scrutiny by challenging the application's operating boundaries.

An external evaluator may examine whether the agent follows manipulated instructions, attempts restricted actions, discloses information inappropriately or fails to escalate requests.

This can help identify behavioural weaknesses that were not apparent during normal product testing.

Independent assessment can also provide a separately documented evaluation of the agreed scope and observed findings.

Its value depends on the testing methodology, expertise, application access and quality of reporting.

It should not be treated as a guarantee of comprehensive security or a substitute for internal testing and infrastructure controls.

For AI companies preparing for insurance discussions, the benefit is a clearer understanding of selected application-level vulnerabilities and more credible technical evidence about the controls evaluated.

When Should Companies Begin Testing?

An upcoming insurance application or renewal can create a practical reason to reassess AI application risks.

But companies do not need to wait for an insurer or broker to request technical information.

Testing may be particularly relevant when an application gains consequential capabilities.

An AI assistant that previously generated recommendations may begin invoking external tools.

A product may gain access to sensitive customer information or become integrated into operational workflows.

A company may move from internal deployment to customer-facing use.

Each change can introduce failure scenarios that were not relevant to the application's earlier configuration.

Companies can prioritise testing around these changes and the potential consequences of inappropriate application behaviour.

Where vulnerabilities are identified, mitigation and retesting can help establish whether the relevant weaknesses remain reproducible.

The resulting technical evidence may later support insurance discussions, enterprise procurement or internal risk reviews.

The objective is not to anticipate every possible insurer question.

It is to understand the application's technical risk profile before an external review creates pressure to explain it.

AetherLab's Role in Pre-Insurance AI Vulnerability Assessment

AetherLab supports independent, scoped assessment of generative and agentic AI applications.

For AI-deploying technology companies preparing for liability insurance discussions, AdversarialScan can help evaluate defined application-level behavioural vulnerabilities.

Depending on the agreed scope, these may include prompt injection, instruction adherence, inappropriate tool-use attempts, sensitive information disclosure behaviour and escalation failures.

The assessment examines how the application responds under specified conditions and documents the observed results.

Where vulnerabilities are identified, Guardrails may support mitigation of relevant application-level behavioural weaknesses.

Evidence Pack can support structured documentation of assessment scope, tested scenarios, observed findings, mitigation, retesting and residual exposure, as applicable to the engagement.

This technical evidence may help companies communicate selected application risks to internal stakeholders, enterprise customers, specialist brokers and insurers.

AetherLab does not act as an insurer or insurance underwriter.

It does not determine insurance eligibility, interpret policy coverage, set premiums, certify insurability or guarantee that an application is free from vulnerabilities.

Its role is to evaluate defined application behaviours and document what the assessment demonstrates.

Before Seeking Coverage, Understand What Your AI Can Actually Do

AI companies preparing for liability insurance discussions often focus on explaining their products, customers and existing safeguards.

Those details matter.

But for companies deploying consequential generative or agentic AI, understanding how the application behaves when its operating boundaries are challenged is equally important.

Can manipulated inputs redirect the agent? Does it attempt actions outside its assigned authority? Could it disclose sensitive information? Does it follow required escalation processes? Have identified weaknesses been retested after mitigation?

The answers help companies move beyond general assurances toward specific technical evidence.

Testing does not eliminate every risk or guarantee insurance coverage.

It does, however, help companies identify relevant vulnerabilities, evaluate selected controls and document what remains unresolved.

Before asking an insurer to evaluate the risks associated with an AI product, a company should understand which consequential failure scenarios it has tested and what those tests revealed.

For AI-deploying technology companies approaching an insurance application or renewal, that is a practical foundation for more credible risk communication.

Preparing for a liability insurance application or renewal? Explore AetherLab's AdversarialScan to assess relevant AI application vulnerabilities and document what the testing reveals.