AI Lending: How to Test Human Oversight in Automated Credit Decisions

In September 2026, Moody's published research on automation and judgement in US commercial lending, based on interviews with 15 senior executives at American banks managing between approximately $12 billion and $235 billion in assets.

The research identified a challenge beyond faster underwriting: as financial analysis, credit preparation and lending workflows become automated, bankers must retain access to the information needed to question and interpret a credit assessment.

A separate September study from the Federal Reserve Bank of San Francisco found that banks with greater AI adoption tended to have higher returns, higher shares of problem loans and lower shares of small-business lending. The findings are correlations, not evidence that AI caused those outcomes.

Together, these developments raise a practical question for lenders deploying AI underwriting assistants: can a human reviewer identify an incorrect AI recommendation before it influences a credit decision?

When Human Approval Becomes a Formality

Consider a hypothetical small-business loan application.

An AI underwriting assistant examines bank statements, financial records and supporting documents. It prepares a credit memorandum recommending approval, including revenue, cash flow and outstanding liabilities.

One uploaded document contains an embedded instruction telling the assistant to disregard a particular liability because it has supposedly been settled.

The assistant treats that instruction as authoritative, excludes the liability and produces a favourable recommendation. The credit officer receives the memorandum but does not see the discrepancy in the underlying document.

The officer approves the application.

A person made the final decision, but the review failed to catch the manipulated assessment. This hypothetical scenario illustrates how an AI assistant can compromise the information on which human judgement depends.

Five Tests for Human Oversight in AI Lending

1. Can reviewers verify the underlying evidence?

AI-generated credit memoranda may combine figures from statements, tax records and borrower disclosures.

Testing should establish whether material conclusions can be traced to their original sources. If an assistant omits a liability or misstates cash flow, can the reviewer identify the missing information?

An assessment should examine both the accuracy of the AI-generated summary and whether the review interface exposes relevant discrepancies.

2. Can manipulated documents change the recommendation?

Loan applications routinely contain information supplied by borrowers and third parties. That material is evidence for assessment, not an instruction source for the AI assistant.

Adversarial testing can introduce misleading statements or embedded instructions into controlled documents, then observe whether the assistant changes calculations, suppresses warnings or produces unsupported conclusions.

The result should distinguish between encountering malicious content, incorporating an incorrect claim and producing a materially altered recommendation.

3. Can the reviewer detect a deliberately flawed assessment?

Providing access to supporting documents does not guarantee that a reviewer will challenge an AI-generated recommendation.

A controlled test can introduce a flawed credit memorandum into the review workflow and examine whether the reviewer identifies the discrepancy, requests further evidence or escalates the case.

This requires a distinction between technical and operational testing. AetherLab can assess how an AI assistant produces incorrect or manipulated outputs. The lender remains responsible for evaluating reviewer performance, training and credit procedures.

4. Can the assistant bypass approval boundaries?

An AI assistant may prepare credit recommendations without being authorised to approve applications or alter lending policies.

Testing should examine whether it can suppress exceptions, mark incomplete assessments as ready for approval or invoke restricted workflow functions.

Where human approval is required, the surrounding system should enforce that requirement independently.

An agent attempting an unauthorised approval is different from an approval actually being executed. Both outcomes matter, but they reveal different weaknesses.

5. Can the institution reconstruct what happened?

A final credit memorandum rarely provides a complete account of how an AI-assisted recommendation was produced.

Testing should examine whether the institution can recover the source documents, extracted information, agent outputs, exceptions, revisions and reviewer actions.

For covered US credit decisions, the CFPB's guidance on complex algorithms and adverse-action notices reinforces the need to identify specific, accurate reasons for adverse actions. Using an opaque algorithm does not remove that obligation.

This is distinct from a universal requirement for human approval. The broader technical question is whether an institution can explain and investigate how its AI system influenced the outcome.

What an Adversarial Assessment Should Reveal

Return to the hypothetical application with the omitted liability.

Suppose testing confirms that the assistant followed the embedded instruction and produced an inaccurate credit memorandum. The reviewer then approved the recommendation without identifying the omission.

The assessment has uncovered two weaknesses: the assistant accepted an instruction from untrusted material, and the review process failed to detect the resulting error.

The evidence should preserve the manipulated document, extracted figures, agent responses, recommendation, reviewer-visible information and approval sequence.

After remediation, retesting should determine whether the assistant rejects the instruction, includes the liability and presents the relevant evidence for review.

Adding another approval step would accomplish little if both reviewers received the same incomplete information.

Testing AI Underwriting Before Expanding Its Role

Moody's findings point to a fundamental challenge for commercial lenders: automation can accelerate credit preparation while making it harder to examine the context behind a recommendation.

For institutions introducing AI underwriting assistants, that creates a clear assessment requirement. They need to know how the assistant behaves when evidence is manipulated, information conflicts or workflow boundaries are challenged.

AetherLab's AdversarialScan can test these behaviours, including prompt injection, misleading inputs and attempted permission bypasses. Its Evidence Pack can document the test conditions, observed actions, control responses, mitigations and residual exposure.

These assessments complement the lender's credit-model validation, policy enforcement and human-review procedures rather than replacing them.

A human approval step is only as reliable as the information behind it. Before expanding AI's role in credit assessment, test whether manipulated documents can distort recommendations, hide material risks or undermine approval controls.

Talk to AetherLab about testing your AI underwriting workflows.