AI Tools Misleading Insurance Complainants, Ombudsman Warns

New Zealand’s Insurance & Financial Services Ombudsman Scheme has issued a warning about the increasing use of artificial intelligence tools by consumers filing insurance complaints, citing cases of misleading advice and hallucinated information.

Karen Stevens, Insurance & Financial Services Ombudsman, reports that AI systems are providing consumers with unrealistic expectations about complaint outcomes. In one notable instance, a Google AI summary incorrectly suggested that “insurance claim decisions are frequently overturned” and that “up to 80-90% of cases can result in success if people persist.”

“These kinds of claims are misleading,” Stevens stated. “While many complaints are resolved quickly and directly by insurers, sometimes just by a simple explanation, complaints which escalate are often complex and stressful, and not always able to be resolved in the consumer’s favour.”

The problem stems from AI’s tendency to oversimplify complex insurance policy wording and miss crucial exclusions. Often, these AI systems rely on overseas information that doesn’t apply to New Zealand’s regulatory environment. More concerning is the IFSO’s observation of AI hallucinating previous cases—inventing non-existent precedents and using them as examples to support misleading advice.

“Many people come to us hoping we can put things right, but unfortunately if the law or the contract doesn’t support their position, we can’t change that – even when the outcome feels unfair,” Stevens explained.

The IFSO Scheme has also observed consumers using AI to draft complaints, which while helpful for those with language barriers, can sometimes obscure the core issues. “We’ve seen complaints which are 300 pages long,” Stevens noted. “But more words aren’t necessarily better. Clear information about what has gone wrong for someone is much more useful than multiple pages referencing legislation and case law.”

This trend reflects the broader adoption of AI assistants across various consumer advocacy contexts. As AI tools become more accessible, they are increasingly being used by consumers to navigate complex institutional processes like insurance claims and complaints. However, the financial services sector presents unique challenges for AI tools due to its specialized terminology, jurisdiction-specific regulations, and contractual nuances.

Another significant concern raised by the Ombudsman relates to privacy and data security. Stevens cautioned that consumers should be wary of inputting personal or confidential information into some AI tools, particularly free ones, as “that information may be stored or reused in ways consumers don’t expect.”

The issue highlights the growing tension between technological accessibility and accuracy in consumer advocacy. While AI tools can democratize access to assistance, they may simultaneously create new barriers through misinformation or unrealistic expectations.

For consumers considering using AI to assist with insurance complaints, Stevens provided several practical recommendations:

• First review your full policy or contract and consult directly with your provider about how it applies to your specific situation
• Verify any AI-generated information against trusted, New Zealand-based sources
• Exercise caution when sharing personal information with AI tools
• When using AI to draft complaints, ensure the tool clearly and simply communicates the core issues

“AI can be very useful, but it’s important to use it with caution,” Stevens concluded.

The IFSO’s warning comes at a time when regulatory bodies worldwide are grappling with how to address the growing influence of AI tools in consumer-facing financial services. As these technologies continue to evolve, the gap between what AI systems promise and what legal and contractual frameworks allow may present ongoing challenges for ombudsman services and regulatory authorities.