What if AI makes an unfair or unsafe decision? Can you turn to your insurers for coverage?

Artificial Intelligence (AI) is transforming how financial institutions operate and interact with their customers.

As opportunities arise for organisations to incorporate AI into their products, services, processes, and decision-making, it is important for organisations to understand the risks associated with AI and whether their current insurance arrangements provide coverage for those risks.

Traditional lines of insurance coverage have been underwritten with the risks associated with human judgement in mind. In this article we ask whether traditional insurance policies will respond when a machine makes an unfair or unsafe decision without human oversight.

Before jumping into this issue, it is important to point out that it is well-established law that financial institutions are legally responsible for the services provided to customers, i.e., responsibility cannot be outsourced to a third-party service provider or, for that matter, an AI model. Further, our below comments are focused on Generative AI rather than Traditional AI. While Traditional AI is designed to analyse data and respond to a particular set of inputs, e.g., Siri or Alexa, Generative AI learns from data and can create new outputs from data, e.g., Chat GPT.

The potential risks arising out of the use of Generative AI are endless. Some examples include the creation of incorrect legal contracts for customers; the misappropriation of personal customer data; the amplification of systemic risks through algorithmic trading; the unauthorised use of copyrighted code or content; and board liability for decisions that are primarily based on AI. Below we review a few risks and how current insurance policies may respond to those risks.

(i) Explainability

As Generative AI systems are by their very nature deep-learning models, they often lack complete explainability, which means they are unable to explain the reasons for a decision or how a decision was arrived at. Lack of explainability poses a significant risk for financial institutions, as decisions involving such things as loan applications or insurance claim payments have the potential to significantly impact customers’ lives. With human-made decisions, loan officers or claims handlers can explain why certain decisions were made - even if those explanations are unreasonable, the existence of an explanation in and of itself enables the ability to correct or defend the decision.

With Generative AI, financial institutions that are unable to explain and defend their decisions will be exposed to significant liability claims from customers who challenge AI decisions. This is especially true if there is a systemic issue and customers (or regulators) allege a class claim. Although current professional indemnity (PI) and errors and omissions (E&O) policies should respond to these types of customer claims, current PI and E&O insurance limits may not be sufficient to cover a systemic class claim related to Generative AI decisions.

(ii) Data Privacy

A high risk with the use of Generative AI is that it relies on data to make decisions. Oftentimes this data includes sensitive personally identifiable information (PII). Disclosure or misuse of PII may result in privacy breaches, regulatory fines, reputational damage, and third-party claims from customers. A cyber policy would likely provide coverage for these types of situations. However, similar to the above, the current limits may not be sufficient to respond to a systemic issue caused by AI.

(iii) Distribution of Harmful Content

As Generative AI continues to evolve, it will become more difficult to differentiate reality from illusion. For example, an AI-generated email sent on behalf of the company could inadvertently contain offensive language, issue harmful guidance to customers, or even include a fake video of a company’s CEO. In this situation, the biggest risk is likely reputational, which could lead to a loss of customers and, for a public company, a decline in its share price. Although current insurance policies generally won’t cover pure reputational damage, most D&O policies include reputational cost coverage for insured persons and public companies often have coverage for crisis costs incurred in response to an event causing a material stock drop

(iv) Third-Party Property Damage

It will also be important for organisations to consider whether their current general liability and casualty insurance coverages are geared to pick up liability for damages for third party property or injuries arising out of an AI system. While coverage is likely currently available, as Generative AI technologies become increasingly integrated into various products and services, organisations may be held liable for defects, malfunctions, or injuries caused by the AI components in their offerings. For these types of policies to appropriately respond, understanding whether these failures are attributed to a lack of maintenance, design flaws, or human errors will be key.

What can you do to help mitigate the risks to your organisation posed by Generative AI?

In order to prepare for the risks associated with the use of Generative AI, it is important that AI is embedded into your risk management framework. This means reassessing your risk framework in the context of AI, as AI is likely to increase or decrease different types of risks and change the overall risk profile of your organisation. Further, as AI matures and new risks manifest themselves, risk identification and quantification will need to be continually reassessed and amended.

Additionally, it is key to ensure you stay informed about new laws regulating the use of AI. Although there are currently no laws in South Africa specifically regulating AI, we expect South Africa will follow foreign legislation when drafting its own AI laws. The EU, which is at the forefront of this issue, has proposed a risk-based approach to AI based on different risk levels posed by each AI system. Fundamental to the EU’s proposed law is human oversight aimed at preventing or minimizing the risks of AI to health, safety and fundamental rights. We expect human oversight to be paramount in any laws regulating AI as regulators and legislators continue to navigate this issue.

With respect to insurance, we recommend that you speak with your broker to ensure you understand what coverage you currently have for AI-related risks. Additionally, as the risks associated with AI continue to evolve and increase, ensure you have run loss scenario testing and understand the quantification of possible losses that may be caused by AI. As AI-related risk severity increases, it will also be important to review your insurance limits. Finally, ensure your broker advises you of any new insurance products or forms of alternative risk transfer that may respond to future AI risks.

The views expressed in this article are the writers own and not representative of their employers.