The questions you should ask about AI deployment
The rapid adoption of artificial intelligence (AI), generative AI and agentic AI is changing how financial services businesses operate; it is also introducing a widening set of governance risks. South Africa’s own AI policy process has already illustrated this point, as described in a recent FAnews newsletter, ‘Data security top of mind’.
AI-enabled shortcuts can unravel
That piece outlines the withdrawal of the country’s Draft National AI Policy, after fictitious references were found in the document, showing how quickly AI-enabled shortcuts can unravel. The policy was yanked before serious harm was caused; but imagine what could happen if similar errors creep into insurance contract wordings or begin impacting on claims, pricing or underwriting decisions. Aside from the obvious concerns, legal and regulatory issues can arise from something as simple as sharing a document with Chat GPT, Claude or Gemini.
Local insurers are already integrating AI and automation across their value chains, prompting the insurance and litigation team at Deneys, an Africa-focused law firm, to issue a stern warning about the legal risks emerging from AI deployment. In an FAnews exclusive, they write that “AI deployments are creating interconnected legal exposures that few insurers have governance frameworks to address.” The full article is reproduced here with minor edits, followed an FAnews paragraph to close.
Privilege at risk
The distinction between consumer and enterprise-grade AI platforms is not a marketing classification, but a legally material difference regarding whether the confidentiality requirement underpinning legal professional privilege can be satisfied at all.
Under south African law, both legal advice privilege and litigation privilege rest on a foundation of confidentiality. Public-facing AI platforms routinely retain input data, permit its use for model training and allow third-party access under terms few users scrutinise. Once confidential legally privileged documents have been shared with a public AI platform without sufficient safeguards, control over who can access them will likely be lost. If legal privilege over relevant documents is lost, those documents are no longer protected from disclosure. This means they may have to be disclosed, and may be used against you in litigation.
The consequence is not hypothetical. In United States v Heppner, defence materials generated on a publicly available AI platform were held not to attract privilege because the platform’s terms of service defeated the confidentiality requirement. South African courts have not yet engaged with equivalent facts, but nothing in the applicable legal framework suggests the analysis would differ.
Public facing platforms
A Supreme Court of Appeal ruling in Ibex v Tiso Blackstar (2024) confirmed that litigation privilege turns on the dominant purpose at the time a document is created, and that privilege can be lost by conduct that is inconsistent with the intention to maintain confidentiality, for example through wide public disclosure. South African courts will likely take the approach that inputting legally privileged materials into public facing AI platforms is inconsistent with the intention of maintaining confidentiality.
Until enterprise-grade tools with contractual data governance protections and closed processing environments are in place, the interim position is straightforward: privileged material should not be submitted to public-facing consumer grade AI platforms. Non-legal advisors using AI to conduct ‘legal analysis’ or generate ‘legal advice’ which could be relied on by the business to make decision should be cautious, because legal professional privilege will likely not apply where an advisor is not involved in a legal professional capacity.
Fraud has been democratised
The same generative AI technology that insurers are deploying internally has materially lowered the barrier to sophisticated insurance fraud. Fabricated medical reports complete with diagnostic codes and forged signatures; damage photographs manipulated using generative-fill tools; deepfake video and audio; altered CCTV footage; and telematics data altered through third-party devices are no longer the preserve of organised syndicates. The specialist knowledge that previously limited this type of fraud has been stripped away.
Without early intervention to assess validity, claimants may adopt a nothing-to-lose mentality, emboldened by the apparent credibility of AI-generated content. Fraud must be proved on a balance of probabilities, and courts will not easily impute fraud to an insured. Building the inferential chain requires authenticated evidence: device-original image files with intact metadata; hash values; raw telematics data exports with system versioning rather than processed reports; server-side platform logs tied to specific account or device identifiers; and supplier documentation obtained through subpoena.
The chain of custody is critical too. You need to show who collected the evidence, when, using what method and how integrity was preserved. The window to secure this evidence is narrow because flight data from a drone incident can be overwritten once an aircraft resumes operation; and telematics records can be amended. An insurer that has not built evidence preservation into its first-response protocol is managing fraud risk that AI has substantially worsened using processes that predate it.
Regulation is catching up
The Prudential Authority’s (PA’s) 2025-2030 Regulatory Strategy notes that financial technology continues to have a significant impact on the financial sector, and supports that the PA is enhancing its regulatory and supervisory framework to deal with the impact of AI. The private sector is doing its part. Insurers and banks have already deployed AI in Anti-Money Laundering and Counter-Terrorism Financing compliance workflows with great effect, and there are many other ways in which AI can alleviate compliance headaches.
The risk in AI deployment is not limited to the courts. Many insurance disputes are resolved through arbitration or mediation, where confidentiality and disclosure standards create unique AI-related exposures. Professionals in the field are leading the response. For example, arbitration guidelines published last year by the Chartered Institute of Arbitrators and the Arbitration Foundation of Southern Africa require early disclosure of AI use and prohibit arbitrators from delegating decision-making to AI systems. Any AI use in drafting the outcome of the arbitration should be independently verified by the arbitrator.
The confidentiality risks from cloud-based tools in arbitration proceedings are structurally identical to those in the privilege context. While the Gauteng Division of the High Court’s mandatory mediation protocol does not specifically deal with AI use, it emphasises that any disclosures and documents used in the context of the mediation remain confidential and cannot later be used if the matter proceeds to trial. Mediators do not assume decision-making roles and therefore should not assume the risks occasioned by AI use. The parties to mediation can however agree to AI use for documents exchanged in the mediation agreement.
Risks cannot be managed in isolation
An alleged fraud dispute that proceeds to arbitration engages both the evidence authentication problem and the AI disclosure requirements. A compliance matter that generates litigation raises both the regulatory exposure and the privilege question around how legal strategy was developed. These risks are not discrete and cannot be managed in isolation within separate business units.
Governance that addresses which tools are used and by whom, what dominant purpose governs each deployment, how outputs are classified and circulated, and what evidence preservation obligations apply from the moment AI-generated materials are created is what separates a managed AI deployment from one that is steadily accumulating legal exposure. That concludes the Deneys contribution.
Responding to heightened complexity
This brief introduction to AI-related governance pitfalls should give insurers and insurance brokers pause. Yes, AI tools can improve efficiency; but their use introduces significant hidden risks that could resurface unexpectedly. The practical takeaway from today’s newsletter is to refrain from feeding confidential data into public AI platforms. Client and claims data should only be shared on secure, private systems where there is clarity on who has access to the inputs and outputs, and where the process can withstand legal and regulatory scrutiny.
Writer’s thoughts:
AI use is pervasive nowadays. Case in point, and somewhat ironically, today’s guest article opens with the ‘not this, but that’ framing often associated with generative AI. Do you fully understand where, how and by whom AI is being used in your business? Please comment below, interact with us on X at @fanews_online or email us your thoughts [email protected]. FAnews would like to thank the Deneys insurance and litigation team for their valuable input.
