Data security ‘top of mind’
Early adopters of large language models (LLMs) such as ChatGPT, Claude, Copilot and Gemini have learned some hard lessons as they progress from experimenting with artificial intelligence-backed applications in their personal capacities to unleashing it in their professional workstreams.
An epic AI faux pas
The example that caught your writer’s eye was the forced withdrawal by the Department of Communications and Digital Technologies of South Africa’s Draft National Artificial Intelligence Policy after the document’s reference list was found to contain fictitious sources. Aside from the irony of an AI misstep undoing an AI policy, the occurrence highlights the well-documented challenge of AI hallucination.
An AI hallucination occurs when a generative AI system produces made-up information that appears credible, often hoodwinking users who are not subject experts. The problem is partly structural because these systems are designed to generate plausible text rather than verify truth. And if pressed to support a claim, an AI may invent source references and even fabricate URLs that look authoritative but do not exist.
It comes as no surprise, then, that the Financial Sector Conduct Authority (FSCA) and Prudential Authority (PA) report on ‘Artificial Intelligence in the South African Financial Sector’ flags hallucination, inconsistent responses and dependency on data quality as an industry-wide micro-prudential model risk. The report, based on a comprehensive survey of financial services stakeholders, offers some useful lessons on how AI is being adopted across the country’s financial sector.
Basic automation morphs into ML and NLP
“This report fills a gap by presenting data-driven insights on AI adoption trends, investment intentions, use cases and associated risks, alongside governance and ethical considerations,” the regulators said. At the outset, they explained the evolution of AI from its use in basic automation and data analysis to the integration of machine learning (ML) and natural language processing (NLP) into systems. Among the many operational benefits of AI, institutions are now able to process vast amounts of structured or unstructured data at speed.
There are, however, countless challenges facing financial services providers as they integrate AI into their businesses. One concern is that individual employees adopt unsanctioned AI tools, introducing unforeseen risks across the firm. Another is that company-sanctioned AI installations are not properly ring-fenced or scripted. That type of oversight could see an AI tool designed to improve administrative capabilities expose sensitive company data to employees who were never meant to access it.
Last December, FAnews reported on the seven lessons the regulators have taken from this research. Today, our write-up focuses on some of the practical findings teased out from hundreds of survey responses. The first observation is that banks and payment providers are the leading adopters of AI domestically, with around half of respondents saying they had adopted the technology. Interest is far lower in the pensions (14%), investments (11%) and insurance (8%) segments.
Desktop versus strategic AI adoption
Your writer felt the uptake suggested by the survey was on the low side given the rather broad framing of the survey question. Respondents had been asked whether they had adopted either traditional AI, including rules-based and machine learning systems, or generative AI for automation or decision support. Two reasons for the low response are that the survey took place over a year ago, and that firms were likely responding at a high level, thinking of system-wide implementation rather than desktop or departmental exploration.
The second observation was that survey participants, on average, planned to invest less than R1 million in AI, indicating cautious spending and a focus on incremental implementation. Again, banks led the way, with more than half having set aside more than R20 million over 2024. As an aside, you might find the actual spend on AI adoption gets lost in a sea of subscription expenses rather than capital investment. A firm with 50 seats could, for example, deploy Copilot business-wide for around R200 000 in licensing fees, over and above the fees for Microsoft 365.
The small financial and risk practices that FAnews has spoken to seem to favour the licensing approach, giving a select group of employees access to Copilot or similar generative AI solutions with the aim of improving their administrative productivity. An individual or small team is then tasked with exploring and testing other AI enhancements for the business. Many have also indicated that they expect product providers to lead AI investment and pass the benefits through to intermediaries via improved product support and client servicing.
An opportune inbox arrival
Midway through this newsletter, your writer received an opportune presser from professional services firm PwC. Citing its latest AI performance research, it said around 82% of global organisations were already running AI pilots, while few had scaled enterprise-wide adoption. It also revealed that almost two-thirds of workers were using AI.
Unfortunately, organisations across Africa are falling behind global leaders in translating their AI ambitions into measurable returns. “Africa’s challenge is in adopting AI at scale and implementing it fast enough to remain competitive,” said Dion Shango, PwC Africa CEO. He challenged firms to “scale the right AI to transform how they create value.” This echoed FAnews’ informal findings that small firms are putting the cost reduction and productivity gains from the new technology ahead of using AI to create new revenue streams.
The FSCA-PA report also hints at why early adoption is skewed towards efficiency and productivity. The regulators identified firms’ IT and operations divisions as those primarily involved in traditional AI adoption, whereas marketing and sales favoured generative AI. In insurance, machine learning is primarily being deployed to assist in claims management and underwriting. Generative AI, meanwhile, show up in areas such as content creation, financial reporting, internal and external chatbots and real-time servicing, to name a few. Overall, AI is being used to enhance existing product development and distribution methodologies rather than redesign them; its use is operational rather than strategic.
Treating customers fairly not guaranteed
There are other nuances to keep in mind. “An emerging trend to monitor is the extent that AI is being employed to the advantage of the provider rather than to the advantage of the customer,” the regulators said. Some interventions, such as leveraging AI for enhanced cybersecurity, introduce clear benefits across the value chain. It remains to be seen to what extent AI-derived operational efficiencies bring down prices versus simply padding product provider margins.
The FSCA-PA report singled out data privacy and protection, data security and data quality as critical risks across all sectors, later saying that the safeguarding of customer data was a leading ethical consideration. Firms also shortlisted cybersecurity, reputational risk and concerns over AI biases, hallucinations and machine learning ethics as areas for special attention. Asked about constraints, firms held up data privacy and protection laws as a major hurdle, followed by insufficient talent and challenges around solution transparency and explainability.
Firms in the South African financial sector are focusing on fraud detection as a leading machine learning use case and on product and service improvements for generative AI. But they will have to go further to catch up with their global peers. To this end, Africa’s firms are being encouraged to prioritise growth-oriented use cases and recognise the adoption advantages already on display within their organisations.
Turning AI ambition into impact
“Turning AI ambition into measurable impact requires focus and discipline,” said Christiaan Nel, AI Africa Leader at the firm. “Leaders must invest with intent, prioritise growth and create the conditions for AI to scale, combining strong foundations with workforce readiness and ecosystem thinking.” And that, dear reader, is as good a point as any to end today’s commentary.
Writer’s thoughts:
AI-based applications have become so affordable that individual employees can experiment with multiple tools on their smartphones and laptops. How do you prevent your employees’ AI experimentation from spiling into your financial or risk advice practice systems? Please comment below, interact with us on X at @fanews_online or email us your thoughts [email protected].
