AI-related liabilities: a call for caution
In an era where Artificial Intelligence (AI) is reshaping the landscape of healthcare, understanding the associated risks and liabilities has become crucial.
FAnews recently spoke to JP Ellis, Head of Legal & Claims at EthiQal, who shared his insights on the implications of AI in the healthcare sector, particularly in relation to professional indemnity insurance.
Understanding emerging risks
Ellis highlights the significant potential AI technologies hold for healthcare, stating, “AI can analyse vast datasets, helping doctors make quicker, more accurate diagnoses by identifying patterns that humans might miss.” He also notes that in pharmaceuticals, AI accelerates drug development, identifying promising compounds and predicting patient responses more efficiently than traditional methods. However, he cautions that these advancements come with considerable risks.
One primary concern is data bias. Ellis explains, “When AI systems are trained on non-representative data (for instance, datasets that don’t include diverse patient demographics), the results can be skewed.” This may lead to misdiagnoses or ineffective treatments, especially for underrepresented groups. The ethical and legal ramifications are significant; treatments effective for one demographic might be harmful to another.
Cybersecurity threats are another major risk. With the digitisation of healthcare and the storage of sensitive patient data, Ellis warns, “Hackers may exploit vulnerabilities in AI algorithms, potentially disrupting care or even manipulating medical devices and records.”
Furthermore, he points out the danger of algorithmic errors. A misinterpretation of medical images by an AI tool could result in incorrect diagnoses or treatment recommendations, raising patient safety and liability concerns for healthcare providers.
Addressing AI-related errors in insurance
As AI continues to integrate into healthcare, professional indemnity insurance is adapting to meet these emerging risks. Traditionally, these policies covered errors or omissions in clinical practice. However, with AI's increasing presence, insurers face the challenge of distinguishing between human errors and those driven by AI.
For example, if a doctor uses an AI tool to analyse a patient’s medical history, and the AI misinterprets the data, determining liability becomes complex. Ellis emphasises, “Professional indemnity insurers must consider how AI systems are embedded into clinical workflows and how they interact with healthcare professionals' decision-making processes.”
Enhancements in PI policies
Locally, Ellis observes that many South African insurers are yet to fully address AI-related risks in their policies. He remarks, “This is not entirely surprising given the absence of a formal legislative or regulatory framework governing AI in healthcare in South Africa.” However, internationally, there is a growing focus on enhancing insurance policies. “Professional indemnity insurers in more developed markets are beginning to re-evaluate their policies in response to the rise of AI,” he says.
These enhancements include explicitly defining AI-related risks and how they affect healthcare professionals' responsibilities. Additionally, he mentions, “We’re also seeing exclusions for unapproved AI tools and cyber risk extensions to address concerns around data breaches and privacy violations.”
Advice for healthcare professionals
For healthcare professionals looking to minimise their exposure to AI-related liabilities, Ellis advises caution. “Given that South African healthcare professionals currently do not have access to AI technologies that have been approved by any regulatory authority, caution should be exercised when incorporating AI tools into clinical practice.” He stresses the importance of remaining engaged in the clinical decision-making process, rather than relying solely on AI outputs.
Healthcare providers should consistently review AI recommendations and apply their clinical judgment, ensuring they retain ultimate responsibility for diagnosis and treatment decisions. “Healthcare professionals must remember that AI is a supportive tool - not a replacement for human expertise,” he adds, highlighting the importance of documenting instances where AI has influenced clinical decisions.
Staying informed on legal frameworks
To ensure their insurance offerings remain relevant, Ellis emphasises the importance of staying informed about evolving legal and regulatory frameworks related to AI in healthcare. “We achieve this by maintaining close ties with professional bodies, reviewing industry reports, engaging with legal publications, and regularly liaising with regulators.”
He points to a significant recent development: the draft National AI Policy Framework published by the Department of Communications and Digital Development on August 14, 2024. This marks a step toward regulating AI in South Africa, which will prompt insurers to adapt their policies accordingly.
Claims related to AI errors and trends
As AI-related claims in healthcare are still relatively new, they present unique challenges. Ellis notes, “A key question is whether an error resulted from AI technology malfunction, improper use by the clinician, or a combination of both.”
Such claims necessitate thorough investigations, often involving AI experts, clinical professionals, and legal advisers. “These claims are handled similarly to traditional malpractice claims but with an additional layer of scrutiny regarding AI’s role in clinical decision-making,” he explains.
Looking ahead, Ellis anticipates a significant rise in AI-related liabilities as AI tools become more autonomous in clinical decision-making.
“These liabilities will involve accountability for autonomous decision-making, data security breaches, and ethical concerns regarding AI in patient care,” he predicts. He acknowledges the uncertainties introduced by the National Health Insurance Act, which complicate predictions. However, he assures that preparations are being made by monitoring trends, adjusting risk models, and educating clients on best practices to mitigate AI-related risks.
Supporting clients in adapting to AI
To support clients adapting to the increasing use of AI technologies, Ellis advocates for comprehensive risk assessments.
“Providing training and ongoing education on safely implementing AI tools, as well as maintaining compliance with regulations, is essential.” He encourages open communication between clients and insurers to ensure coverage adequately addresses AI-related risks.
In closing, Ellis remarks, “AI is no longer science fiction - it has become part of modern medicine.” He underscores the dual nature of AI as both an opportunity and a challenge for the healthcare sector. As insurance providers adapt to these new risks, he emphasises the need for collaboration among all stakeholders. “The key is collaboration to ensure that AI serves patient benefits without compromising safety or professional integrity.” As the healthcare landscape continues to evolve, proactive strategies and clear communication will be vital in navigating the complexities of AI and its implications for patient care and professional liability.
Writer’s Thoughts
As AI transforms healthcare, it is crucial for professionals and insurers to stay informed and adapt to emerging risks. Proactive collaboration and continuous education will be essential in ensuring AI enhances patient care while mitigating potential liabilities.
As AI continues to evolve in healthcare, how can we strike the right balance between technological advancement and safeguarding trust and safety? Please comment below, interact with us on X at @fanews_online or email me your thoughts [email protected]
