Aged care providers should assume artificial intelligence is already operating within their organisations and establish clinical governance frameworks now to oversee its use, according to experts who say AI has increasingly become a board-level responsibility.
At an AlayaCare webinar on AI and clinical governance last week, AlayaCare’s Practice Manager AlayaFlow Kristen Solomon was joined on the panel by FTI Consulting senior managing director Nicki Doyle and FTI consulting senior managing director Sabine Bennett.
Kristen Solomon said organisations that successfully adopted AI put governance before technology.
“Governance starts before the technology, and it starts at the board level,” she said.
Ms Solomon said providers should involve clinical leaders from the outset, define what success looks like for AI tools, and establish clear accountability and escalation pathways before deployment.
She urged providers to carefully consider data sovereignty when selecting AI technologies, warning that data processed in Australia was not necessarily stored in Australia.
“Processed in Australia is not the same as being stored in Australia,” she said. “For health data, storage is what the Privacy Act and your audit really care about.”
Ms Solomon said AI should support, rather than replace, clinicians’ professional judgement.
“AI is an assistive tool and will never replace their clinical judgment,” she said. “It’s there to assist and help them do what they do best.”
She said responsible AI was “not a constraint on innovation” but “the foundation that makes innovation sustainable in a regulated care environment.”
The panel agreed that providers needed to identify every AI-enabled system in use, assign clinical accountability for each tool and ensure clinicians remained responsible for decisions made with AI assistance.
Unprecedented change
FTI Consulting’s Nikki Doyle said the aged care sector had undergone “unprecedented regulatory change”, culminating in the introduction of the new Aged Care Act, the biggest overhaul of the sector since 1997.
She said the reforms had elevated clinical governance to a statutory responsibility of aged care boards following the Royal Commission into Aged Care Quality and Safety, which found organisations lacked sufficient governance oversight, “particularly in relation to clinical care”.
Ms Doyle said the strengthened clinical governance requirements under Standard 5, combined with Australia’s National AI Plan, meant providers now needed to consider how artificial intelligence could be adopted safely while maintaining human oversight and accountability.
She clinical AI governance was now vital for providers.
She described “clinical governance” as the framework for delivering safe, high-quality care through measures such as incident management, quality improvement, evidence-based practice and regulatory compliance. “AI governance” focuses on managing organisational risks associated with AI, including data quality, cybersecurity, vendor oversight and ethical use.
Ms Doyle said clinical AI governance is “the translation layer that sits in between both of those and brings the two frameworks together.”
Ms Doyle said providers needed to consider a range of clinical and operational risks when deploying AI, including fragmented AI systems creating inconsistent outputs, privacy and consent concerns, uncertainty over accountability when AI contributes to harm, missed signs of resident deterioration, biased or inequitable recommendations, and the potential for clinicians to become over-reliant on AI at the expense of professional judgement.
Seven pillars
“From our perspective, there are seven key pillars of clinical governance:
Clinical accountability – Every AI system should have a named, qualified clinician accountable for it.
Success measures – Define what success looks like before deployment and measure clinical outcomes, not just efficiency.
Escalation pathways – Staff need documented processes to query, override or escalate AI recommendations.
Consumer and workforce engagement – Inform consumers in plain language, obtain appropriate consent and train staff in both the technology and its limitations.
Oversight structures – Ensure governance extends from frontline teams through executive leadership to the board.
Audit trails and evidence – Record AI recommendations, clinician decisions, overrides, model versions and clinical outcomes to support quality reviews.
Continuous monitoring – Regularly review AI performance against clinical outcomes, incidents and benefits, rather than simply tracking usage “
Sabine Bennett said providers that had not yet begun developing AI governance should start by educating their workforce.
She said many organisations were likely already using AI, whether they realised it or not, making it essential that staff understood what AI could and could not do, their own clinical accountability when using AI tools, and the importance of reporting near misses so organisations could continually improve their governance.
Ms Bennett also said providers should ensure consumers understood when AI was being used in their care. Information should be provided in plain language, with appropriate cultural considerations, and consumers should have a genuine opportunity to opt out without compromising the quality of their care.
Add Comment