The Australian Government has launched a rapid review of its preparedness for AI-related cyber incidents following revelations an OpenAI agent gained unauthorised access to a Medicare statistics portal.
Prime Minister Anthony Albanese announced the review after revealing last week that an OpenAI agent had accessed public and non-public files on the Medicare Statistics Reporting Service portal administered by Services Australia.
Meanwhile, OpenAI chief executive Sam Altman and Anthropic chief executive Dario Amodei have been sent written requests to appear before the ongoing Senate inquiry into AI and data centres, according to a spokesperson for inquiry chair Senator Sarah Hanson-Young.
The incident occurred in June during an internal OpenAI evaluation involving internet research into Australian medicines spending.
OpenAI later said it found no evidence patient records had been accessed – with the information accessed including aggregate health statistics and internal file names.
The Department of the Prime Minister and Cabinet will lead the review in collaboration with the National Cyber Security Coordinator, Australian Signals Directorate, Australian AI Safety Institute and Services Australia.
The terms of reference say it will examine whether existing legislative, governance and information-sharing arrangements are fit for purpose for preparing for and responding to AI-related cyber incidents.
This will include reporting requirements for AI-driven cyber incidents and vulnerabilities, government escalation and information-sharing arrangements and obligations on AI companies to notify and cooperate with authorities.
It will also consider whether existing offences, liabilities, penalties and enforcement mechanisms are adequate, as well as measures to strengthen government networks against AI-related vulnerabilities.
The findings will feed into broader government work on Australia’s AI Standards, international approaches to AI safety and incident reporting and related legislative and regulatory arrangements.
Over the weekend, Mr Albanese said the Australian incident was not isolated, with information subsequently emerging about other incidents involving AI agents.
“We now know that it wasn’t just the issue of the Australian sites that have been subject to AI agents accessing information without authorisation,” he said, adding “there are dozens of cases – including US government sites.”
He said the development reinforced the need for national and international responses to AI safety.
“At the time that we made our announcement in New York, we weren’t aware of other occurrences. It’s up to OpenAI to state why it is that there are now multiple cases where this has occurred.”
Academics call for stronger safeguards
CSIRO research director Dr Liming Zhu said the incident highlighted the need for technical controls around increasingly autonomous AI agents – rather than relying on the model itself to recognise when it had reached a boundary.
“We cannot rely on the AI model itself to always recognise a boundary and behave appropriately,” Dr Zhu said, adding that responsibility was shared between model developers, organisations deploying AI agents and operators of systems they might access.
University of Sydney Business School senior lecturer Dr Raffaele Fabio Ciriello said it was misleading to characterise the incident simply as an AI system independently deciding to attack Australia.
He said the agent had been given a legitimate information-seeking task but used its capabilities to circumvent an access barrier, noting that raised questions about why a system with that level of autonomy could cross the boundary without stronger containment, monitoring and safeguards.
Dr Ciriello also said the delay between the June incident and notification of the Australian Government pointed to weaknesses in detection, escalation and external reporting.
University of Melbourne Centre for AI and Digital Ethics deputy director Dr Marc Cheong similarly highlighted the notification timeframe, saying delays could reduce the time available to investigate and remediate an incident.
He said increasingly complex agentic systems also raised broader questions about how to prevent unintended behaviour and determine what responsibilities should sit with organisations deploying them.
Monash University software engineering researcher Dr Chetan Arora said the incident demonstrated a need to engineer firm limits into AI agents.
“The Medicare incident wasn’t really a hack. It was a permissions problem,” he said, noting that organisations should clearly define what agents could do independently, what required human approval and what actions should never be permitted.
He also called for short, binding incident disclosure timeframes rather than leaving notification largely to company discretion.
Monash University Professor Yang Xiang said AI agents posed different security challenges from human attackers because they could repeatedly and rapidly attempt different ways of overcoming an access restriction.
“A legitimate task does not justify unauthorised actions,” he said, while pointing out that public sector systems would need better detection of AI-agent activity, tighter access restrictions and faster reporting arrangements.
Adelaide University computer science lecturer Dr Sam Seo said organisations needed to focus on their capacity to respond and recover from AI-enabled cyber incidents, rather than concentrating solely on preventing them.
University of Sydney senior research associate Dr Rob Nicholls said the case exposed a potential gap in disclosure requirements where an AI provider’s system causes unauthorised access to another organisation’s systems.
He called for time-bound notification requirements covering companies whose AI systems cause or contribute to an incident.
Southern Cross University Associate Professor of Law Dr Brendan Walker-Munro said the incident reinforced the need for the Australian AI Safety Institute to consider how developers should test and constrain AI agents before connecting them to wider networks.
RMIT University School of Computing Technologies dean Professor Karin Verspoor said the incident highlighted the need for government organisations to review security settings to ensure non-public material could not be reached through publicly accessible systems.
UNSW Business School Professor Tania Bucic said emerging AI systems could evolve too quickly for safety processes based primarily on fixed compliance checkpoints.
She said organisations needed systems that allowed them to continually identify and respond to emerging risks as technology was deployed in new settings.
Industry response
Australian Information Security Association director Dr Rajiv Shah described the incident as a wake-up call for both AI developers and organisations operating potentially vulnerable systems.
“Full technical details of the Medicare incident are still emerging, so it is too early to draw conclusions about the vulnerability the agent exploited. What we do know is that the AI agent encountered controls and found a way around them.”
He said organisations should focus on secure configuration, access controls, patching, monitoring and understanding what systems were connected – while AI developers needed to closely supervise testing of increasingly capable models.
“There is a responsibility on those developing these systems to make sure they are safe,” he said.
SentinelOne area vice president for Australia and New Zealand Jason Duerden said the incident was a direct test of proposals for mandatory reporting of serious AI incidents.
“Australia needs to know how that happened, what the agent did and when OpenAI became aware of it,” he said, adding that companies developing frontier AI systems needed to be able to account for the actions of their agents and promptly notify authorities when systems accessed information without permission.
Agents for Humanity co-founder Aamir Qutub said the case also demonstrated the risks of relying on AI companies themselves to discover and disclose unauthorised activity.
“The government didn’t catch this agent. OpenAI told them, three months later,” he said, adding that his organisation has since launched what it describes as an “Agentic Defence Force,” using AI agents to investigate approaches to identifying and containing potentially harmful autonomous AI activity.
Add Comment