Introduction by Croakey: If you’ve not yet caught up on Croakey’s must-watch video overview of #Intelligence26, the AI futures conference hosted by the Royal Australian and New Zealand College of Radiologists (RANZCR), then here’s a compelling summary of the debates about AI’s potential in healthcare to improve diagnoses and workplace productivity.
Marie McInerney reports for the Croakey Conference News Service on the dangers of overestimating both AI’s possible efficiencies and its impact on advances in care.
Marie McInerney writes:
Pressure from governments and hospital management to rush AI into healthcare systems, driven by hype and hope about efficiency gains, is a major challenge facing health professionals.
It’s particularly an issue for radiologists and radiation oncologists, who boast of being the most ‘tech savvy’ health profession and whose image-rich daily work is at the forefront of AI’s juggernaut arrival in health.
As #Intelligence26 speakers indicated, radiology can now dream of AI tools that may cut up to a third of routine administration and reporting work over the next five years, and there are great hopes for AI’s ability to triage and clean up images, highlight medical abnormalities, and better predict outcomes for patients.
One prospect highlighted at the conference is that AI-supported chest X-ray interpretation in an Emergency Department can support earlier identification of serious cardiopulmonary disease within routine clinical care. Another is that AI may offer “the most significant step change in breast cancer mortality” since screening was introduced in Australia in the 1990s.
However, while radiologists are on the whole excited about the prospects to improve their work and care, the recent conference heard wide-ranging concerns that AI algorithms are still making “stupid mistakes” and delivering far fewer efficiency gains than predicted.
Speakers expressed concerns that tools and algorithms will be introduced without being properly designed, tested, evaluated and regulated. They were also troubled about trials that don’t understand or assess for clinical use, that lack mature and diverse data, and overlook AI’s hidden costs, including to the environment, equity, medical education and workplace stress.
RANZCR president Dr Rajiv Rattan told the event that the profession’s biggest challenge might not be the technology itself, but “the narrative around it”.
It’s a huge temptation, he said, for governments and regulators facing workforce shortages, ever increasing costs, and widening access gaps to try to win efficiency gains.
In that context, he said, AI can appear as a shortcut: “a substitute for training more clinicians and building workforce capacity”.
“It is up to us to change that narrative, it is up to us to very emphatically say that AI is most powerful when it augments human expertise,” he told the two-day Sydney event.
Outlook on change
As reported earlier at Croakey, the United States Food and Drug Authority has cleared 1,500 AI software tools to date, and three quarters are radiology focused.
It’s evidence of the profession’s leading role in the AI revolution, which is followed by other imaging-based disciplines, such as pathology, ophthalmology, and dermatology, and the next-in-line “signal-based disciplines” like neurology, anaesthesiology, and cardiology.
For Dr Curtis Langlotz, Professor of Radiology, Medicine, and Biomedical Data Science and Director of the Center for AI in Medicine and Imaging at Stanford University, this position is a critical advantage gained from storing information as digital data for decades.
He told the conference there’s “never been a more exciting time” to be a radiologist, and counselled the profession to see AI as “just the latest” in a stream of tech updates it has experienced, managed, and benefited from.
It was easy to see, for example, from one of his slides below, how the ChatGPT-style features of AI might improve patient literacy.
Where’s the evidence?
The trouble is, as multiple speakers told the conference, the evidence is not there yet to prove either AI’s efficiency gains or its capacity to transform diagnostic radiology.
In the UK, colleagues have queried the safety and efficiency, as well as the rigour of evidence around the AI administration and communications tools that the National Health System (NHS) is currently rolling out to “help cut waiting lists and improve care for millions of patients”.
The incoming tech includes a new AI triage tool in the NHS App that helps direct patients to the most appropriate NHS service, and widespread access to AI notetaking tools to reduce administration for NHS staff.
The accelerated rollout has prompted concern that the evidential bar for digital technology in health is “much lower” than for new medicines, and that claims about AI safety and benefits “are often shaped by commercial or political interests”, as authors from the University of Cambridge and London’s Health Foundation warned last week in The BMJ.
The evidence for the capacity of one triage tool, they wrote, rests on a single practice study, available only as a 2025 preprint, which “was not a trial nor designed to measure queuing at all; it had no control group and no adjustment for secular trends, and the preprint server itself carries a note that the work should ‘not be used to guide clinical practice.’”
New Zealand radiologist Dr Martin Gunn told the conference many radiology tools are tested still only in limited trials.
Radiology desperately needs more prospective, real-world studies, to know if the tools perform out of a biomedical research hub, he said.
“We still don’t know if they’re improving health.”
Other speakers, including including Dutch expert Professor Andre Dekker, agreed that there is much to overcome in order to be more sure of AI’s benefits. Access to mature, high‑quality, interoperable data is essential, yet patient records remain siloed and subject to strict privacy regulations.
Data bias, where machine learning is trained on limited groups of people or patients, can discriminate against marginalised or minority populations and mislead in general, warned Associate Professor Hyun Soo Ko, a radiologist and clinical researcher at the Peter MacCallum Cancer Centre in Melbourne.
Ko worries that brilliant minds in research incubators might create wonderful bespoke AI models, which then “might not work outside [the laboratory], in a rural area or for other minority communities, increasing health disparities”.
Huge potential, great risks
Dekker, a keynote speaker on clinical implementation of AI in radiation oncology, is an early AI adopter and sees “huge potential” for both efficiency and efficacy in research and practice.
One of the most exciting developments in the Netherlands, in his view, is where AI can now legally override a clinical decision over who is chosen for the relatively few slots available for proton therapy, a more accurate and precise way to irradiate a tumour without damaging the surrounding healthy tissues, versus traditional photon therapy.
This is the first time that AI/data-based predictions have been approved as “the most objective way” to select patients, he said.
“And if the AI says no, we are not allowed by law to treat you [with proton therapy] in my country,” he told the conference. Interestingly, this shift came at the recommendation of radiation oncologists, “because they know they can’t predict the future and they know AI is better with that”.
But Dekker shares wider concerns about hype and promise, quality evidence, bias, risks in clinical implementation, and the impact on the workforce, including who might be held responsible in the case of errors, a fear raised recently by Australian doctors.
He is also seeing growing pressure from funders and policy makers to “introduce AI [and] be quick about it” to meet the challenges of an ageing population, growing demand and workforce shortages in healthcare.
Scepticism required
Dekker, who is professor of Clinical Data Science at Maastricht University Medical Center and Maastro Clinic, said he believes ultimately that AI can deliver on expectations and hope, but it will take real time, real investment, and real care.
“[Policy makers] think you just switch on a licence from OpenAI or from Copilot, and that AI will help you,” he said. “This is not true.”
He advises governments and health systems to beware just investing in the “fanciest, newest” AI technology. Rather they should invest in mature and shareable data and in getting professions “AI ready” so AI can reach its potential and people can understand and manage its limitations in healthcare, he said.
-
Andre Dekker’s key messages on AI adoption in radiation oncology. Image Marie McInerney.
At the clinical level, Dekker warns against blind resistance to the introduction of AI but suggested “a bit of scepticism is not bad”.
“AI is great, but if you really want to apply it for the high stakes that we have in in healthcare, it really still needs a lot of work,” he said.
It’s a lesson he learnt the hard way, he told the conference.
In March 2020, as the peak of the COVID-19 pandemic hit Europe, his team at the Maastricht University Medical Center accepted the offer of an algorithm originating from China which promised, in just a few seconds, to be able to confirm from a lung x-ray whether someone had the coronavirus and also predict how quickly the virus was progressing.
In fact, “it didn’t work at all,” Dekker said. Not because the algorithm was bad, but because it had been “completely trained on Chinese scans, Chinese patients, their protocol”.
The experience highlighted that AI algorithms “cannot be expected to perform higher than the level of their training data” and that clinicians must make sure they rigorously validate AI tools for their particular patients and purposes.
Dekker gave other alarming examples that have emerged in radiology, including one algorithm that had only learned to ‘read’ from supine scans — people lying on their backs.
“If you give it a person that’s prone, on their belly, it starts making mistakes because it’s never seen a flip side,” he said.
In another, perhaps more worrying case, an algorithm trained on abdominal CT scans to find a liver would try to find one when, say, presented with a CT of a brain, because it “want[ed] to produce something”.
As a medical physicist responsible for the implementation of AI, he said, that creates “a totally different risk management situation to handle”.
Debating efficiencies
The jury is also still out on actually how much more efficient AI will make radiology and radiation therapy or how much time it will free up for clinicians.
Much is made about the potential of AI to examine every pixel on an image “without getting tired or distracted”, as Curt Langlotz says, and to compare it to every other image it has ever encountered.
Crunching the numbers recently to contest predictions that radiology might be an endangered species, Langlotz found that radiologists spend about two-thirds of their time on interpretation tasks and about a third on communication and administration.
He concluded they were likely to see a 33 percent reduction over the next five years in their hours worked as the routine tasks are instead done by AI.
But before funders get too excited about staff savings and clinicians too fearful for their jobs, he said that this reduction of hours will occur in the context of “incredible growth in the volume of imaging and the relatively flat number of trainees that we’re training each year”.
“Many radiologists are overworked and burned out, and so I think that these efficiency gains that we’ll see with AI are really likely to help us work through the incredible volume of work that we are seeing, rather than any kind of displacement,” he told Croakey.
“Maybe we’ll be able to get home for dinner a little sooner.”
The complexity trap
However, Dekker and others are more sceptical of AI’s efficiency gains, saying those delivered to date in healthcare “are way more modest” than predicted from early studies and trials.
One worry raised at the event was that AI will take over the easier tasks, leaving clinicians with more complex work for longer periods in a day. This prompted an attendee to comment: “We can’t do eight hours of only complex stuff every day of the week.”
It’s a claim supported by a cross-sectional study from China, which has shown frequent AI use is associated with an increased risk of radiologist burnout, particularly among those with high workload or lower AI acceptance.
This is because “the simple tasks, like perhaps writing a report …is taken over by AI and you get left with the complex tasks, and then you get more of them because the hospital wants you to do more,” Dekker said.
Future of decision-making
Importantly #Intelligence26 raised critical questions about what all these trends will mean for the way radiologists and radiation oncologists make decisions.
Will they submit to ‘automation bias’, where they begin to rely too much on AI’s reading, missing errors and beginning to lose expertise in certain areas? Or ‘confirmation bias’, where clinicians fail to cross-check AI diagnoses because they match their own initial suppositions?
Might they instinctively bristle against what AI highlights or recommends, especially if they’re worried about losing their jobs, and therefore cut any potential budget savings by second guessing or tweaking?
Or will they have to keep such a close eye on AI, because it can produce crucial errors including so-called hallucinations, that it negates the cost of purchasing and using it in the first place?
Martin Gunn commented recently via social media on a multicase, multireader study involving chest radiographs interpreted by five radiologists using preliminary reports generated by a radiology-specific multimodal AI model.
Gunn observed that about 40 percent of the AI reports still required human editing, with some examples of AI performance being “particularly sobering: an enormous mass (tumour) that AI called normal, and a hallucinated reduction in pneumothorax size”.
#Intelligence26 discussed many other issues relevant to Croakey readers, including the often hidden or ignored equity (global and local) and environmental costs of AI, the impact of AI on medical education, and issues around the costs of AI licences, consent, transparency and trust. We will report more on these in upcoming articles.
• Disclaimer from Marie McInerney: AI assisted with transcribing but all quotes were checked by the journalist.
Bookmark this link to follow Croakey Conference News Service’s coverage of #Intelligence26 and see this playlist of video interviews.
Add Comment