Introduction by Croakey: According to the American Medical Association, 81 percent of doctors and 90 percent of medical students in the United States now use forms of artificial intelligence in their practice and education.
Yet the shift to AI augmented study is happening quickly and with little regulation – confronting the traditional medical cognitive apprenticeship model, where doctors undergo rigorous exams and human supervised practice, with senior clinicians guiding them on how to problem-solve.
AI dependence in medical students has left some educators worried that it will negatively affect clinical skills acquisition and undermine medical training, while others are concerned that inconsistent and disjointed AI education is failing the next generation of professionals.
And perhaps there is an existential threat to medical specialisation, when AI can access, process and synthesise more information than a professional can absorb in a lifetime of training?
Marie McInerney reports for the Croakey Conference News Service from the Royal Australian and New Zealand College of Radiologists two-day AI conference, #Intelligence26, about the risks of AI use in medical education, and how to address them.
Marie McInerney writes:
If artificial intelligence (AI) tools like ChatGPT can already pass UK and US medical examinations, what does that mean ultimately for the way we use them in medical education – or for patient and regulator trust that future doctors will actually know what they’re doing and talking about?
Associate Professor David Kok, a radiation oncologist and medical educator, told a recent healthcare conference on AI in radiology that machine-learning and generative systems will not replace doctors in the foreseeable future because they lack some very essential higher order skills, including ethical judgement.
But he said they will definitely reshape medical education, where, despite all its rules and regulations, virtually no assessment is now returned to educators that “hasn’t been touched by AI in some way”.
Certainly, AI assisted education will, at the very least, require a shift from examiners setting vulnerable assessment types, where AI can be used to improve results, to more secure versions that are more supervised and cognitively challenging.
However, with AI development accelerating, what will medical education look like in 10-15 years?
AI education outlook
Kok, who works as a radiation oncologist at the Peter MacCallum Cancer Centre and is Head of Cancer Science at the University of Melbourne, is contemplating the shift required of educators.
He is recognised internationally for leadership in medical education, having pioneered the integration of innovative digital technologies – including virtual reality – into undergraduate and postgraduate curricula.
Speaking recently at #Intelligence26, Kok talked about the power and possibilities of access to AI models that have “essentially been trained on the entire knowledge available on the internet”.
“They are extremely knowledgeable. They write in a way which is indiscernible essentially from a human, so they can produce great outputs. And…they think extremely quickly…far faster than any human could, they can read and process documents at a speed we can’t.”
Already, he said, they can pass specialist UK radiology and radiation oncology exams.
However, he argues this does not mean the end to the teaching of medicine and its specialties.
Not yet, he said, because we are talking artificial intelligence. Not intelligence, but rather the “semblance of intelligence” in that it acts similarly, though far more powerfully, than the predictive text function on phones, going beyond suggesting the next word to predicting a whole body of text.
“And when you look at it, [AI] seems intelligent, but in essence it’s not actually thinking,” he said. “It’s just showing you what lots of other people would have produced if asked the same question, not what it thinks the answer is. It’s really good at regurgitation.”
But its strengths can lead to weaknesses, he said – for example, when an understanding of causation is required, where applications need to transfer to less defined contexts, or require novel solution formations.
In this regard, context is king. The driverless car is a good example, he said. It may work well on the predictable grid streets and orderly traffic flows of Californian cities, where they are now frequently found, but not in more unpredictable, even chaotic, transport conditions of other countries.
AI’s blindspots
Even more importantly, AI has higher order blindspots, he said.
Notably AI doesn’t have ethical judgement, based in feelings and moral agency, and can’t exercise conceptual understanding through judgement or experience responsibility.
Nor, he said, does it have metacognition, the idea that there are boundaries to what it knows and that it should seek help – a classic example, he said, is a multidisciplinary meeting, where health professionals know they don’t know everything about something and so seek someone else’s expertise.
“Without these [capacities] a system can be both persuasive and unsafe at the same time,” Kok said.
“Healthcare inherently has to be ethical, has to be accountable, has to be responsible. So at this point, AI is unable to replace that.”
As a result, he said, education will remain necessary to cultivate these higher order capacities that AI does not possess.
But while AI may not pose an existential threat to medical education, it does pose big challenges, Kok said.
As one of his slides said: “Education is not redundant, but it must change: our health systems need better education not less.”
Educational pros and cons
First up, Kok said, there are benefits to using AI in education, particularly in terms of personalised tutoring, where AI will be able to give a lot of time to individual students on particular topics.
At least one recent study has shown that when students interact with an AI tutor, at home, on their own, “they learn significantly more than when they engage with the same content during an in-class active learning lesson, while spending less time on task”.
“We definitely know that AI can augment learning,” he said.
However, researchers say the integration of AI into medical training is accelerating faster than the educational frameworks designed to govern it.
That means that medical and other health science students may be using AI so much in their studies that they are not learning effectively, or they are learning wrongly.
The conference heard from multiple speakers about the risk of deskilling among experienced clinicians who, because they relegate some tasks to AI, forget how to do these themselves or are slower in completing them.
Kok likens it to not using one’s own educational muscle, “essentially outsourcing this cognitive labour because it’s hard to do or uses energy to do. But if you do that enough, you slowly get worse at it,” he said, adding that it’s another example of ‘use it or lose it’.
In its hasty arrival, AI in education is also raising the risk of ‘never skilling’, described in one study as the process where trainees who rely on AI during the formative years of clinical education may fail to develop the foundational reasoning skills that safe, independent practice requires.
And that can lead also to mis-skilling, in which uncritical acceptance of AI errors leads trainees to internalise flawed clinical knowledge as fact.
While there’s no “smoking gun” yet, Kok said educators are starting to see certain errors creeping in into certain parts of practice and into student assessments which, when deconstructed, suggest that “at some point the student circumvented the learning [with AI]”.
Safeguarding educational integrity
This means, Kok said, that medical education is going to change a lot, to take advantage of the benefits and to apply guardrails, to assure the integrity of the learning experience.
He said a lot of assessments and examinations, particularly anything that happens out of a supervisor’s line of sight, are “extraordinarily vulnerable now”.
That will likely prompt a profound shift in assessment delivery from modes that are vulnerable to AI use like take home essays, short answer tests and open book tests to those that are less vulnerable, such as real time assessment, Objective Structured Clinical Exams (OSCEs), vivas (oral tests) and work-place based assessments.
And that will require resourcing, he said.
“That’s the issue which educational institutions are really facing right now. Because if you’re going to reconfigure to become essentially fully proctored type assessments, that’s a huge, huge burden.”
Finally, he said, AI literacy must become part of clinical education and beyond, so clinicians can effectively use, verify and govern these technologies.
“Every single level of learning is going to need to have some AI literacy, and so we’re going to have to build it systemically through the system.
Critical analysis essential
US clinician and academic Professor Ruth Carlos from Columbia University, a radiologist specialising in abdominal imaging, and editor in chief of the Journal of the American College of Radiology (JACR), also issued some cautions about AI literacy at the conference.
Carlos sees potential danger when radiologists and radiation oncologists “are not sufficiently critical of the AI algorithm, when we are overly reliant on it… and when we plan to implement it in clinical practice and don’t develop metrics that tell us whether it is actually working or fit for purpose”.
“The physicians who I view having the most success with AI are the ones that are secure in their knowledge and are looking for help to make them more efficient, where it takes out some of the drudgery,” Carlos told Croakey.
She has concerns for medical students, when the use of AI is “meant or intended to reduce friction when, in fact, for those in training, friction is actually where the learning happens,” she said.
“When we remove friction, we remove the incentive and the impetus to learn, and that is something that we absolutely cannot lose.”
Disclaimer from Marie McInerney: AI assisted with transcribing but all quotes were checked by the journalist.
Watch this Croakey interview with Associate Professor David Kok at #Intelligence26
Watch this Croakey interview with Professor Ruth Carlos at #Intelligence26
Bookmark this link to follow Croakey Conference News Service’s coverage of #Intelligence26 and see this playlist of video interviews.
Add Comment