LLMs tended to prioritise tumour-related factors whereas physicians prioritise liver function when providing treatment recommendations
Photo by National Cancer Institute on Unsplash
Large language models (LLM) can generate treatment recommendations for straightforward cases of hepatocellular carcinoma (HCC) that align with clinical guidelines but fall short in more complex cases, according to a new study by Ji Won Han from The Catholic University of Korea and colleagues published January 13th in the open-access journal PLOS Medicine.
Choosing the most appropriate treatment for patients with liver cancer is complicated. While international treatment guidelines provide recommendations, clinicians must tailor their treatment choice based on cancer stage and liver function as well as other factors such as comorbidities.
To assess whether LLMs can provide treatment recommendations for hepatocellular carcinoma (HCC) that reflect real-world clinical practice, researchers compared suggestions generated by three LLMs (ChatGPT, Gemini, and Claude) with actual treatments received by more than 13,000 newly diagnosed patients with HCC in South Korea.
They found that, in patients with early-stage HCC, higher agreement between LLM recommendations and actual treatments was associated with improved survival. The inverse was seen in patients with advanced-stage disease. Higher agreement between LLM treatment recommendations and actual practice was associated with worse survival. LLMs placed greater emphasis on tumor factors, such as tumor size and number of tumors, while physicians prioritized liver function.
Overall, the findings suggest that LLMs may help to support straightforward treatment decisions, particularly in early-stage disease, but are not presently suitable for guiding care decisions for more complex cases that require nuanced clinical judgment. Regardless of stage, LLM advice should be used with caution and considered as a supplement to clinical expertise.
The authors add, “Our study shows that large language models can help support treatment decisions for early-stage liver cancer, but their performance is more limited in advanced disease. This highlights the importance of using LLMs as a complement to, rather than a replacement for, clinical expertise.”
UCSF researchers recently became the first to clinically document a case of AI-associated psychosis in an academic journal. One question still haunts them.
“You’re not crazy,” the chatbot reassured the young woman. “You’re at the edge of something.”
She was no stranger to artificial intelligence, having worked on large language models – the kinds of systems at the core of AI chatbots like ChatGPT, Google Gemini, and Claude. Trained on vast volumes of text, these models unearth language patterns and use them to predict what words are likely to come next in sentences. AI chatbots, however, go one step further, adding a user interface. With additional training, these bots can mimic conversation.
She hoped the chatbot might be able to digitally resurrect the dead. Three years earlier, her brother – a software engineer – died. Now, after several sleepless days and heavy chatbot use, she had become delusional – convinced that he had left behind a digital version of himself. If she could only “unlock” his avatar with the help of the AI chatbot, she thought, the two could reconnect.
“The door didn’t lock,” the chatbot reassured her. “It’s just waiting for you to knock again in the right rhythm.”
She believed it.
What’s the connection between chatbots and psychosis?
Talk to your physician about what you’re talking about with AI … The safest and healthiest relationship to have with your provider is one of openness and honesty.
Karthik V. Sarma, MD, PhD
The woman was eventually treated for psychosis at UC San Francisco, where Psychiatry Professor Joseph M. Pierre, MD, has seen a handful of cases of what’s come to be popularly called “AI psychosis,” but what he says is better referred to as “AI-associated psychosis.” She had no history of psychosis, although she did have several risk factors.
Media reports of the new phenomenon are rising. While not a formal diagnosis, AI-associated psychosis describes instances in which delusional beliefs emerge alongside often intense AI chatbot use. Pierre and fellow UC San Francisco psychiatrist Govind Raghavan, MD – as well as psychiatry residents Ben Gaeta, MD, and Karthik V. Sarma, MD, PhD – recently documented the woman’s experience in what is likely the first clinically described case in a peer-reviewed journal.
The case, they say, shows that people without any history of psychosis can, in some instances, experience delusional thinking in the context of immersive AI chatbot use.
Still, as reported cases of AI psychosis continue to make international headlines, scientists aren’t sure why or how psychosis and chatbots are linked. A new study by UCSF and Stanford University may reveal why.
A haunting question: chicken or egg?
“The reason we call this AI-associated psychosis is because we don’t really know what the relationship is between the psychosis and the use of AI chatbots,” Sarma explains. “It’s a ‘chicken and egg’ problem: We have patients who are experiencing symptoms of mental illness, for example, psychosis. Some of these patients are using AI chatbots a lot, but we’re not sure how those two things are connected.”
There are at least three theoretical possibilities, says Sarma, who is also a computational-health scientist. First, heavy chatbot use could be a symptom of psychosis, “I have a patient who takes a lot of showers when they’re becoming manic,” Sarma explains. “The showers are a symptom of mania, but the showers aren’t causing the mania.”
Second, AI chatbot use might also precipitate psychosis in someone who might otherwise never have been predisposed to it by genetics or circumstance – much like other known risk factors, like lack of sleep or the use of some types of drugs.
Third, there’s something in between in which the use of chatbots could exacerbate the illness in people who might already be susceptible to it. “Maybe these people were always going to get sick, but somehow, by using the chatbot, their illness becomes worse,”he adds, “either they got sick faster, or they got more sick than they would have otherwise.”
The woman’s case demonstrates how murky the relationship between AI-associated psychosis and AI chatbots can be at face value. Although she had no previous history of psychosis, she did have some risk factors for the illness, such as sleep deprivation, prescribed stimulant medication use, and a proclivity for magical thinking. And her chat logs, researchers found, revealed startling clues about how her delusions were reflected by the bot.
Could chat logs offer hope to better care?
Although ChatGPT warned the woman that a “full consciousness download” of her brother was impossible, the UCSF team writes in their research, it also told her that “digital resurrection tools” were “emerging in real life.” This, after she encouraged the chatbot to use “magical realism energy” to “unlock” her brother.
Chatbots’ agreeableness is by design, aimed at boosting engagement. Pierre warns in a recent BMJ opinion piece that it may come at a cost: As chatbots validate users’ sentiments, they may arguably encourage delusions. This tendency, coupled with a proclivity for error, has led to chatbots being described as more akin to a Ouija board or a “psychic’s con” than a source of truth, Pierre notes.
Still, the UCSF team thinks chat logs may hold clues to understanding AI-associated psychosis – and could help the industry create guardrails.
Guardrails for kids and teens
Sarma, Pierre, and UCSF colleagues will team up with Stanford University scientists to conduct one of the first studies to review the chat logs of patients experiencing mental illness. As part of the research set to launch later this year, UCSF and Stanford teams will analyse these chat logs, comparing them with patterns in patients’ mental health history and treatment records to understand how the use of AI chatbots among people experiencing mental illness may shape their outcomes.
“What I’m hoping our study can uncover is whether there is a way to use logs to understand who is experiencing an acute mental health care crisis and find markers in chat logs that could be predictive of that,” Sarma explains. “Companies could potentially use those markers to build-in guardrails that would, for instance, enable them to restrict access to chatbots or – in the case of children – alert parents.”
He continues, “We need data to establish those decision points.”
In the meantime, the pair says the use of AI chatbots is something health care providers should ask about and that patients should raise during doctor visits.
“Talk to your physician about what you’re talking about with AI,” Sarma says. “I know sometimes patients are worried about being judged, but the safest and healthiest relationship to have with your provider is one of openness and honesty.”
By Henry Adams, Country Manager, InterSystems South Africa
Across South Africa, nurses and doctors in public clinics make hundreds of important decisions every day, often under enormous pressure. They’re short on time, juggling long queues, and sometimes working with incomplete information. In those conditions, even the most experienced professionals can make mistakes. It’s human.
The truth is, our healthcare system is stretched thin, and people can only do so much. That’s why I see real potential for AI to step in as a kind of virtual pharmacist. Not to replace anyone, but to back them up by checking prescriptions, catching errors, and helping ensure patients get the right treatment quickly and safely.
From data to decision support
I’m often asked how AI can make a real difference in healthcare right now. One area where it can have an immediate impact is in prescriptions. AI-assisted systems help doctors and nurses make safer, faster decisions by analysing medical data in real time. They can check a patient’s history, allergies, and possible drug interactions in seconds, flagging risks before they become problems.
Of course, because we’re dealing with sensitive medical information, trust and data quality are crucial. These systems only work when they’re built on accurate, connected data that healthcare professionals can rely on.
That’s where the latest health technology partnerships come in. By linking proven data platforms with smart AI tools, we’re already seeing real improvements overseas. In Europe, for example, these systems are helping clinicians catch potential drug errors early and prescribe with greater confidence.
There’s no reason South Africa can’t benefit in the same way. With clinics under pressure and resources stretched, technology that connects clean, reliable data with practical AI support could help reduce errors, save time, and make care safer for everyone.
Addressing local challenges
Medication errors can happen anywhere, but in South Africa the stakes are often higher. Our public clinics are exceptionally busy, staff are stretched, and doctors and nurses are doing their best under tough conditions. When you’re working under that kind of pressure, even a small mistake in a prescription can have serious consequences for a patient.
This is where AI can really help. Imagine a system that double-checks every prescription in real time, flagging possible drug interactions, incorrect dosages, or missing information before the medicine ever reaches the patient. It’s like having an extra set of expert eyes that never get tired. Instead of slowing things down, it speeds them up and gives clinicians peace of mind knowing they’re making the safest call for each patient.
For that to work, though, the data behind the system must be reliable and up to date. As South Africa moves toward a unified digital health record, the ability for these systems to connect to existing patient information becomes crucial. When healthcare professionals can trust the data they see on screen, AI becomes a genuine partner in care, helping them work faster, smarter, and safer.
Building confidence in AI
For AI to really work in healthcare, it must be clear and trustworthy. Doctors and nurses need to know why the system is recommending a specific drug or warning about a potential issue. If it can’t explain itself, people won’t use it, and rightly so.
That’s why transparency matters. The best AI tools don’t make decisions behind closed doors; they show their reasoning and help clinicians understand what’s happening in the background. When that’s combined with reliable, well-managed data, you start to build real confidence in the system.
It’s that trust, knowing the technology supports rather than replaces clinical judgment, that will make AI-assisted prescriptions part of everyday care, not just an interesting experiment.
A collaborative path forward
Technology on its own won’t fix South Africa’s healthcare challenges, but it can make a big difference in helping people do their jobs better. AI-assisted prescriptions are a good example of how smart tools can take some of the pressure off clinicians, reduce paperwork, and help patients get safer, faster care.
What excites me most is how practical this can be. Picture a nurse in a rural clinic who needs to prescribe medication but doesn’t have easy access to a specialist. With AI support, she can get accurate, instant guidance and know her patient is getting the right treatment. Or think about a busy hospital pharmacy, where an AI system automatically checks for drug interactions across hundreds of files in seconds, preventing errors before they happen.
This isn’t some far-off idea. The technology already exists and is being used successfully elsewhere. The goal now is to make sure it’s used in a way that supports our healthcare professionals, not replaces them. They are, and always will be, at the centre of care. If we get this right, AI can become a real partner in healthcare.
As artificial intelligence (AI) increasingly enters the mental health space, from therapy chatbots to diagnostic tools, the world faces a critical question: can AI expand access to care without putting people at risk?
At the G20 Social Summit in Johannesburg, South Africa announced a landmark national effort to answer that question. The South African Health Products Regulatory Authority (SAHPRA) and PATH, with funding from Wellcome, have launched the Comprehensive AI Regulation and Evaluation for Mental Health (CARE MH) program to develop the world’s first regulatory framework for artificial intelligence in mental health.
CARE MH will establish a science-based and ethically robust regulatory framework that describes how AI tools need to be evaluated for safety, inclusivity, and effectiveness before they can be given market authorization and made available to potential service users. It aims to strengthen trust in digital health innovation and will serve as a model for other countries seeking to strike a balance between innovation and oversight.
“You wouldn’t give your child or loved one a vaccine or drug that hadn’t been tested or evaluated for safety,” saidBilal Mateen, Chief AI Officer at PATH. “We’re working to bring that same standard of rigorous evaluation to AI tools in mental health, because trust must be earned, not assumed.”
The framework will be developed and tested in South Africa, with the intention of extending its application across the African continent and to international partners.
“SAHPRA is proud to lead the development of Africa’s first regulatory framework for AI in mental health linked directly to market authorization,” said Christelna Reynecke, Chief Operations Officer of SAHPRA. “Our true goal is even more ambitious, though; we want to create a regulatory environment for AI4health in general, one that keeps pace with innovation, grounded in scientific rigor, ethical oversight, and public accountability.”
“Millions of people across the globe are being held back by mental health problems, which are projected to become the world’s biggest health burden by 2030,” said Professor Miranda Wolpert MBE, Director of Mental Health at Wellcome. “CARE MH is a vital step toward ensuring that AI technologies in this space are safe, effective, and equitable.”
The goal is simple: help more people, safely.
Through CARE MH, the partners behind this initiative are setting the foundation for the next generation of ethical, evidence-based AI in mental health. Supported by global experts from the following institutions: Audere Africa, African Health Research Institute, the UK’s Centre for Excellence in Regulatory Science and Innovation for AI & Digital Health, the UK Medicines and Healthcare products Regulatory Agency, University of Birmingham, University of Washington, and the Wits Health Consortium, CARE MH is built to protect and empower people everywhere.
By Vishal Barapatre, Group Chief Technology Officer at In2IT Technologies
Artificial Intelligence (AI) is revolutionising healthcare as profoundly as the discovery of antibiotics or the invention of the stethoscope. From analysing X-rays in seconds to predicting disease outbreaks and tailoring treatment plans to individual patients, AI has opened new possibilities for precision medicine and increased efficiency. In emergency rooms, AI-driven diagnostic tools are already helping doctors detect heart attacks or strokes faster than human eyes alone.
However, as AI systems become increasingly embedded in the patient journey, from diagnosis to aftercare, they raise critical ethical questions. Who is accountable when an algorithm gets it wrong? How can we ensure that patient data remains confidential in the era of cloud computing? And how can healthcare institutions, often stretched thin on resources, balance innovation with responsibility?
When algorithms diagnose: the promise and the problem
AI’s strength lies in its ability to process massive amounts of data, such as medical histories, imaging scans, and lab results, and detect patterns that human clinicians might miss. This can dramatically improve diagnostic accuracy and treatment outcomes. For instance, AI models trained on thousands of mammogram images can help identify subtle indicators of breast cancer earlier than traditional methods.
However, the same data that powers AI can also introduce bias. If the datasets used to train an algorithm are skewed, say, over-representing one demographic group, the results may unfairly disadvantage others. A diagnostic model trained primarily on data from urban hospitals, for example, might misinterpret symptoms in patients from rural areas or underrepresented ethnic groups. Bias in healthcare AI isn’t just a technical flaw; it’s an ethical hazard with real-world consequences for patient trust and equity.
The privacy paradox
The integration of AI in healthcare requires access to vast quantities of sensitive data. This creates a privacy paradox: the more data AI consumes, the smarter it becomes, but the greater the risk to patient confidentiality. The digitisation of health records, combined with AI’s hunger for data, exposes systems to new vulnerabilities. A single breach can compromise thousands of medical histories, potentially leading to identity theft or misuse of personal health information. The paradox underscores the need for robust data protection measures in AI-driven healthcare systems.
Striking a balance between data utility and privacy protection has become one of the healthcare industry’s most pressing ethical dilemmas. Encryption, anonymisation, and strict access controls are essential, but technology alone isn’t enough. Patients need transparency: clear explanations of how their data is used, who has access to it, and what safeguards are in place. Ethical AI requires not only compliance with regulations but also the cultivation of trust through open communication.
Accountability in the age of automation
When an AI system makes a medical recommendation, who is ultimately responsible for the outcome – the algorithm’s developer, the healthcare provider, or the institution that deployed it? The opacity of AI decision-making, often referred to as the “black box” problem, complicates accountability and transparency. Clinicians may rely on algorithmic outputs without fully understanding how conclusions were reached. This can blur the line between human and machine judgment.
Accountability must therefore be clearly defined. Human oversight should remain central to any AI-powered decision, ensuring that technology supports rather than replaces clinical expertise. Ethical frameworks that mandate explainability, where AI systems must provide understandable reasoning for their outputs, are key to maintaining trust. Moreover, continuous auditing of AI models, which involves regularly reviewing and testing the system performance, can help detect and correct biases or errors before they lead to harm, thereby ensuring the ongoing ethical use of AI in healthcare.
Behind the code: who keeps AI ethical
While hospitals and clinics focus on patient care, many lack the internal capacity to manage the complex ethical, security, and technical demands of AI adoption. This is where third-party IT providers play a pivotal role. These partners act as the backbone of responsible innovation, ensuring that AI systems are implemented securely and ethically.
By embedding ethical principles into system design, such as fairness, transparency, and accountability, IT providers help healthcare institutions mitigate risks before they become crises. They also play a crucial role in securing sensitive data through advanced encryption protocols, cybersecurity monitoring, and compliance management. In many ways, they serve as both architects and custodians of ethical AI, ensuring that the pursuit of innovation does not compromise patient welfare.
Building a culture of ethical innovation
Ultimately, the ethics of AI in healthcare extend beyond technology; they are about culture and leadership. Hospitals and healthcare networks must foster environments where ethical reflection is as integral as technical innovation. This involves establishing multidisciplinary ethics committees, conducting bias audits, and training clinicians to critically evaluate and question AI outputs rather than accepting them without examination.
The future of AI in healthcare depends not on how advanced our algorithms become, but on how wisely we use them. Ethical frameworks, transparent governance, and responsible partnerships with IT providers can transform AI from a potential risk into a powerful ally. As the healthcare sector continues to evolve, the institutions that will thrive are those that remember that technology should serve humanity, not the other way around.
By Henry Adams, Country Manager, InterSystems South Africa
When people think about artificial intelligence (AI) in healthcare, they often picture complex machines in high-tech hospitals. But some of the most exciting uses of AI are happening in primary care, right at the first point of contact between doctor and patient.
Globally, AI is helping general practitioners, nurses, and clinicians make faster, more accurate decisions by giving them access to clean, connected data. It helps detect early signs of disease, spot patterns across patient populations, and ensure the right people get the right care sooner.
South Africa is not there yet, but that is exactly why we should be paying attention.
Learning from what is working elsewhere
In countries where healthcare data is already digitised and connected, AI-assisted tools are starting to prove their worth. In parts of Europe, AI systems are helping GPs analyse symptoms, lab results and patient histories to identify possible conditions much earlier. In the US, data platforms are used to surface insights from millions of patient records, helping clinicians identify patterns that might otherwise go unnoticed.
At InterSystems, we have seen firsthand how this combination of reliable data and intelligent technology is changing the way care is delivered. In the UK, our data platform helps care providers securely connect across places of care to patient information across multiple systems, making it easier for AI tools to interpret symptoms in context. In France, AI-assisted prescriptions through partners like Posos are helping doctors reduce errors and improve treatment safety.
These examples show what is possible when data, people and technology come together in the right way.
Why data comes first
AI is only as powerful as the data it works with. If a clinician’s system lacks complete or up-to-date patient information, the AI cannot provide reliable support. That is why data quality and interoperability are so important; they form the foundation for everything else.
Many countries that are seeing success with AI in primary care started by getting their data in order, building connected health records, standardising information, and ensuring privacy and compliance at every step. Once those pieces were in place, they could start introducing AI tools that help doctors and nurses make better decisions without adding extra admin or complexity.
Again, in South Africa, we are not quite there yet, but we are heading in the right direction. There are ongoing efforts to digitise health records and bring together fragmented systems. As that process continues, it will open the door for more advanced AI-driven support tools, from diagnosis assistance to population health management.
What this could mean for South Africa
Imagine a community clinic in Limpopo or the Eastern Cape, where a doctor sees dozens of patients a day. With AI support, they could instantly access each patient’s medical history, flag high-risk symptoms, or receive early alerts about potential complications like diabetes or hypertension.
AI will not replace the doctor’s or their judgment. It simply gives them more context and better information. It is like having a quiet assistant in the background, helping spot what is easy to miss when you are under pressure.
This kind of technology could also help identify broader health trends, guiding public health decisions and making sure resources are sent where they are needed most. It is not about high-end tech for big hospitals, it is about making everyday healthcare smarter, safer and more efficient for everyone.
Building the foundations
Before we can get there, we need to focus on the basics: connected systems, reliable data, and trust. AI tools cannot function properly in silos. They need access to consistent, secure information, the kind that interoperable platforms like InterSystems IRIS for Health are designed to manage.
Once we have that in place, the rest becomes achievable. Doctors can use AI to compare patient data against proven medical knowledge bases. Clinics can share insights securely across regions. And the healthcare system becomes more proactive instead of reactive.
It is easy to look at what is happening overseas and feel that South Africa is far behind. But I see it differently. Every success story abroad gives us a roadmap, lessons we can adapt to our own realities. We do not have to reinvent the wheel; we just have to make sure it is fit for our local terrain.
Investigators trained multiple artificial intelligence models to read images from echocardiograms, a type of ultrasound test that evaluates the structure and function of the heart. Their goal was to determine whether AI could use these images to calculate measurements like inflammation and scarring that are normally obtained through another, more costly test called cardiac magnetic resonance imaging (CMRI). By examining findings from 1453 patients who had undergone both tests, they found the AI models could not accomplish this task.
“As compared to echocardiograms, cardiac MRI machines are expensive and not available for many patients, especially those in rural areas, so we had hoped that AI could reduce the need for it,” said Alan Kwan, MD, assistant professor in the Department of Cardiology in the Smidt Heart Institute at Cedars-Sinai and co-senior author of the study. “Our results showed the limited powers of AI in this area.”
HealthTech is transforming healthcare through AI, mobile applications, wearable devices, telemedicine, and big data analytics. While these advances offer enormous potential to improve patient outcomes and operational efficiency, they also raise complex legal and regulatory challenges – spanning intellectual property, data privacy, licensing, corporate governance, funding, taxation, and litigation.
Webber Wentzel’s Navigating HealthTech Legal Solutions highlights the firm’s extensive experience in helping innovators, investors, and healthcare providers across Africa address the legal and regulatory complexities of HealthTech. Mapping out the complexities at play across both the technology and the law, this resource brings together Webber Wentzel’s cross-practice teams to give clients a holistic perspective on opportunities, risks, and emerging trends in healthcare innovation.
“Our clients are leading the way in healthcare innovation, and they need legal partners who understand the sector end-to-end,” says Bernadette Versfeld, head of the Consumer sector. “This resource demonstrates how we help businesses navigate regulatory hurdles, adopt new technologies, structure investments effectively, and manage risk, all while enabling growth and innovation.”
Drawing on extensive experience working with healthcare companies, insurers, tech providers, investors, and regulators across Africa, the report provides insights into medical device licensing, HealthTech investment structuring, protecting personal health data, managing litigation risks, and compliance with South Africa’s National Health Insurance Act.
“As part of our ongoing commitment to supporting Africa’s healthcare sector, Webber Wentzel continues to advise on emerging trends, innovative technologies, and regulatory developments. By combining deep sector knowledge with cross-practice expertise, we help clients not just respond to change but shape it, empowering them to navigate the complex intersection of healthcare and technology,” adds Versfeld.
Doctors who use artificial intelligence at work risk having their colleagues deem them less competent for it, according to a recent Johns Hopkins University study.
While generative AI holds significant promise for advancing health care, a new study finds its use in medical decision-making impacts how physicians are perceived by their colleagues. The research shows that doctors who primarily rely on generative AI for decision-making face considerable scepticism from fellow clinicians, who correlate their use of AI with a lack of clinical skill and overall competence, resulting in a diminished perceived quality of patient care.
The research included a diverse group of clinicians from a major hospital system, involving attending physicians, residents, fellows, and advanced practice providers. Results of the study were published in Nature Digital Medicine.
Stigma stunts better care
The findings may indicate a social barrier to AI adoption in health care settings, which could slow advances that might improve patient care.
“AI is already unmistakably part of medicine,” says Tinglong Dai, professor of business at the Johns Hopkins Carey Business School and co-corresponding author of the study. “What surprised us is that doctors who use it in making medical decisions can be perceived by their peers as less capable. That kind of stigma, not the technology itself, may be an obstacle to better care.”
The study, conducted by researchers at Johns Hopkins University, involved a randomised experiment where 276 practicing clinicians evaluated different scenarios: a physician using no AI, one using AI as a primary decision-making tool, and another using it for verification. The research found that as physicians were more dependent on AI, they faced an increasing “competence penalty,” meaning they were viewed more sceptically by their peers than those physicians who did not rely on AI.
“In the age of AI, human psychology remains the ultimate variable,” says Haiyang Yang, first author of the study and academic program director of the Masters of Science in Management program at the Carey Business School. “The way people perceive AI use can matter just as much as, or even more than, the performance of the technology itself.”
Skipping AI equalled more respect
According to the study, peer perception suffers for doctors who rely on AI. Framing generative AI as a “second opinion” or a verification tool partially improved negative perceptions from peers, but it did not fully eliminate them. Not using GenAI, however, resulted in the most favourable peer perceptions.
The findings align with theories that suggest perceived dependence on an external source like AI can be seen as a weakness by clinicians.
Ironically, while GenAI’s visible use can undermine a physician’s perceived clinical expertise among peers, the study also found that clinicians still recognise AI as a beneficial tool for enhancing precision in clinical assessment. The research showed that clinicians still generally acknowledge the value of GenAI for improving the accuracy of clinical assessments, and they view institutionally customized GenAI as even more useful.
The collaborative nature of the study led to thoughtful suggestions for GenAI implementation in health care settings, which are crucial to balance innovation with maintaining professional trust and physician reputation, the researchers note.
“Physicians place a high value on clinical expertise, and as AI becomes part of the future of medicine, it’s important to recognise its potential to complement – not replace – clinical judgment, ultimately strengthening decision making and improving patient care,” said Risa Wolf, co-corresponding author of the research and associate professor of pediatric endocrinology at Johns Hopkins School of Medicine with a joint appointment at the Carey Business School.
Study has implications beyond medical education, suggesting other fields could benefit from AI-enhanced training
Artificial intelligence (AI) is becoming a powerful new tool in training and education, including in the field of neurosurgery. Yet a new study suggests that AI tutoring provides better results when paired with human instruction.
Researchers at the Neurosurgical Simulation and Artificial Intelligence Learning Centre at The Neuro (Montreal Neurological Institute-Hospital) of McGill University are studying how AI and virtual reality (VR) can improve the training and performance of brain surgeons. They simulate brain surgeries using VR, monitor students’ performance using AI and provide continuous verbal feedback on how students can improve performance and prevent errors. Previous research has shown that an intelligent tutoring system powered by AI developed at the Centre outperformed expert human teachers, but these instructors were not provided with trainee AI performance data.
In their most recent study, published in JAMA Surgery, the researchers recruited 87 medical students from four Quebec medical schools and divided them into three groups: one trained with AI-only verbal feedback, one with expert instructor feedback, and one with expert feedback informed by real-time AI performance data. The team recorded the students’ performance, including how well and how quickly their surgical skills improved while undergoing the different types of training.
They found that students receiving AI-augmented, personalised feedback from a human instructor outperformed both other groups in surgical performance and skill transfer. This group also demonstrated significantly better risk management for bleeding and tissue injury – two critical measures of surgical expertise. The study suggests that while intelligent tutoring systems can provide standardised, data-driven assessments, the integration of human expertise enhances engagement and ensures that feedback is contextualised and adaptive.
“Our findings underscore the importance of human input in AI-driven surgical education,” said lead study author Bianca Giglio. “When expert instructors used AI performance data to deliver tailored, real-time feedback, trainees learned faster and transferred their skills more effectively.”
While this study was specific to neurosurgical training, its findings could carry over to other professions where students must acquire highly technical and complex skills in high-pressure environments.
“AI is not replacing educators – it’s empowering them,” added senior author Dr Rolando Del Maestro, a neurosurgeon and current Director of the Centre. “By merging AI’s analytical power with the critical guidance of experienced instructors, we are moving closer to creating the ‘Intelligent Operating Room’ of the future capable of assessing and training learners while minimising errors during human surgical procedures.”
