Early applications of ChatGPT in medical practice, education and research

Abstract
ChatGPT, which can automatically generate written responses to queries using internet sources, soon went viral after its release at the end of 2022. The performance of ChatGPT on medical exams shows results near the passing threshold, making it comparable to third-year medical students. It can also write academic abstracts or reviews at an acceptable level. However, it is not clear how ChatGPT deals with harmful content, misinformation or plagiarism; therefore, authors using ChatGPT professionally for academic writing should be cautious. ChatGPT also has the potential to facilitate the interaction between healthcare providers and patients in various ways. However, sophisticated tasks such as understanding the human anatomy are still a limitation of ChatGPT. ChatGPT can simplify radiological reports, but the possibility of incorrect statements and missing medical information remain. Although ChatGPT has the potential to change medical practice, education and research, further improvements of this application are needed for regular use in medicine.
Introduction
ChatGPT (OpenAI, San Francisco, CA, USA) is an AI chatbot which was introduced in November 20221 and soon after went viral. ChatGPT has the ability to respond to various kinds of queries, automatically generating responses using internet sources. People across different fields, generations and continents started using ChatGPT,2 leading to a continuous increase in its popularity. Medicine is a field in which simplifying artificial intelligence (AI)-based technologies are highly important. It is obvious that applications such as ChatGPT have the potential to change medicine, with uses ranging from the automated extraction of electronic medical records3 to the development of sophisticated treatment plans. This article presents an overview of the early applications of ChatGPT in medicine.
ChatGPT in medical education
A recent study evaluated the performance of ChatGPT on the United States Medical Licensing Examination (USMLE). The study revealed that ChatGPT passed all three exams (Step 1, Step 2 CK, Step 3) near the passing threshold without any previous training.4 On the other hand, Gilson et al state that there is a significant decrease in performance with an increased difficulty of the questions. However, the authors compare the performance of ChatGPT to a third-year medical student.5 Antaki et al tested ChatGPT for use in two multiple-choice question banks for the Ophthalmic Knowledge Assessment Program (OKAP) exam. The authors found similar results, with ChatGPT achieving 55.8% and 42.7% accuracy in the exams.6 Another study from Korea aimed at directly correlating the knowledge of Chat-GPT to that of medical students on the topic of parasitology. The authors revealed that the performance of ChatGPT was lower than medical students and concluded that ChatGPT's ability is not yet at an entirely acceptable level.7
Academic writing
Gao et al tested ChatGPT's ability to write academic abstracts. They included 50 abstracts from five high-impact medical journals and asked ChatGPT to produce research abstracts using provided titles and journal requirements. The authors concluded that all ChatGPT-derived abstracts were acceptably written, but only 8% of them respected the formatting requirements of the journals. 68% of the generated abstracts by ChatGPT were correctly identified by the reviewers due to the ‘vaguer’ and more ‘formulaic’ type of writing. An AI output detector showed similar results in detecting the ChatGPT-derived abstracts.8
In a recent article by Guo et al,9 the main attributes of ChatGPT's writing style are identified. The authors mention that ChatGPT writes in an organised manner and prefers a straightforward concept in the questions. Its answers are long and detailed, it shows less harmful information, and refuses to answer when it has no information about topics. However, it might ‘fabricate facts’ to give an answer, which should make users cautious regarding the professional use of ChatGPT. The authors observed that the main difference between ChatGPT's writing style and human writing is that the human writer is more subjective, colloquial and emotional, giving human abstracts a personal note.9 OpenAI warns that ChatGPT could sometimes ‘responds to harmful instructions’.10,11
A recent study evaluated ChatGPT's ability to generate a literature review on the concept of the ‘digital twin’ in healthcare, asking it to paraphrase selected literature from 2020 to 2022. Although the results were promising, the iThenticate plagiarism detection tool identified many plagiarism matches.12
Interaction with patients and radiological reporting
Thurzo et al reviewed AI-based applications, including ChatGPT, in the dental field. They concluded that ChatGPT could facilitate the interaction between healthcare providers and patients in various ways, from analysing patient messages to personalising the communication between healthcare professionals and patients. However, they found that ChatGPT has limitations in relation to sophisticated tasks such as understanding the human anatomy.13 Nov et al tested ChatGPT against healthcare providers' responses to patients. They found that ChatGPT had a similar rate of correct answers compared to the providers.14 In a case study conducted by Jeblick et al,15 radiologists had the task of evaluating the quality of simplified radiology reports generated with ChatGPT. The results showed that the reports were ‘correct, complete, and not potentially harmful to patients’. However, incorrect statements and missing medical information that could potentially have led to harmful conclusions were also detected. Although this case study comprises small sample numbers, the authors emphasise the great potential of ChatGPT in radiology while also mentioning the need for further improvements.15
Conclusion
ChatGPT seems to fulfil a long-held desire to simplify medical practice, education and research. However, ChatGPT is still a very novel and early-stage application which needs further improvements to be widely usable in medicine. Although ChatGPT is a highly sophisticated application, which could change medical practice, research and education substantially, the last instance should remain human judgment.
- © Royal College of Physicians 2023. All rights reserved.
References
- ↵
- Liebrenz M
- ↵
- Aydın Ö
- ↵
- Biswas S
- ↵
- Kung TH
- ↵
- Gilson A
- ↵
- Antaki F
- ↵
- ↵
- Gao CA
- ↵
- Guo B
- ↵
- Kitamura FC
- ↵
- OpenAI TB
- ↵
- Aydın Ö
- ↵
- Thurzo A
- ↵
- Nov O
- ↵
- Jeblick K
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- No citing articles found.