Brent Fox Auburn University School of Pharmacy Artificial Intelligence Chatbots: Applications in HealthcareIn the last issue, I introduced — and hopefully clarified — large language models and artificial intelligence (AI) chatbots, which collectively are the most talked about technologies today. For those who may have missed the last issue, here is a very brief review. Chatbots are a type of artificial intelligence that we interact with when we use the “Help” feature on a website. The responder on the other end of our question is a computer that uses the text of our question to make an intelligent guess of what we want to know.

This type of chatbot has a limited number of questions/prompts to which it can respond. Large language models (LLMs) enable artificial intelligence chatbots like ChatGPT and Med-PaLM (more to come) to have conversational interactions with individuals via submitted questions. Compared to the typical help feature chatbots, large language models enable ChatGPT and Med-PaLM to respond to a much greater and more complicated range of questions/requests. If you are interested in digging deeper into the details of large language models, check the May/June issue of ComputerTalk.

Using 195 questions submitted by unique individuals to an online social media forum to which a verified, unique physician had previously responded, the researchers compared the physician’s original response to a response generated by ChatGPT (version 3.5).

Responses were compared on the basis of the quality of the response and the empathy provided in the response. In nearly 79% of cases, evaluators preferred the chatbot responses over physician responses. In terms of assessment of quality, chatbot responses were rated an average 4.13, while physicians’ responses were rated an average 3.26 (1-5 scale, with 5 being “Very Good”). Looking at empathy ratings, chatbot responses averaged 3.65, while physician responses averaged 2.15 (1–5 scale, with 5 being “Very Empathetic”).

The study described above provides an interesting starting point to consider potential roles of chatbots (enabled by large language models) as tools to support healthcare professionals. Maybe a chatbot can provide an initial response to a patient inquiry. The patient’s provider then reviews/edits/approves of the response, as deemed appropriate. A review of some existing eforts to integrate ChatGPT into healthcare provides insight into the potential future of this technology. However, before I dig into existing eforts, it is worth noting that the evolving nature of both the technology and its uses in healthcare point to an uncertain intersection of the technology and concerns over patient privacy.

AI AND CHATBOT INTEGRATIONS WITH HEALTHCARE

A review of existing ChatGPT integrations in healthcare begins with medical transcription. Consider a medical transcription tool for the range of patient encounters. It enables a highly automated method to address a time-intensive task for physicians — documentation. Existing use cases include transcription of in-person and remote patient interactions. Another existing use of ChatGPT provides clinical assessments based on patient-specific information and primary literature, providing a mapped presentation of relevant clinical information. The goal is to present a clinical patient summary to the provider, who can then use it in their patient care encounters.


A sample page from Med-PaLM (https://sites.research.google/ med-palm/). Where might AI chatbots "t in healthcare and patient care?
A sample page from Med-PaLM. Where might AI chatbots “t in healthcare and patient care?

Another current use of ChatGPT technology includes personalized app-based coaching to help with intermittent fasting. An additional, very interesting use of ChatGPT enables individuals to ask questions about their health risks, medications, etc., and receive responses based on their genetic profile. Certainly, this last example raises the concerns over patient privacy mentioned above. Sticking with the healthcare sector, large language models powering AI chatbots like ChatGPT are trained using extremely large amounts of internet-based text.

The content and topical areas of this text range across a variety of domains and are not specific to healthcare. However, Med-PaLM from Google is intended for use in the healthcare domain. According to Google Research, Med-PaLM is “designed to provide high quality answers to medical questions” (see image below). While Google does not — understandably — explicitly describe how they designed Med-PaLM for the healthcare domain, the company does indicate that large language models for healthcare require a focus on safety, equity, and bias to protect patients.

In one efort to assess accuracy, Google’s Med-PaLM 2 answered a series of United States Medical Licensing Examination-style questions. The USMLE is the licensure examination for medical doctors in the United States. According to Google, Med-PaLM 2 scored greater than 85% on the questions, which was a 19% increase over Med-PaLM. Both versions of Med-PaLM exceeded the passing score of 60%.

A commonly cited use case for AI chatbots in healthcare is to research medical conditions or questions. Considering the vast amount of medical information found in journals, textbooks, and similar authoritative sources, this use case makes sense and is highlighted by the JAMA Internal Medicine example above.

But what about other opportunities? Where might AI chatbots like Med-PaLM fit? At the patient level, many suggest that the accuracy and efficiency of diagnosing medical conditions can be improved. In this space, we have previously written about a movement in healthcare to open the medical chart to patients. Similarly, AI chatbots may be leveraged as tools to provide patients with detailed information about their medical diagnoses and conditions, as well as their medications.

Precision medicine is intended to use a patient’s unique genetic information to identify and select targeted medical treatments. Large language models that power AI chatbots like Med-PaLM 2 are envisioned as enablers of precision medicine due to their ability to identify patterns in large data sets. Speaking of large data sets, the public health sector is also a potential area of contribution. Here, the LLM analyzes vast amounts of public health data to identify relevant health trends or patterns.

The list of expected contributions for LLMs in healthcare extends beyond those provided here. ComputerTalk readers can probably name many potential areas where LLMs can be leveraged in healthcare. However, it is important to note that this is an emerging technology whose use in the healthcare space requires extensive real-world testing and validation.

A single error in a patient care scenario can have signi!cant — and potentially fatal — effects on the patient. One area of low-hanging fruit for LLMs in healthcare is summarizing the vast amounts of medical literature. Even in that use case, where is the liability if an error in summarization leads to a patient care decision that has a negative impact on a patient? Despite the clear questions in front of us, the potential applications and exploration of LLMs in healthcare are not going away. I am looking forward to seeing where this technology fits in the healthcare landscape. Please feel free to send me questions or comments. CT