With the advent of large language models (LLMs), such as the well-known ChatGPT, there has been a surge of interest in how to leverage these technologies in healthcare. These queries are far from baseless, as LLMs have already demonstrated significant value in various non-clinical fields. It is entirely reasonable to explore their potential in medical imaging. The biomedical industry has begun to innovate and propose solutions based on the perceived needs of physicians and the medical imaging workforce, often from an engineering standpoint. However, LLMs offer a unique opportunity to develop solutions through a collaborative approach that includes both physicians and industry professionals.

In other words, only by integrating insights from clinicians can we ensure that the benefits of LLMs are realized in ways that genuinely enhance clinical practice. This collaborative approach is particularly relevant in the field of ultrasound imaging, where the unique real-time nature of the modality, combined with operator-dependent variability, presents both opportunities and challenges. This blog post explores the exciting possibilities of LLMs in ultrasound imaging through two specific approaches: scan-time AI assistance and review-time AI assistance.

The Dream: Scan-Time AI by Real-Time Integration of LLMs

Imagine having a smart assistant right by your side during an ultrasound exam, processing data in real time and offering insights instantaneously. This “scan-time AI” is not a distant dream but an emerging reality. By integrating LLMs into ultrasound machines, clinicians can receive immediate feedback on the screen. This AI-powered assistance can highlight areas of interest, suggest potential diagnoses, and recommend additional views or techniques to optimize image quality, making the diagnostic process more accurate and efficient.

However, the journey to seamless real-time AI integration comes with its own set of challenges. The primary hurdle is ensuring that the AI operates with split-second precision, as any lag could disrupt the examination flow. Additionally, the integration must be intuitive, ensuring that AI suggestions complement the clinician’s expertise without causing distraction. The ultimate goal is to create a harmonious partnership where AI augments the clinician’s skills and enhances patient care.

As their name implies, LLMs are designed to communicate with language at the center. Early examples include chat-like communication with the user, which, at first glance, may not seem viable for medical imaging workflows. However, LLM literature is advancing very rapidly, and with the invention of multi-modal LLMs, communication with ultrasound systems will no longer be limited to text but also extend to other modalities such as voice and images. Voice commands can streamline the process, allowing clinicians to focus on the patient and the probe without needing to manipulate controls manually. For instance, a clinician could say, “Compare the thickness of the renal cortex with the medulla” and the ultrasound machine would reason through the command, detect the said anatomical structures, perform the measurement, and display the results, thus improving efficiency and ergonomics. However, voice interaction in a clinical environment brings its own set of complexities. The bustling background noise, the need for precise and unambiguous commands, and the potential for AI misinterpretation are significant factors to consider. Furthermore, voice interaction must be evaluated for its impact on privacy within the clinical setting. When these issues with voice communication in clinical settings are addressed, using LLMs through voice commands for ultrasound examinations will become much smoother and more efficient.

We’re There: Review-Time AI for Post-Examination Analysis

While real-time AI offers immediate benefits during the scan, “review-time AI” focuses on the critical post-scan phase. LLMs can meticulously review ultrasound images and generate detailed reports, highlighting key findings and suggesting differential diagnoses. This application can significantly alleviate the documentation burden on clinicians, allowing them to dedicate more time to patient care.

The necessity for LLMs in review-time AI stems from the sheer volume and complexity of data clinicians must analyze. By automating the initial review and providing structured reports, LLMs enhance the consistency and quality of ultrasound interpretations. This approach also facilitates collaborative care, as AI-generated reports can be easily shared and reviewed by other specialists, ensuring a comprehensive evaluation of the patient’s condition.

A Call to Action for Physicians

Physicians play a pivotal role in shaping the future of AI technologies. While engineers and data scientists provide the technical backbone, clinicians’ insights and feedback are crucial in developing AI systems that truly address healthcare needs. Physicians are encouraged to experiment with these new-age AI tools in their daily routines, providing critical feedback that will steer the evolution of AI in a direction that genuinely enhances clinical practice.

Integrating LLMs into ultrasound imaging is not merely a technological advancement but a paradigm shift that requires active collaboration between clinicians and technologists. By exploring the exciting possibilities of scan-time and review-time AI and addressing the challenges of voice interaction, we can pave the way for a more efficient and accurate diagnostic process. Physicians, your involvement and insights are crucial. Together, we can shape a future where AI not only complements but also elevates the art and science of ultrasound imaging. Let’s embrace this transformative journey and lead the way to a new era of medical innovation.