From Image to Insight; Can Vision Language Models see like Radiologists?

Background

Diagnosing liver lesions using multiparametric MRI requires detecting complex, often subtle features - such as washout, fat content, capsular enhancement, and iron deposition - However, these features can be subtle and challenging to detect, often requiring the expertise of experienced radiologists, which limits the scalability of supervised AI approaches.

With the rise of vision-language models (VLMs) like GPT-4V and DeepSeek-VL, a key question is whether such models can reliably identify and describe these features across diverse MRI sequences. This project explores the diagnostic potential of VLMs as assistive tools, with a particular focus on benchmarking their ability to “see” and reason over relevant imaging features in comparison to expert radiologist interpretations and pathology-confirmed diagnoses.

Aim

The primary goal of this projectis to systematically evaluate how well these models extract and describe clinically meaningful features from annotated MRI data, and to what extent their predictions aligns with real-world diagnostic outcomes. Depending on results and interest, the project may also explore fine-tuning existing models to improve domain-specific performance, or - if performance proves reliable - developing a lightweight graphical interface to make these models more accessible and interpretable in a clinical context. This research is part of the Liver Artificial Intelligene - Consortium, a collaborative initiative focused on advancing AI models in liver imaging, bringing together interdisciplinary expertise to unlock new diagnostic possibilities.

Related research:

• https://doi.org/10.1016/j.jpi.2024.100368
• https://doi.org/10.1016/j.ajpath.2023.03.012

Supervisors

• Frederik Hartmann
• Ruben Niemantsverdriet
• Maarten Thomeer
• Stefan Klein
• Martijn Starmans

Feel free to mail me if you are interested in this project or want more information!