MedXpert AI: Multimodal Clinical Decision Assistant
Author : Ravi Surya, Dr. Ravi Babu G, Shamith D Bhat, Vashishta P and Suhas Y Gowda
Abstract :
Modern healthcare faces significant challenges regarding diagnostic errors, often stemming from the fragmented analysis of textual patient symptoms and radiology images, as existing systems typically process these modalities in isolation. MedXpert AI addresses this gap by providing a multimodal clinical decision assistant that synthesizes ClinicalBERT for symptom text analysis and DenseNet121/Vision Transformers for radiology imaging to deliver comprehensive diagnoses. This system fuses distinct data modalities using a transformer-based architecture and ensures clinical trustworthiness by visualizing attention weights to explain predictions, while integrating Gemini 1.5 Flash as a robust fallback agent for low-confidence cases and detailed, nonprescriptive recommendations. By effectively combining visual and textual data, the system demonstrates improved diagnostic reliability compared to unimodal approaches, establishing itself as a promising tool for enhancing decision-making in clinical and telemedicine environments.
Keywords :
ClinicalBert, DenseNet121/Vision Transformers Unimodal Approaches, Gemini 1.5 Flash.
