Sonam Vadera
Fellow in Clinical AI, Cohort 2
Fellowship Bio
Clinically, I work as a musculoskeletal radiologist. My research with the UHB AI and Digital Health Team focuses on detecting and analysing AI errors, aiming to monitor AI performance and safety post-deployment. I am involved in validating AIaMD technologies throughout the development-to-deployment pipeline using a collaborative multi-stakeholder approach.
Fellowship Project
Local validation of a commercial AI system for autonomous reporting of chest radiographs: a large-centre UK retrospective analysis
University Hospitals Birmingham NHS Foundation Trust
Amid ever-growing interest in the potential benefits of AI in healthcare, there is a critical need for standardised methods to monitor AI medical devices’ performance and safety. This project aimed to address this by applying the Medical Algorithmic Audit (MAA) framework—a comprehensive safety monitoring system designed to detect errors and failure modes in AI medical devices. The main goal was to use the MAA framework to validate the utility and safety of an AI tool that autonomously reports normal chest radiographs. This project also extended to other AI medical devices, including a skin cancer triage tool, leveraging the MAA to ensure their reliability and safety. In my role, I worked closely with the UHB Digital Transformation Team and led discussions and coordination with diverse stakeholders, ensuring a collaborative approach. I applied the MAA framework to various AI medical devices, and developed a failure modes and effects analysis. My clinical expertise was crucial in understanding the practical implications of these devices and how to integrate them into clinical workflows. Additionally, I engaged in legal discussions and worked closely with regulatory bodies to navigate the challenges of autonomous reporting in the NHS. Presenting my findings to various relevant bodies has been instrumental in fostering a collaborative approach to the implementation of these tools, and promoting shared learning and innovation. To date, over 140,000 chest radiographs have been processed and assessed for concordance between the radiologist and AI report. A case-by-case review of discordant cases is underway, and I am undertaking exploratory error and subgroup error analyses. Next steps include summarising findings and dissemination to key stakeholders to inform future implementations. This project not only aims to validate a specific AI tool but also sets the groundwork for a standardised evaluation pathway, ensuring that AI in healthcare remains safe, effective and equitable.
Fellowship Testimonial
I have had the privilege of working with one of the largest and most developed NHS AI evaluation teams in the UK, gaining extensive experience in bringing AI devices from the market to the NHS. I have firsthand experience of the challenges involved and have honed my problem-solving skills in this context. I have developed skills that equip me to lead on appropriate selection and procurement of effective AI medical devices, ensuring their safe implementation, and conducting robust post-deployment monitoring. My work has involved engaging with multiple regulatory bodies, including MHRA and CQC. Access to workshops, conferences, and research/policy projects within the wider UHB AI & Digital Healthcare Group has enabled me to participate in meetings with their international network of experts, collaborating with policymakers, institutional entrepreneurs, and academics. These interactions have deepened my understanding of the regulatory and innovation challenges and opportunities of AI in healthcare. The fellowship masterclasses have been pivotal in broadening my perspective on AI in healthcare from various stakeholders’ viewpoints. Further opportunities have included consulting a SME in early phases of AIaMD development. This experience provided insight into the challenges developers face and underscored the importance of clinician input to address the unique needs of healthcare providers and patients. It highlighted the frequent gaps in understanding regulatory requirements, medical data intricacies, and the clinical relevance of AI medical devices. Ethical considerations and regulatory compliance are now integral to my approach to technology development, and I am confident in discussing these topics with a diverse range of stakeholders. Overall, the fellowship has equipped me with a comprehensive understanding of effectively integrating AI into the complex and highly regulated healthcare sector. I am confident in my ability to navigate these complexities and guide the successful deployment of AI solutions in the NHS.