Exploration of Medicine

From: Bridging the validation gap in artificial intelligence in radiology

Table 1. Operational framework linking validation levels with study design, metrics, and clinical application.

Validation level	Typical study design	Key metrics	Example in radiology workflows
Technical validity	Retrospective dataset evaluation	Area under the curve (AUC), sensitivity, specificity	Detection of intracranial hemorrhage on CT
Workflow validity	Prospective or observational implementation studies	Reporting time, user interaction, and adoption rate	AI-based triage integrated into picture archiving and communication system (PACS), prioritizing urgent cases
Clinical validity	Prospective clinical studies or randomized trials	Change in management, diagnostic accuracy in practice, and patient outcomes	AI-assisted selection of patients for stroke thrombectomy

Return to article view

Declarations

Author contributions

ANB: Conceptualization, Investigation, Writing—original draft, Writing—review & editing. The author read and approved the submitted version.

Conflicts of interest

The author declares that there are no conflicts of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent to publication

Not applicable.

Availability of data and materials

Not applicable.

Funding

Not applicable.

Copyright

Publisher’s note

Open Exploration maintains a neutral stance on jurisdictional claims in published institutional affiliations and maps. All opinions expressed in this article are the personal views of the author(s) and do not represent the stance of the editorial team or the publisher.

References

Bhandari A. Revolutionizing Radiology With Artificial Intelligence. Cureus. 2024;16:e72646. [DOI] [PubMed] [PMC]

Warraich HJ, Tazbaz T, Califf RM. FDA Perspective on the Regulation of Artificial Intelligence in Health Care and Biomedicine. JAMA. 2025;333:241–7. [DOI] [PubMed]

Díaz O, Rodríguez-Ruíz A, Sechopoulos I. Artificial Intelligence for breast cancer detection: Technology, challenges, and prospects. Eur J Radiol. 2024;175:111457. [DOI] [PubMed]

Navarro-Ballester A. Artificial intelligence-driven radiological biomarkers: A narrative review of artificial intelligence in meningioma diagnosis. NeuroMarkers. 2025;2:100033. [DOI]

Ajmal CS, Yerram S, Abishek V, Nizam VPM, Aglave G, Patnam JD, et al. Innovative Approaches in Regulatory Affairs: Leveraging Artificial Intelligence and Machine Learning for Efficient Compliance and Decision-Making. AAPS J. 2025;27:22. [DOI] [PubMed]

van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M. Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol. 2021;31:3797–804. [DOI] [PubMed] [PMC]

Wu K, Wu E, Theodorou B, Liang W, Mack C, Glass L, et al. Characterizing the Clinical Adoption of Medical AI Devices through U.S. Insurance Claims. NEJM AI. 2024;1:1–13. [DOI]

van de Sande D, Chung EFF, Oosterhoff J, van Bommel J, Gommers D, van Genderen ME. To warrant clinical adoption AI models require a multi-faceted implementation evaluation. NPJ Digit Med. 2024;7:58. [DOI] [PubMed] [PMC]

Gorenstein L, Soffer S, Apter S, Konen E, Klang E. AI in radiology: is it the time for randomized controlled trials? Eur Radiol. 2023;33:4223–5. [DOI] [PubMed]

10.

Tanguay W, Acar P, Fine B, Abdolell M, Gong B, Cadrin-Chênevert A, et al. Assessment of Radiology Artificial Intelligence Software: A Validation and Evaluation Framework. Can Assoc Radiol J. 2022;74:326–33. [DOI] [PubMed]

11.

Yu AC, Mohajer B, Eng J. External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. Radiol Artif Intell. 2022;4:e210064. [DOI] [PubMed] [PMC]

12.

Abdar M, Khosravi A, Islam SMS, Acharya UR, Vasilakos AV. The need for quantification of uncertainty in artificial intelligence for clinical data analysis: increasing the level of trust in the decision-making process. IEEE Syst Man Cybern Mag. 2022;8:28–40. [DOI]

13.

Marconi L, Cabitza F. Show and tell: A critical review on robustness and uncertainty for a more responsible medical AI. Int J Med Inform. 2025;202:105970. [DOI] [PubMed]

14.

Ojha J, Presacan O, G. Lind P, Monteiro E, Yazidi A. Navigating Uncertainty: A User-Perspective Survey of Trustworthiness of AI in Healthcare. ACM Trans Comput Healthc. 2025;6:1–32. [DOI]

15.

Berkowitz JS, Patock JR, Nawaz A, Gonzalez-Hernandez G, Tatonetti NP. A crisis of overconfidence: Why confidence, not accuracy, is the real risk in clinical AI. BioData Min. 2026;19:10. [DOI] [PubMed] [PMC]

16.

Windecker D, Baj G, Shiri I, Kazaj PM, Kaesmacher J, Gräni C, et al. Generalizability of FDA-Approved AI-Enabled Medical Devices for Clinical Use. JAMA Netw Open. 2025;8:e258052. [DOI] [PubMed] [PMC]

17.

Suleman MU, Mursaleen M, Khalil U, Saboor A, Bilal M, Khan SA, et al. Assessing the generalizability of artificial intelligence in radiology: a systematic review of performance across different clinical settings. Ann Med Surg. 2025;87:8803–11. [DOI] [PubMed] [PMC]

18.

Hassoon A, Lin C, Woo HYJ, Irimia R, Marsteller JA, Li A, et al. Guiding artificial intelligence in public health and medicine with epidemiology: A lifecycle framework for mitigating AI misalignment. Ann Epidemiol. 2025;112:119–26. [DOI] [PubMed]

19.

Blezek DJ, Olson-Williams L, Missert A, Korfiatis P. AI Integration in the Clinical Workflow. J Digit Imaging. 2021;34:1435–46. [DOI] [PubMed] [PMC]

20.

Cross JL, Choma MA, Onofrey JA. Bias in medical AI: Implications for clinical decision-making. PLOS Digit Health. 2024;3:e0000651. [DOI] [PubMed] [PMC]