Can AI distinguish a bone radiograph from photos of flowers or cars? Evaluation of bone age deep learning model on inappropriate data inputs
Date
2021-08-05Journal
Skeletal RadiologyPublisher
Springer NatureType
Article
Metadata
Show full item recordSee at
https://doi.org/10.1007/s00256-021-03880-yhttp://www.ncbi.nlm.nih.gov/pmc/articles/pmc8339162/
Abstract
Objective: To evaluate the behavior of a publicly available deep convolutional neural network (DCNN) bone age algorithm when presented with inappropriate data inputs in both radiological and non-radiological domains. Methods: We evaluated a publicly available DCNN-based bone age application. The DCNN was trained on 12,612 pediatric hand radiographs and won the 2017 RSNA Pediatric Bone Age Challenge (concordance of 0.991 with radiologist ground-truth). We used the application to analyze 50 left-hand radiographs (appropriate data inputs) and seven classes of inappropriate data inputs in radiological (i.e., chest radiographs) and non-radiological (i.e., image of street numbers) domains. For each image, we noted if (1) the application distinguished between appropriate and inappropriate data inputs and (2) inference time per image. Mean inference times were compared using ANOVA. Results: The 16Bit Bone Age application calculated bone age for all pediatric hand radiographs with mean inference time of 1.1 s. The application did not distinguish between pediatric hand radiographs and inappropriate image types, including radiological and non-radiological domains. The application inappropriately calculated bone age for all inappropriate image types, with mean inference time of 1.1 s for all categories (p = 1). Conclusion: A publicly available DCNN-based bone age application failed to distinguish between appropriate and inappropriate data inputs and calculated bone age for inappropriate images. The awareness of inappropriate outputs based on inappropriate DCNN input is important if tasks such as bone age determination are automated, emphasizing the need for appropriate oversight at the data input and verification stage to avoid unrecognized erroneous results.Rights/Terms
© 2021. ISS.Identifier to cite or link to this item
http://hdl.handle.net/10713/16350ae974a485f413a2113503eed53cd6c53
10.1007/s00256-021-03880-y
Scopus Count
Collections
Related articles
- Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning.
- Authors: Yi PH, Kim TK, Wei J, Shin J, Hui FK, Sair HI, Hager GD, Fritz J
- Issue date: 2019 Jul
- Bone age determination using only the index finger: a novel approach using a convolutional neural network compared with human radiologists.
- Authors: Reddy NE, Rayan JC, Annapragada AV, Mahmood NF, Scheslinger AE, Zhang W, Kan JH
- Issue date: 2020 Apr
- MABAL: a Novel Deep-Learning Architecture for Machine-Assisted Bone Age Labeling.
- Authors: Mutasa S, Chang PD, Ruzal-Shapiro C, Ayyala R
- Issue date: 2018 Aug
- Deep Convolutional Neural Network-based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs.
- Authors: Sim Y, Chung MJ, Kotter E, Yune S, Kim M, Do S, Han K, Kim H, Yang S, Lee DJ, Choi BW
- Issue date: 2020 Jan
- Deep Learning Method for Automated Classification of Anteroposterior and Posteroanterior Chest Radiographs.
- Authors: Kim TK, Yi PH, Wei J, Shin JW, Hager G, Hui FK, Sair HI, Lin CT
- Issue date: 2019 Dec