|
The ultimate goal of this project is to investigate whether the effect of a computer-aided detection (CAD) system on
readers' performance (especially, in situation of an upgrade of the CAD system, or between two different CAD systems
with similar design) can be accurately predicted without having to perform a multi-reader multi-case (MRMC) observer
study and, if such prediction is possible, to establish the underlying methodology. Our current study is intended to
provide evidence that would substantiate efforts toward such investigation. The objectives of this study were 1) to
investigate the relationship between the number of radiologists reading a dataset of thoracic computed tomography (CT)
images to identify lung nodules and the number of distinct findings and 2) to determine the number of readers needed to
identify almost all clinically distinct findings in a dataset. We used data from a multi-reader multi-case (MRMC)
observer study that consisted of six radiologists interpreting 85 thoracic CT examinations. To further illustrate our
approach, we also utilized simulated data consisting of twelve readers interpreting 198 samples equally distributed
between three levels of detection difficulty. For each possible reader grouping, the number of distinct findings identified
by the readers in the group was calculated. Five types of regression models used to describe the relationship between the
average number of distinct findings per case and the number of readers needed were compared. The result showed that
the logistic model best fitted both the thoracic CT data and the simulated data. Our assumption is that adding more
readers after a certain reader set size would mostly add redundant findings and, therefore, the benefit would be
negligible. Using this model, the predicted number of readers was found to depend on the type of findings considered.
Our study showed that the number of clinically distinct findings that can be identified by radiologists on CT lung
examinations without the use of a CAD system may be limited and that identifying almost all of these findings may only
require a limited number of readers.
|
