The special section on digital pathology in the Journal of Medical Imaging is a reflection of the growing research interest in the field and an attempt to showcase the exciting developments in image analytics and machine learning on digitized tissue slides. The call for papers for the special section went out in the spring of 2016, and in response to the call, we received a total of 12 papers. Each of the papers was handled by one of the three guest editors for the special section and was submitted to a rigorous review process involving at least two reviewers. Of the 12 papers, 7 were accepted, reflecting a 67% acceptance rate.

The glomerulus is the blood filtering unit of the kidney. Each human kidney contains ∼1  million glomeruli. Several renal conditions originate from structural damage to glomerular microcompartments, such as proteinuria, the excessive loss of blood proteins into urine. The gold standard for evaluating structural damage in renal pathology is histopathological and immunofluorescence examination of needle biopsies under a light microscope. This method is limited by qualitative or semiquantitative manual scoring approaches to the evaluation of glomerular structural features. Computational quantification of equivalent features promises to improve the precision of glomerular structural analysis. One large obstacle to the computational quantification of renal tissue is the identification of complex glomerular boundaries automatically. To mitigate this issue, we developed a computational pipeline capable of extracting and exactly defining glomerular boundaries. Our method, composed of Gabor filtering, Gaussian blurring, statistical F-testing, and distance transform, is able to accurately identify glomerular boundaries with mean sensitivity/specificity of 0.88/0.96 and accuracy of 0.92, on n=1000 glomeruli images stained with standard renal histological stains. Our method will simplify computational partitioning of glomerular microcompartments hidden within dense textural boundaries. Automatic quantification of glomeruli will streamline structural analysis in clinic and can pioneer real-time diagnoses and interventions for renal care.

The Gleason grade is the most common architectural and morphological assessment of prostate cancer severity and prognosis. There have been numerous algorithms developed to approximate and duplicate the Gleason scoring system, mostly developed in standard H&E brightfield microscopy. Immunofluorescence (IF) image analysis of tissue pathology has recently been proven to be robust in developing prognostic assessments of disease, particularly in prostate cancer. We leverage a method of segmenting gland rings in IF images for predicting the pathological Gleason, both the clinical and the image specific grades, which may not necessarily be the same. We combine these measures with nuclear specific characteristics. In 324 images from 324 patients, our individual features correlate well univariately with the Gleason grades and in a multivariate setting have an accuracy of 85% in predicting the Gleason grade. Additionally, these features correlate strongly with clinical progression outcomes [concordance index (CI) of 0.89], significantly outperforming the clinical Gleason grades (CI of 0.78). Finally, in multivariate models for multiple prostate cancer progression endpoints, replacing the Gleason with these features results in equivalent or improved performances. This work presents the first assessment of morphological gland unit features from IF images for predicting the Gleason grade, and even replacing it in prostate cancer prognostics.

Analysis and morphological comparison of the arteriolar and venular components of a microvascular network are essential to our understanding of multiple diseases affecting every organ system. We have developed and evaluated the first fully automatic software system for differentiation of arterioles from venules on high-resolution digital histology images of the mouse hind limb immunostained with smooth muscle α-actin. Classifiers trained on statistical and morphological features by supervised machine learning provided useful classification accuracy for differentiation of arterioles from venules, achieving an area under the receiver operating characteristic curve of 0.89. Feature selection was consistent across cross validation iterations, and a small set of two features was required to achieve the reported performance, suggesting the generalizability of the system. This system eliminates the need for laborious manual classification of the hundreds of microvessels occurring in a typical sample and paves the way for high-throughput analysis of the arteriolar and venular networks in the mouse.

Computational histomorphometric approaches typically use low-level image features for building machine learning classifiers. However, these approaches usually ignore high-level expert knowledge. A computational model (M_im) combines low-, mid-, and high-level image information to predict the likelihood of cancer in whole slide images. Handcrafted low- and mid-level features are computed from area, color, and spatial nuclei distributions. High-level information is implicitly captured from the recorded navigations of pathologists while exploring whole slide images during diagnostic tasks. This model was validated by predicting the presence of cancer in a set of unseen fields of view. The available database was composed of 24 cases of basal-cell carcinoma, from which 17 served to estimate the model parameters and the remaining 7 comprised the evaluation set. A total of 274 fields of view of size 1024×1024  pixels were extracted from the evaluation set. Then 176 patches from this set were used to train a support vector machine classifier to predict the presence of cancer on a patch-by-patch basis while the remaining 98 image patches were used for independent testing, ensuring that the training and test sets do not comprise patches from the same patient. A baseline model (M_ex) estimated the cancer likelihood for each of the image patches. M_ex uses the same visual features as M_im, but its weights are estimated from nuclei manually labeled as cancerous or noncancerous by a pathologist. M_im achieved an accuracy of 74.49% and an F-measure of 80.31%, while M_ex yielded corresponding accuracy and F-measures of 73.47% and 77.97%, respectively.

Single-cell studies using noninvasive imaging is a challenging, yet appealing way to study cellular characteristics over extended periods of time, for instance to follow cell interactions and the behavior of different cell types within the same sample. In some cases, e.g., transplantation culturing, real-time cellular monitoring, stem cell studies, in vivo studies, and embryo growth studies, it is also crucial to keep the sample intact and invasive imaging using fluorophores or dyes is not an option. Computerized methods are needed to improve throughput of image-based analysis and for use with noninvasive microscopy such methods are poorly developed. By combining a set of well-documented image analysis and classification tools with noninvasive microscopy, we demonstrate the ability for long-term image-based analysis of morphological changes in single cells as induced by a toxin, and show how these changes can be used to indicate changes in biological function. In this study, adherent cell cultures of DU-145 treated with low-concentration (LC) etoposide were imaged during 3 days. Single cells were identified by image segmentation and subsequently classified on image features, extracted for each cell. In parallel with image analysis, an MTS assay was performed to allow comparison between metabolic activity and morphological changes after long-term low-level drug response. Results show a decrease in proliferation rate for LC etoposide, accompanied by changes in cell morphology, primarily leading to an increase in cell area and textural changes. It is shown that changes detected by image analysis are already visible on day 1 for 0.25-μM etoposide, whereas effects on MTS and viability are detected only on day 3 for 5-μM etoposide concentration, leading to the conclusion that the morphological changes observed occur before and at lower concentrations than a reduction in cell metabolic activity or viability. Three classifiers are compared and we report a best case sensitivity of 88% and specificity of 94% for classification of cells as treated/untreated.

We introduce a Markov random field (MRF)-driven region-based active contour model (MaRACel) for histological image segmentation. This Bayesian segmentation method combines a region-based active contour (RAC) with an MRF. State-of-the-art RAC models assume that every spatial location in the image is statistically independent, thereby ignoring valuable contextual information among spatial locations. To address this shortcoming, we incorporate an MRF prior into energy term of the RAC. This requires a formulation of the Markov prior consistent with the continuous variational framework characteristic of active contours; consequently, we introduce a continuous analog to the discrete Potts model. Based on the automated segmentation boundary of glands by MaRACel model, explicit shape descriptors are then employed to distinguish prostate glands belonging to Gleason patterns 3 (G3) and 4 (G4). To demonstrate the effectiveness of MaRACel, we compare its performance to the popular models proposed by Chan and Vese (CV) and Rousson and Deriche (RD) with respect to the following tasks: (1) the segmentation of prostatic acini (glands) and (2) the differentiation of G3 and G4 glands. On almost 600 prostate biopsy needle images, MaRACel was shown to have higher average dice coefficients, overlap ratios, sensitivities, specificities, and positive predictive values both in terms of segmentation accuracy and ability to discriminate between G3 and G4 glands compared to the CV and RD models.

Despite the current rapid advance in technologies for whole slide imaging, there is still no scientific consensus on the recommended methodology for image quality assessment of digital pathology slides. For medical images in general, it has been recommended to assess image quality in terms of doctors’ success rates in performing a specific clinical task while using the images (clinical image quality, cIQ). However, digital pathology is a new modality, and already identifying the appropriate task is difficult. In an alternative common approach, humans are asked to do a simpler task such as rating overall image quality (perceived image quality, pIQ), but that involves the risk of nonclinically relevant findings due to an unknown relationship between the pIQ and cIQ. In this study, we explored three different experimental protocols: (1) conducting a clinical task (detecting inclusion bodies), (2) rating image similarity and preference, and (3) rating the overall image quality. Additionally, within protocol 1, overall quality ratings were also collected (task-aware pIQ). The experiments were done by diagnostic veterinary pathologists in the context of evaluating the quality of hematoxylin and eosin-stained digital pathology slides of animal tissue samples under several common image alterations: additive noise, blurring, change in gamma, change in color saturation, and JPG compression. While the size of our experiments was small and prevents drawing strong conclusions, the results suggest the need to define a clinical task. Importantly, the pIQ data collected under protocols 2 and 3 did not always rank the image alterations the same as their cIQ from protocol 1, warning against using conventional pIQ to predict cIQ. At the same time, there was a correlation between the cIQ and task-aware pIQ ratings from protocol 1, suggesting that the clinical experiment context (set by specifying the clinical task) may affect human visual attention and bring focus to their criteria of image quality. Further research is needed to assess whether and for which purposes (e.g., preclinical testing) task-aware pIQ ratings could substitute cIQ for a given clinical task.

Scanning-beam digital x-ray (SBDX) is an inverse geometry x-ray fluoroscopy system capable of tomosynthesis-based 3-D catheter tracking. This work proposes a method of dose-reduced 3-D catheter tracking using dynamic electronic collimation (DEC) of the SBDX scanning x-ray tube. This is achieved through the selective deactivation of focal spot positions not needed for the catheter tracking task. The technique was retrospectively evaluated with SBDX detector data recorded during a phantom study. DEC imaging of a catheter tip at isocenter required 340 active focal spots per frame versus 4473 spots in full field-of-view (FOV) mode. The dose-area product (DAP) and peak skin dose (PSD) for DEC versus full FOV scanning were calculated using an SBDX Monte Carlo simulation code. The average DAP was reduced to 7.8% of the full FOV value, consistent with the relative number of active focal spots (7.6%). For image sequences with a moving catheter, PSD was 33.6% to 34.8% of the full FOV value. The root-mean-squared-deviation between DEC-based 3-D tracking coordinates and full FOV 3-D tracking coordinates was less than 0.1 mm. The 3-D distance between the tracked tip and the sheath centerline averaged 0.75 mm. DEC is a feasible method for dose reduction during SBDX 3-D catheter tracking.

Attenuation correction is essential for quantitative reliability of positron emission tomography (PET) imaging. In time-of-flight (TOF) PET, attenuation sinogram can be determined up to a global constant from noiseless emission data due to the TOF PET data consistency condition. This provides the theoretical basis for jointly estimating both activity image and attenuation sinogram/image directly from TOF PET emission data. Multiple joint estimation methods, such as maximum likelihood activity and attenuation (MLAA) and maximum likelihood attenuation correction factor (MLACF), have already been shown that can produce improved reconstruction results in TOF cases. However, due to the nonconcavity of the joint log-likelihood function and Poisson noise presented in PET data, the iterative method still requires proper initialization and well-designed regularization to prevent convergence to local maxima. To address this issue, we propose a joint estimation of activity image and attenuation sinogram using the TOF PET data consistency condition as an attenuation sinogram filter, and then evaluate the performance of the proposed method using computer simulations.

We present a fast, noise-efficient, and accurate estimator for material separation using photon-counting x-ray detectors (PCXDs) with multiple energy bin capability. The proposed targeted least squares estimator (TLSE) is an improvement of a previously described A-table method by incorporating dynamic weighting that allows the variance to be closer to the Cramér–Rao lower bound (CRLB) throughout the operating range. We explore Cartesian and average-energy segmentation of the basis material space for TLSE and show that, compared with Cartesian segmentation, the average-energy method requires fewer segments to achieve similar performance. We compare the average-energy TLSE to other proposed estimators—including the gold standard maximum likelihood estimator (MLE) and the A-table—in terms of variance, bias, and computational efficiency. The variance and bias were simulated in the range of 0 to 6 cm of aluminum and 0 to 50 cm of water with Monte Carlo methods. The Average-energy TLSE achieves an average variance within 2% of the CRLB and mean absolute error of 3.68±0.06×10−6  cm. Using the same protocol, the MLE showed variance within 1.9% of the CRLB ratio and average absolute error of 3.10±0.06×10−6  cm but was 50 times slower in our implementations. Compared with the A-table method, TLSE gives a more homogenously optimal variance-to-CRLB ratio in the operating region. We show that variance in basis material estimates for TLSE is lower than that of the A-table method by as much as ∼36% in the peripheral region of operating range (thin or thick objects). The TLSE is a computationally efficient and fast method for material separation with PCXDs, with accuracy and precision comparable to the MLE.

Recent work has shown that single-coil, magnetic induction tomography (MIT) is useful for visualizing three-dimensional electrical conductivity distributions within biological targets. Coil-induced eddy currents and the associated secondary field are detected as an inductive loss while the coil is relocated to several unique positions and orientations near a target. Image reconstruction is then accomplished by inversion of a convolution integral that quantitatively maps inductive loss with conductivity. Previously, coil position and orientation had to be established by a template, which required assignment of fixed locations for the coil to visit. Here, our existing device is modified so that coil position and orientation are optically tracked while measuring inductive loss. Optical tracking is accomplished via a set of infrared reflective spheres mounted on the same enclosure that supports the coil. The coil center can be tracked with submillimeter accuracy while orientation angle is known to within a fraction of a degree. This work illustrates the use of single-coil MIT in full, position-orientation-tracked scan mode while imaging laboratory phantoms consisting of features having biologically relevant conductivity.

Photon-counting detector CT has a large number of acquisition parameters that require optimization, particularly the energy threshold configurations. Fast and accurate estimation of both signal and noise in photon-counting CT (PCCT) images can facilitate such optimization. Using the detector response function of a research PCCT system, we derived mathematical models for both signal and noise estimation, taking into account beam spectrum and filtration, object attenuation, water beam hardening, detector response, radiation dose, energy thresholds, and the propagation of noise. To determine the absolute noise value, a noise lookup table (LUT) for all available energy thresholds was acquired using a number of calibration scans. The noise estimation algorithm then used the noise LUT to estimate noise for scans with a variety of combination of energy thresholds, dose levels, and object attenuations. Validation of the estimation algorithms was performed on a whole-body research PCCT system using semianthropomorphic water phantoms and solutions of calcium and iodine. Clinical feasibility of noise estimation was assessed with scans of a cadaver head and a living swine. The algorithms achieved accurate estimation of both signal and noise for a variety of scanning parameter combinations. Maximum discrepancies were below 15%, while most errors were below 5%.

An x-ray energy spectrum plays an essential role in computed tomography (CT) imaging and related tasks. Because of the high photon flux of clinical CT scanners, most of the spectrum estimation methods are indirect and usually suffer from various limitations. In this study, we aim to provide a segmentation-free, indirect transmission measurement–based energy spectrum estimation method using dual-energy material decomposition. The general principle of this method is to minimize the quadratic error between the polychromatic forward projection and the raw projection to calibrate a set of unknown weights, which are used to express the unknown spectrum together with a set of model spectra. The polychromatic forward projection is performed using material-specific images, which are obtained using dual-energy material decomposition. The algorithm was evaluated using numerical simulations, experimental phantom data, and realistic patient data. The results show that the estimated spectrum matches the reference spectrum quite well and the method is robust. Extensive studies suggest that the method provides an accurate estimate of the CT spectrum without dedicated physical phantom and prolonged workflow. This paper may be attractive for CT dose calculation, artifacts reduction, polychromatic image reconstruction, and other spectrum-involved CT applications.

Owing to the inconsistent image quality existing in routine obstetric ultrasound (US) scans that leads to a large intraobserver and interobserver variability, the aim of this study is to develop a quality-assured, fully automated US fetal head measurement system. A texton-based fetal head segmentation is used as a prerequisite step to obtain the head region. Textons are calculated using a filter bank designed specific for US fetal head structure. Both shape- and anatomic-based features calculated from the segmented head region are then fed into a random forest classifier to determine the quality of the image (e.g., whether the image is acquired from a correct imaging plane), from which fetal head measurements [biparietal diameter (BPD), occipital–frontal diameter (OFD), and head circumference (HC)] are derived. The experimental results show a good performance of our method for US quality assessment and fetal head measurements. The overall precision for automatic image quality assessment is 95.24% with 87.5% sensitivity and 100% specificity, while segmentation performance shows 99.27% (±0.26) of accuracy, 97.07% (±2.3) of sensitivity, 2.23 mm (±0.74) of the maximum symmetric contour distance, and 0.84 mm (±0.28) of the average symmetric contour distance. The statistical analysis results using paired t-test and Bland–Altman plots analysis indicate that the 95% limits of agreement for inter observer variability between the automated measurements and the senior expert measurements are 2.7 mm of BPD, 5.8 mm of OFD, and 10.4 mm of HC, whereas the mean differences are −0.038±1.38  mm, −0.20±2.98  mm, and −0.72±5.36  mm, respectively. These narrow 95% limits of agreements indicate a good level of consistency between the automated and the senior expert’s measurements.

Computer-assisted automated exudate detection is crucial for large-scale screening of diabetic retinopathy (DR). The motivation of this work is robust and accurate detection of low contrast and isolated hard exudates using fundus imaging. Gabor filtering is first performed to enhance exudate visibility followed by Tsallis entropy thresholding. The obtained candidate exudate pixel map is useful for further removal of falsely detected candidates using sparse-based dictionary learning and classification. Two reconstructive dictionaries are learnt using the intensity, gradient, local energy, and transform domain features extracted from exudate and background patches of the training fundus images. Then, a sparse representation-based classifier separates the true exudate pixels from false positives using least reconstruction error. The proposed method is evaluated on the publicly available e-ophtha EX and standard DR database calibration level 1 (DIARETDB1) databases and high exudate detection performance is achieved. In the e-ophtha EX database, mean sensitivity of 85.80% and positive predictive value of 57.93% are found. For the DIARETDB1 database, an area under the curve of 0.954 is obtained.

Automated segmentation of cortical and noncortical human brain structures has been hitherto approached using nonrigid registration followed by label fusion. We propose an alternative approach for this using a convolutional neural network (CNN) which classifies a voxel into one of many structures. Four different kinds of two-dimensional and three-dimensional intensity patches are extracted for each voxel, providing local and global (context) information to the CNN. The proposed approach is evaluated on five different publicly available datasets which differ in the number of labels per volume. The obtained mean Dice coefficient varied according to the number of labels, for example, it is 0.844±0.031 and 0.743±0.019 for datasets with the least (32) and the most (134) number of labels, respectively. These figures are marginally better or on par with those obtained with the current state-of-the-art methods on nearly all datasets, at a reduced computational time. The consistently good performance of the proposed method across datasets and no requirement for registration make it attractive for many applications where reduced computational time is necessary.

Vascular segmentation plays an important role in the assessment of peripheral arterial disease. The segmentation is very challenging especially for arteries with severe stenosis or complete occlusion. We present a cascading algorithm for vascular centerline tree detection specializing in detecting centerlines in diseased peripheral arteries. It takes a three-dimensional computed tomography angiography (CTA) volume and returns a vascular centerline tree, which can be used for accelerating and facilitating the vascular segmentation. The algorithm consists of four levels, two of which detect healthy arteries of varying sizes and two that specialize in different types of vascular pathology: severe calcification and occlusion. We perform four main steps at each level: appropriate parameters for each level are selected automatically, a set of centrally located voxels is detected, these voxels are connected together based on the connection criteria, and the resulting centerline tree is corrected from spurious branches. The proposed method was tested on 25 CTA scans of the lower limbs, achieving an average overlap rate of 89% and an average detection rate of 82%. The average execution time using four CPU cores was 70 s, and the technique was successful also in detecting very distal artery branches, e.g., in the foot.

With the advancement of three-dimensional (3-D) real-time echocardiography in recent years, automatic creation of patient specific geometric models is becoming feasible and important in clinical decision making. However, the vast majority of echocardiographic segmentation methods presented in the literature focus on the left ventricle (LV) endocardial border, leaving segmentation of the right ventricle (RV) a largely unexplored problem, despite the increasing recognition of the RV’s role in cardiovascular disease. We present a method for coupled segmentation of the endo- and epicardial borders of both the LV and RV in 3-D ultrasound images. To solve the segmentation problem, we propose an extension of a successful state-estimation segmentation framework with a geometrical representation of coupled surfaces, as well as the introduction of myocardial incompressibility to regularize the segmentation. The method was validated against manual measurements and segmentations in images of 16 patients. Mean absolute distances of 2.8±0.4  mm, 3.2±0.7  mm, and 3.1±0.5  mm between the proposed and reference segmentations were observed for the LV endocardium, RV endocardium, and LV epicardium surfaces, respectively. The method was computationally efficient, with a computation time of 2.1±0.4  s.

An important challenge to using fluorodeoxyglucose-positron emission tomography (FDG-PET) in clinical trials of brain tumor patients is to identify malignant regions whose metabolic activity shows significant changes between pretreatment and a posttreatment scans in the presence of high normal brain background metabolism. This paper describes a semiautomated processing and analysis pipeline that is able to detect such changes objectively with a given false detection rate. Image registration and voxelwise comparison of the pre- and posttreatment images were performed. A key step is adjustment of the observed difference by the estimated background change at each voxel, thereby overcoming the confounding effect of spatially heterogeneous metabolic activity in the brain. Components of the proposed method were validated via phantom experiments and computer simulations. It achieves a false response volume accuracy of 0.4% at a significance threshold of 3 standard deviations. It is shown that the proposed methodology can detect lesion response with 100% accuracy with a tumor-to-background-ratio as low as 1.5, and it is not affected by the background brain glucose metabolism change. We also applied the method to FDG-PET patient images from a clinical trial to assess treatment effects of lapatinib, which demonstrated significant changes in metabolism corresponding to tumor regions.

A four-dimensional magnetic resonance imaging (4-D-MRI) technique with Sagittal–Coronal–Diaphragm Point-of-Intersection (SCD-PoI) as a respiratory surrogate is proposed. To develop an image-based respiratory surrogate, the SCD-PoI motion tracking method is used for retrospective 4-D-MRI reconstruction. Single-slice sagittal MR cine was acquired at a location near the center of the diaphragmatic dome. Multiple-slice coronal MR cines were acquired for 4-D-MRI reconstruction. As a motion surrogate, the diaphragm motion was measured from the PoI among the sagittal MRI cine plane, coronal MRI cine planes, and the diaphragm surface. These points were defined as the SCD-PoI. This point is used as a one-dimensional diaphragmatic navigator in our study. The 4-D-MRI technique was evaluated on a 4-D digital extended cardiac-torso (XCAT) human phantom, a motion phantom, and seven human subjects (five healthy volunteers and two cancer patients). Motion trajectories of a selected region of interest were measured on 4-D-MRI and compared with the known XCAT motion that served as references. The mean absolute amplitude difference (D) and the cross-correlation coefficient (CC) of the comparisons were determined. 4-D-MRI of the XCAT phantom demonstrated highly accurate motion information (D=1.13  mm, CC=0.98). Motion trajectories of the motion phantom measured on 4-D-MRI matched well with the references (D=0.54  mm, CC=0.99). 4-D-MRI of human subjects showed minimal artifacts and clearly revealed the respiratory motion of organs and tumor (mean D=1.08±1.03  mm; mean CC=0.96). A 4-D-MRI technique with image-based respiratory surrogate has been developed and tested on phantoms and human subjects.

In computerized detection of clustered microcalcifications (MCs) from mammograms, the traditional approach is to apply a pattern detector to locate the presence of individual MCs, which are subsequently grouped into clusters. Such an approach is often susceptible to the occurrence of false positives (FPs) caused by local image patterns that resemble MCs. We investigate the feasibility of a direct detection approach to determining whether an image region contains clustered MCs or not. Toward this goal, we develop a deep convolutional neural network (CNN) as the classifier model to which the input consists of a large image window (1  cm2 in size). The multiple layers in the CNN classifier are trained to automatically extract image features relevant to MCs at different spatial scales. In the experiments, we demonstrated this approach on a dataset consisting of both screen-film mammograms and full-field digital mammograms. We evaluated the detection performance both on classifying image regions of clustered MCs using a receiver operating characteristic (ROC) analysis and on detecting clustered MCs from full mammograms by a free-response receiver operating characteristic analysis. For comparison, we also considered a recently developed MC detector with FP suppression. In classifying image regions of clustered MCs, the CNN classifier achieved 0.971 in the area under the ROC curve, compared to 0.944 for the MC detector. In detecting clustered MCs from full mammograms, at 90% sensitivity, the CNN classifier obtained an FP rate of 0.69 clusters/image, compared to 1.17 clusters/image by the MC detector. These results indicate that using global image features can be more effective in discriminating clustered MCs from FPs caused by various sources, such as linear structures, thereby providing a more accurate detection of clustered MCs on mammograms.

Multimodal endoscopy using fluorescence molecular probes is a promising method of surveying the entire esophagus to detect cancer progression. Using the fluorescence ratio of a target compared to a surrounding background, a quantitative value is diagnostic for progression from Barrett’s esophagus to high-grade dysplasia (HGD) and esophageal adenocarcinoma (EAC). However, current quantification of fluorescent images is done only after the endoscopic procedure. We developed a Chan–Vese-based algorithm to segment fluorescence targets, and subsequent morphological operations to generate background, thus calculating target/background (T/B) ratios, potentially to provide real-time guidance for biopsy and endoscopic therapy. With an initial processing speed of 2 fps and by calculating the T/B ratio for each frame, our method provides quasireal-time quantification of the molecular probe labeling to the endoscopist. Furthermore, an automatic computer-aided diagnosis algorithm can be applied to the recorded endoscopic video, and the overall T/B ratio is calculated for each patient. The receiver operating characteristic curve was employed to determine the threshold for classification of HGD/EAC using leave-one-out cross-validation. With 92% sensitivity and 75% specificity to classify HGD/EAC, our automatic algorithm shows promising results for a surveillance procedure to help manage esophageal cancer and other cancers inspected by endoscopy.

Computer-assisted diagnostic (CAD) tools are of interest as they enable efficient decision-making in clinics and the screening of diseases. The traditional approach to CAD algorithm design focuses on the automated detection of abnormalities independent of the end-user, who can be an image reader or an expert. We propose a reader-centric system design wherein a reader’s attention is drawn to abnormal regions in a least-obtrusive yet effective manner, using saliency-based emphasis of abnormalities and without altering the appearance of the background tissues. We present an assistive lesion-emphasis system (ALES) based on the above idea, for fundus image-based diabetic retinopathy diagnosis. Lesion-saliency is learnt using a convolutional neural network (CNN), inspired by the saliency model of Itti and Koch. The CNN is used to fine-tune standard low-level filters and learn high-level filters for deriving a lesion-saliency map, which is then used to perform lesion-emphasis via a spatially variant version of gamma correction. The proposed system has been evaluated on public datasets and benchmarked against other saliency models. It was found to outperform other saliency models by 6% to 30% and boost the contrast-to-noise ratio of lesions by more than 30%. Results of a perceptual study also underscore the effectiveness and, hence, the potential of ALES as an assistive tool for readers.

Bone metastases are a frequent occurrence with cancer, and early detection can guide the patient’s treatment regimen. Metastatic bone disease can present in density extremes as sclerotic (high density) and lytic (low density) or in a continuum with an admixture of both sclerotic and lytic components. We design a framework to detect and characterize the varying spectrum of presentation of spine metastasis on positron emission tomography/computed tomography (PET/CT) data. A technique is proposed to synthesize CT and PET images to enhance the lesion appearance for computer detection. A combination of watershed, graph cut, and level set algorithms is first run to obtain the initial detections. Detections are then sent to multiple classifiers for sclerotic, lytic, and mixed lesions. The system was tested on 44 cases with 225 sclerotic, 139 lytic, and 92 mixed lesions. The results showed that sensitivity (false positive per patient) was 0.81 (2.1), 0.81 (1.3), and 0.76 (2.1) for sclerotic, lytic, and mixed lesions, respectively. It also demonstrates that using PET/CT data significantly improves the computer aided detection performance over using CT alone.

With the advent of fully automated image analysis and modern machine learning methods, there is a need for very large image datasets having documented segmentations for both computer algorithm training and evaluation. This paper presents a method and implementation for facilitating such datasets that addresses the critical issue of size scaling for algorithm validation and evaluation; current evaluation methods that are usually used in academic studies do not scale to large datasets. This method includes protocols for the documentation of many regions in very large image datasets; the documentation may be incrementally updated by new image data and by improved algorithm outcomes. This method has been used for 5 years in the context of chest health biomarkers from low-dose chest CT images that are now being used with increasing frequency in lung cancer screening practice. The lung scans are segmented into over 100 different anatomical regions, and the method has been applied to a dataset of over 20,000 chest CT images. Using this framework, the computer algorithms have been developed to achieve over 90% acceptable image segmentation on the complete dataset.

A prostate computer-aided diagnosis (CAD) based on random forest to detect prostate cancer using a combination of spatial, intensity, and texture features extracted from three sequences, T2W, ADC, and B2000 images, is proposed. The random forest training considers instance-level weighting for equal treatment of small and large cancerous lesions as well as small and large prostate backgrounds. Two other approaches, based on an AutoContext pipeline intended to make better use of sequence-specific patterns, were considered. One pipeline uses random forest on individual sequences while the other uses an image filter described to produce probability map-like images. These were compared to a previously published CAD approach based on support vector machine (SVM) evaluated on the same data. The random forest, features, sampling strategy, and instance-level weighting improve prostate cancer detection performance [area under the curve (AUC) 0.93] in comparison to SVM (AUC 0.86) on the same test data. Using a simple image filtering technique as a first-stage detector to highlight likely regions of prostate cancer helps with learning stability over using a learning-based approach owing to visibility and ambiguity of annotations in each sequence.

A method using rough set feature selection and extreme learning machine (ELM) whose learning strategy and hidden node parameters are optimized by self-adaptive differential evolution (SaDE) algorithm for classification of breast masses is investigated. A pathologically proven database of 140 breast ultrasound images, including 80 benign and 60 malignant, is used for this study. A fast nonlocal means algorithm is applied for speckle noise removal, and multiresolution analysis of undecimated discrete wavelet transform is used for accurate segmentation of breast lesions. A total of 34 features, including 29 textural and five morphological, are applied to a k-fold cross-validation scheme, in which more relevant features are selected by quick-reduct algorithm, and the breast masses are discriminated into benign or malignant using SaDE-ELM classifier. The diagnosis accuracy of the system is assessed using parameters, such as accuracy (Ac), sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), Matthew’s correlation coefficient (MCC), and area (Az) under receiver operating characteristics curve. The performance of the proposed system is also compared with other classifiers, such as support vector machine and ELM. The results indicated that the proposed SaDE algorithm has superior performance with Ac=0.9714, Se=0.9667, Sp=0.975, PPV=0.9666, NPV=0.975, MCC=0.9417, and Az=0.9604 compared to other classifiers.

We present a method to track vessels in angiography [contrast filled vessels in two-dimensional (2-D) x-ray fluoroscopy]. Finding correspondence of a vessel tree from consecutive angiogram frames provides significant value in computer-aided clinical applications such as fast vessel tree segmentation, three-dimensional (3-D) vessel topology reconstruction from corresponding centerlines, cardiac motion understanding, etc. However, establishing an accurate vessel tree correspondence (vessel tree tracking) is a nontrivial problem due to nonlinear periodic cardiac and breathing motion in 2-D views, foreshortening, false bifurcations due to 3-D to 2-D projection, occlusion from other anatomies, etc. The vessel tree is represented by BSpline curves. The control points of the BSpline curves are landmarks that are the tracking targets. Our method maximizes the appearance similarity while preserving the vessel structure. A directed acyclic graph (DAG) is employed to represent the appearance and shape structure of the vessel tree: nodes from the DAG encode the appearance of the vessel tree landmarks, and the edges encode the relative locations between landmarks. The vessel tree tracking problem turns into finding the most similar tree from the DAG in the next frame, and it is solved using an efficient dynamic programming algorithm. We performed evaluations on 62 x-ray angiography sequences (above 1000 frames). Experiment results show our algorithm is robust to these challenges and delivers better performance, compared to four existing methods.

Acoustic neuroma surgery is a procedure in which a benign mass is removed from the internal auditory canal (IAC). Currently, this surgical procedure requires manual drilling of the temporal bone followed by exposure and removal of the acoustic neuroma. This procedure is physically and mentally taxing to the surgeon. Our group is working on the development of an acoustic neuroma surgery robot (ANSR) to perform the initial drilling procedure. Planning the ANSR’s drilling region using preoperative CT requires expertise and takes about 35 min. We propose an approach for automatically producing a resection plan for the ANSR that would avoid damage to sensitive ear structures and require minimal editing by the surgeon. We first compute an atlas-based segmentation of the mastoid section of the temporal bone, refine it based on the position of anatomical landmarks, and apply a safety margin to the result to produce the automatic resection plan. In experiments with CTs from nine subjects, our automated process resulted in a resection plan that was verified to be safe in every case. Approximately 2 min were required in each case for the surgeon to verify and edit the plan to permit functional access to the IAC. We measured a mean Dice coefficient of 0.99 and surface error of 0.08 mm between the final and automatically proposed plans. These preliminary results indicate that our approach is a viable method for resection planning for the ANSR and drastically reduces the surgeon’s planning effort.

Cochlear implants (CIs) are surgically implanted medical devices used to treat individuals with severe-to-profound sensorineural hearing loss. Although these devices have been remarkably successful at restoring audibility, many patients experience poor outcomes. Our group has developed the first image-guided CI programming technique where the electrode positions are found in CT images and used to estimate neural activation patterns, which is unique information that audiologists can use to define patient-specific processor settings. Currently, neural activation is estimated using only the distance from each electrode to the neural activation sites, which might be less accurate than using high-resolution electro-anatomical models (EAMs) to perform physics-based estimations of neural activation. We propose a patient-customized EAM approach where the EAM is spatially and electrically adapted to a patient-specific configuration. Spatial adaptation is done through nonrigid registration of the model with the patient CT image. Electrical adaptation is done by adjusting tissue resistivity parameters, so the intracochlear voltage distributions predicted by the model best match those directly measured for the patient via their implant. We found that our approach, demonstrated for N=7 patients, results in mean percent differences between direct and simulated measurements of voltage distributions of 10.9%.

This study aimed to determine whether a reduction in radiation dose was found for percutaneous coronary interventional (PCI) patients using a cardiac interventional x-ray system with state-of-the-art image enhancement and x-ray optimization, compared to the current generation x-ray system, and to determine the corresponding impact on clinical image quality. Patient procedure dose area product (DAP) and fluoroscopy duration of 131 PCI patient cases from each x-ray system were compared using a Wilcoxon test on median values. Significant reductions in patient dose (p≪0.001) were found for the new system with no significant change in fluoroscopy duration (p=0.2); procedure DAP reduced by 64%, fluoroscopy DAP by 51%, and “cine” acquisition DAP by 76%. The image quality of 15 patient angiograms from each x-ray system (30 total) was scored by 75 clinical professionals on a continuous scale for the ability to determine the presence and severity of stenotic lesions; image quality scores were analyzed using a two-sample t-test. Image quality was reduced by 9% (p≪0.01) for the new x-ray system. This demonstrates a substantial reduction in patient dose, from acquisition more than fluoroscopy imaging, with slightly reduced image quality, for the new x-ray system compared to the current generation system.

We tested the agreement of radiologists’ rankings of different reconstructions of breast computed tomography images based on their diagnostic (classification) performance and on their subjective image quality assessments. We used 102 pathology proven cases (62 malignant, 40 benign), and an iterative image reconstruction (IIR) algorithm to obtain 24 reconstructions per case with different image appearances. Using image feature analysis, we selected 3 IIRs and 1 clinical reconstruction and 50 lesions. The reconstructions produced a range of image quality from smooth/low-noise to sharp/high-noise, which had a range in classifier performance corresponding to AUCs of 0.62 to 0.96. Six experienced Mammography Quality Standards Act (MQSA) radiologists rated the likelihood of malignancy for each lesion. We conducted an additional reader study with the same radiologists and a subset of 30 lesions. Radiologists ranked each reconstruction according to their preference. There was disagreement among the six radiologists on which reconstruction produced images with the highest diagnostic content, but they preferred the midsharp/noise image appearance over the others. However, the reconstruction they preferred most did not match with their performance. Due to these disagreements, it may be difficult to develop a single image-based model observer that is representative of a population of radiologists for this particular imaging task.

Computational modeling of visual attention is an active area of research. These models have been successfully employed in applications such as robotics. However, most computational models of visual attention are developed in the context of natural scenes, and their role with medical images is not well investigated. As radiologists interpret a large number of clinical images in a limited time, an efficient strategy to deploy their visual attention is necessary. Visual saliency maps, highlighting image regions that differ dramatically from their surroundings, are expected to be predictive of where radiologists fixate their gaze. We compared 16 state-of-art saliency models over three medical imaging modalities. The estimated saliency maps were evaluated against radiologists’ eye movements. The results show that the models achieved competitive accuracy using three metrics, but the rank order of the models varied significantly across the three modalities. Moreover, the model ranks on the medical images were all considerably different from the model ranks on the benchmark MIT300 dataset of natural images. Thus, modality-specific tuning of saliency models is necessary to make them valuable for applications in fields such as medical image compression and radiology education.

We describe a postacquisition, attribute-based quality assessment method for brain magnetic resonance imaging (MRI) images. It is based on the application of Bayes theory to the relationship between entropy and image quality attributes. The entropy feature image of a slice is segmented into low- and high-entropy regions. For each entropy region, there are three separate observations of contrast, standard deviation, and sharpness quality attributes. A quality index for a quality attribute is the posterior probability of an entropy region given any corresponding region in a feature image where quality attribute is observed. Prior belief in each entropy region is determined from normalized total clique potential (TCP) energy of the slice. For TCP below the predefined threshold, the prior probability for a region is determined by deviation of its percentage composition in the slice from a standard normal distribution built from 250 MRI volume data provided by Alzheimer’s Disease Neuroimaging Initiative. For TCP above the threshold, the prior is computed using a mathematical model that describes the TCP–noise level relationship in brain MRI images. Our proposed method assesses the image quality of each entropy region and the global image. Experimental results demonstrate good correlation with subjective opinions of radiologists for different types and levels of quality distortions.

A flexible handheld imaging probe consisting of a 3  mm×3  mm charge-coupled device camera, light-emitting diode light sources, and near-infrared laser source is designed and developed. The imaging probe is designed with specifications to capture the iridocorneal angle images and posterior segment images. Light propagation from the anterior chamber of the eye to the exterior is considered analytically using Snell’s law. Imaging of the iridocorneal angle region and fundus is performed on ex vivo porcine samples and subsequently on small laboratory animals, such as the New Zealand white rabbit and nonhuman primate, in vivo. The integrated flexible handheld probe demonstrates high repeatability in iridocorneal angle and fundus documentation. The proposed concept and methodology are expected to find potential application in the diagnosis, prognosis, and management of glaucoma.

Quantification of myocardial blood flow (MBF) can aid in the diagnosis and treatment of coronary artery disease. However, there are no widely accepted clinical methods for estimating MBF. Dynamic cardiac perfusion computed tomography (CT) holds the promise of providing a quick and easy method to measure MBF quantitatively. However, the need for repeated scans can potentially result in a high patient radiation dose, limiting the clinical acceptance of this approach. In our previous work, we explored techniques to reduce the patient dose by either uniformly reducing the tube current or by uniformly reducing the number of temporal frames in the dynamic CT sequence. These dose reduction techniques result in noisy time-attenuation curves (TACs), which can give rise to significant errors in MBF estimation. We seek to investigate whether nonuniformly varying the tube current and/or sampling intervals can yield more accurate MBF estimates for a given dose. Specifically, we try to minimize the dose and obtain the most accurate MBF estimate by addressing the following questions: when in the TAC should the CT data be collected and at what tube current(s)? We hypothesize that increasing the sampling rate and/or tube current during the time frames when the myocardial CT number is most sensitive to the flow rate, while reducing them elsewhere, can achieve better estimation accuracy for the same dose. We perform simulations of contrast agent kinetics and CT acquisitions to evaluate the relative MBF estimation performance of several clinically viable variable acquisition methods. We find that variable temporal and tube current sequences can be performed that impart an effective dose of 5.5 mSv and allow for reductions in MBF estimation root-mean-square error on the order of 20% compared to uniform acquisition sequences with comparable or higher radiation doses.

Compressed sensing (CS) has been utilized for acceleration of data acquisition in magnetic resonance imaging (MRI). MR images can then be reconstructed with an undersampling rate significantly lower than that required by the Nyquist sampling criterion. However, the CS usually produces images with artifacts, especially at high reduction rates. We propose a CS MRI method called shearlet sparsity and nonlocal total variation (SS-NLTV) that exploits SS-NLTV regularization. The shearlet transform is an optimal sparsifying transform with excellent directional sensitivity compared with that by wavelet transform. The NLTV, on the other hand, extends the TV regularizer to a nonlocal variant that can preserve both textures and structures and produce sharper images. We have explored an approach of combining alternating direction method of multipliers (ADMM), splitting variables technique, and adaptive weighting to solve the formulated optimization problem. The proposed SS-NLTV method is evaluated experimentally and compared with the previously reported high-performance methods. Results demonstrate a significant improvement of compressed MR image reconstruction on four medical MRI datasets.

Ultrasound B-scan imaging systems operate under some well-known resolution limits. To improve resolution, the concept of stable pulses, having bounded inverse filters, was previously utilized for the lateral deconvolution. This framework has been extended to the axial direction, enabling a two-dimensional deconvolution. The modeling of the two-way response in the axial direction is discussed, and the deconvolution is performed in the in-phase quadrature data domain. Stable inverse filters are generated and applied for the deconvolution of the image data from Field II simulation, a tissue-mimicking phantom, and in vivo imaging of a carotid artery, where resolution enhancement is observed. Specifically, in simulation results, the resolution is enhanced by as many as 8.75 times laterally and 20.5 times axially considering the −6-dB width of the autocorrelation of the envelope images.

Glandular structural features are important for the tumor pathologist in the assessment of cancer malignancy of prostate tissue slides. The varying shapes and sizes of glands combined with the tedious manual observation task can result in inaccurate assessment. There are also discrepancies and low-level agreement among pathologists, especially in cases of Gleason pattern 3 and pattern 4 prostate adenocarcinoma. An automated gland segmentation system can highlight various glandular shapes and structures for further analysis by the pathologist. These objective highlighted patterns can help reduce the assessment variability. We propose an automated gland segmentation system. Forty-three hematoxylin and eosin-stained images were acquired from prostate cancer tissue slides and were manually annotated for gland, lumen, periacinar retraction clefting, and stroma regions. Our automated gland segmentation system was trained using these manual annotations. It identifies these regions using a combination of pixel and object-level classifiers by incorporating local and spatial information for consolidating pixel-level classification results into object-level segmentation. Experimental results show that our method outperforms various texture and gland structure-based gland segmentation algorithms in the literature. Our method has good performance and can be a promising tool to help decrease interobserver variability among pathologists.

Extraction of cell nuclei from hematoxylin and eosin (H&E)-stained histopathological images is an essential preprocessing step in computerized image analysis for disease detection, diagnosis, and prognosis. We present an automated cell nuclei segmentation approach that works with H&E-stained images. A color deconvolution algorithm was first applied to the image to get the hematoxylin channel. Using a morphological operation and thresholding technique on the hematoxylin channel image, candidate target nuclei and background regions were detected, which were then used as markers for a marker-controlled watershed transform segmentation algorithm. Moreover, postprocessing was conducted to split the touching nuclei. For each segmented region from the previous steps, the regional maximum value positions were identified as potential nuclei centers. These maximum values were further grouped into k-clusters, and the locations within each cluster were connected with the minimum spanning tree technique. Then, these connected positions were utilized as new markers for a watershed segmentation approach. The final number of nuclei at each region was determined by minimizing an objective function that iterated all of the possible k-values. The proposed method was applied to the pathological images of the tumor tissues from The Cancer Genome Atlas study. Experimental results show that the proposed method can lead to promising results in terms of segmentation accuracy and separation of touching nuclei.

Ziehl–Neelsen stained microscopy is a crucial bacteriological test for tuberculosis detection, but its sensitivity is poor. According to the World Health Organization (WHO) recommendation, 300 viewfields should be analyzed to augment sensitivity, but only a few viewfields are examined due to patient load. Therefore, tuberculosis diagnosis through automated capture of the focused image (autofocusing), stitching of viewfields to form mosaics (autostitching), and automatic bacilli segmentation (grading) can significantly improve the sensitivity. However, the lack of unified datasets impedes the development of robust algorithms in these three domains. Therefore, the Ziehl–Neelsen sputum smear microscopy image database (ZNSM iDB) has been developed, and is freely available. This database contains seven categories of diverse datasets acquired from three different bright-field microscopes. Datasets related to autofocusing, autostitching, and manually segmenting bacilli can be used for developing algorithms, whereas the other four datasets are provided to streamline the sensitivity and specificity. All three categories of datasets were validated using different automated algorithms. As images available in this database have distinctive presentations with high noise and artifacts, this referral resource can also be used for the validation of robust detection algorithms. The ZNSM-iDB also assists for the development of methods in automated microscopy.

This PDF file contains the errata for “JMI Vol. 4 Issue 02 Paper JMI-2017-0522-ERR” for JMI Vol. 4 Issue 02