Prostate cancer is the most common nonskin-related cancer, affecting one in seven men in the United States. Gleason score, a sum of the primary and secondary Gleason patterns, is one of the best predictors of prostate cancer outcomes. Recently, significant progress has been made in molecular subtyping prostate cancer through the use of genomic sequencing. It has been established that prostate cancer patients presented with a Gleason score 7 show heterogeneity in both disease recurrence and survival. We built a unified system using publicly available whole-slide images and genomic data of histopathology specimens through deep neural networks to identify a set of computational biomarkers. Using a survival model, the experimental results on the public prostate dataset showed that the computational biomarkers extracted by our approach had hazard ratio as 5.73 and C-index as 0.74, which were higher than standard clinical prognostic factors and other engineered image texture features. Collectively, the results of this study highlight the important role of neural network analysis of prostate cancer and the potential of such approaches in other precision medicine applications.
Prostate cancer is the most common non-skin related cancer affecting 1 in 7 men in the United States. Treatment of patients with prostate cancer still remains a difficult decision-making process that requires physicians to balance clinical benefits, life expectancy, comorbidities, and treatment-related side effects. Gleason score (a sum of the primary and secondary Gleason patterns) solely based on morphological prostate glandular architecture has shown as one of the best predictors of prostate cancer outcome. Significant progress has been made on molecular subtyping prostate cancer delineated through the increasing use of gene sequencing. Prostate cancer patients with Gleason score of 7 show heterogeneity in recurrence and survival outcomes. Therefore, we propose to assess the correlation between histopathology images and genomic data with disease recurrence in prostate tumors with a Gleason 7 score to identify prognostic markers. In the study, we identify image biomarkers within tissue WSIs by modeling the spatial relationship from automatically created patches as a sequence within WSI by adopting a recurrence network model, namely long short-term memory (LSTM). Our preliminary results demonstrate that integrating image biomarkers from CNN with LSTM and genomic pathway scores, is more strongly correlated with patients recurrence of disease compared to standard clinical markers and engineered image texture features. The study further demonstrates that prostate cancer patients with Gleason score of 4+3 have a higher risk of disease progression and recurrence compared to prostate cancer patients with Gleason score of 3+4.
The Gleason grading system used to render prostate cancer diagnosis has recently been updated to allow more accurate grade stratification and higher prognostic discrimination when compared to the traditional grading system. In spite of progress made in trying to standardize the grading process, there still remains approximately a 30% grading discrepancy between the score rendered by general pathologists and those provided by experts while reviewing needle biopsies for Gleason pattern 3 and 4, which accounts for more than 70% of daily prostate tis- sue slides at most institutions. We propose a new computational imaging method for Gleason pattern 3 and 4 classification, which better matches the newly established prostate cancer grading system. The computer- aided analysis method includes two phases. First, the boundary of each glandular region is automatically segmented using a deep convolutional neural network. Second, color, shape and texture features are extracted from superpixels corresponding to the outer and inner glandular regions and are subsequently forwarded to a random forest classifier to give a gradient score between 3 and 4 for each delineated glandular region. The F1 score for glandular segmentation is 0.8460 and the classification accuracy is 0.83±0.03.
KEYWORDS: Blood, Databases, Pathology, Image retrieval, Image segmentation, Content based image retrieval, Cancer, Medical imaging, RGB color model, Algorithm development
The purpose of this work was to evaluate a newly developed content-based retrieval approach for characterizing a range
of different white blood cells from a database of imaged peripheral blood smears. Specimens were imaged using a 20×
magnification to provide adequate resolution and sufficiently large field of view. The resulting database included a test
ensemble of 96 images (1000×1000 pixels each). In this work, we propose a four-step content-based retrieval method
and evaluate its performance. The content-based image retrieval (CBIR) method starts from white blood cell
identification, followed by three sequential steps including coarse-searching, refined searching, and finally mean-shift
clustering using a hierarchical annular histogram (HAH). The prototype system was shown to reliably retrieve those
candidate images exhibiting the highest-ranked (most similar) characteristics to the query. The results presented here
show that the algorithm was able to parse out subtle staining differences and spatial patterns and distributions for the
entire range of white blood cells under study. Central to the design of the system is that it capitalizes on lessons learned
by our team while observing human experts when they are asked to carry out these same tasks.
In this paper, we propose a novel image classification method based on sparse reconstruction errors to discriminate
cancerous breast tissue microarray (TMA) discs from benign ones. Sparse representation is employed to
reconstruct the samples and separate the benign and cancer discs. The method consists of several steps including
mask generation, dictionary learning, and data classification. Mask generation is performed using multiple scale
texton histogram, integral histogram and AdaBoost. Two separate cancer and benign TMA dictionaries are
learned using K-SVD. Sparse coefficients are calculated using orthogonal matching pursuit (OMP), and the reconstructive
error of each testing sample is recorded. The testing image will be divided into many small patches.
Each small patch will be assigned to the category which produced the smallest reconstruction error. The final
classification of each testing sample is achieved by calculating the total reconstruction errors. Using standard
RGB images, and tested on a dataset with 547 images, we achieved much better results than previous literature.
The binary classification accuracy, sensitivity, and specificity are 88.0%, 90.6%, and 70.5%, respectively.
A performance study was conducted to compare classification accuracy using both multispectral imaging
(MSI) and standard bright-field imaging (RGB) to characterize breast tissue microarrays. The study was
primarily focused on investigating the classification power of texton features for differentiating cancerous
breast TMA discs from normal. The feature extraction algorithm includes two main processes: texton library
training and histogram construction. First, two texton libraries were built for multispectral cubes and RGB
images respectively, which comprised the training process. Second, texton histograms from each
multispectral cube and RGB image were used as testing sets. Finally, within each spectral band, exhaustive
feature selection was used to search for the combination of features that yielded the best classification
accuracy using the pathologic result as a golden standard. Support vector machine was applied as a classifier
using leave-one-out cross-validation. The spectra carrying the greatest discriminatory power were
automatically chosen and a majority vote was used to make the final classification. The study included 122
breast TMA discs that showed poor classification power based on simple visualization of RGB images. Use
of multispectral cubes showed improved sensitivity and specificity compared to the RGB images (85%
sensitivity & 85% specificity for MSI vs. 75% & 65% for RGB). This study demonstrates that use of texton
features derived from MSI datasets achieve better classification accuracy than those derived from RGB
datasets. This study further shows that MSI provided statistically significant improvements in automated
analysis of single-stained bright-field images. Future work will examine MSI performance in assessing multistained
specimens.
The lack of clear consensus over the utility of multispectral imaging (MSI) for bright-field imaging
prompted our team to investigate the benefit of using MSI on breast tissue microarrays (TMA). We have
conducted performance studies to compare MSI with standard bright-field imaging in hematoxylin stained
breast tissue. The methodology has three components. The first extracts a region of interest using adaptive
thresholding and morphological processing. The second performs texture feature extraction from a local
binary pattern within each spectral channel and compared to features of co-occurrence matrix and texture
feature coding in third component. The third component performs feature selection and classification. For
each spectrum, exhaustive feature selection was used to search for the combination of features that yields
the best classification accuracy. AdaBoost with a linear perceptron least-square classifier was applied. The
spectra carrying the greatest discriminatory power were automatically chosen and a majority vote was used to make the final classification. 92 breast TMA discs were included in the study. Sensitivity of 0.96 and specificity of 0.89 were achieved on the multispectral data, compared with sensitivity of 0.83 and specificity of 0.85 on RGB data. MSI consistently achieved better classification results than those obtained using standard RGB images. While the benefits of MSI for unmixing multi-stained specimens are well documented, this study demonstrated statistically significant improvements in the automated analysis of single stained bright-field images.
KEYWORDS: Image segmentation, Optical coherence tomography, Tissues, Endoscopy, Photomicroscopy, 3D image processing, Natural surfaces, Visualization, In vivo imaging, Colorectal cancer
Colonic crypt morphological patterns have shown a close correlation with histopathological diagnosis. Imaging technologies such as high-magnification chromoendoscopy and endoscopic optical coherence tomography (OCT) are capable of visualizing crypt morphology in vivo. We have imaged colonic tissue in vitro to simulate high-magnification chromoendoscopy and endoscopic OCT and demonstrate quantification of morphological features of colonic crypts using automated image analysis. 2-D microscopic images with methylene blue staining and correlated 3-D OCT volumes were segmented using marker-based watershed segmentation. 2-D and 3-D crypt morphological features were quantified. The accuracy of segmentation was validated, and measured features are in agreement with known crypt morphology. This work can enable studies to determine the clinical utility of high-magnification chromoendoscopy and endoscopic OCT, as well as studies to evaluate crypt morphology as a biomarker for colonic disease progression.
Colorectal cancer is the second leading cause of cancer-related death in the United States. Approximately 50% of these deaths could be prevented by earlier detection through screening.
Magnification chromoendoscopy is a technique which utilizes tissue stains applied to the gastrointestinal
mucosa and high-magnification endoscopy to better visualize and characterize lesions. Prior studies have
shown that shapes of colonic crypts change with disease and show characteristic patterns. Current methods
for assessing colonic crypt patterns are somewhat subjective and not standardized. Computerized
algorithms could be used to standardize colonic crypt pattern assessment. We have imaged resected colonic
mucosa in vitro (N = 70) using methylene blue dye and a surgical microscope to approximately simulate in
vivo imaging with magnification chromoendoscopy. We have developed a method of computerized
processing to analyze the crypt patterns in the images. The quantitative image analysis consists of three
steps. First, the crypts within the region of interest of colonic tissue are semi-automatically segmented
using watershed morphological processing. Second, crypt size and shape parameters are extracted from the
segmented crypts. Third, each sample is assigned to a category according to the Kudo criteria. The
computerized classification is validated by comparison with human classification using the Kudo
classification criteria. The computerized colonic crypt pattern analysis algorithm will enable a study of in
vivo magnification chromoendoscopy of colonic crypt pattern correlated with risk of colorectal cancer.
This study will assess the feasibility of screening and surveillance of the colon using magnification
chromoendoscopy.
Barrett's esophagus (BE) and associated adenocarcinoma have emerged as a major health care problem over the last two decades. Because of the widespread use of endoscopy, BE is being recognized increasingly in all Western countries. In clinical trials of endoscopic optical coherence tomography (EOCT), we defined certain image features that appear to be characteristic of precancerous (dysplastic) mucosa: decreased scattering and disorganization in the microscopic morphology. The objective of the present work is to develop computer-aided diagnosis (CAD) algorithms that aid the detection of dysplasia in BE. The image dataset used in the present study was derived from a total of 405 EOCT images (13 patients) that were paired with highly correlated histologic sections of corresponding biopsies. Of these, 106 images were included in the study. The CAD algorithm used was based on a standard texture analysis method (center-symmetric auto-correlation). Using histology as the reference standard, this CAD algorithm had a sensitivity of 82%, specificity of 74%, and accuracy of 83%. CAD has the potential to quantify and standardize the diagnosis of dysplasia and allows high throughput image evaluation for EOCT screening applications. With further refinements, CAD could also improve the accuracy of EOCT identification of dysplasia in BE.
We are developing computer-aided diagnosis (CAD) algorithms to aid in the classification of dysplasia in Barrett's esophagus (BE) using endoscopic OCT (EOCT). Our previous CAD algorithm yielded single spatial scale texture features, used a single parameter as a classifier, and used only one EOCT image per biopsy site. In this work, we aim to overcome these limitations. We present progress in development of Fourier domain fractal analysis with classification trees using multiple images to classify a single site for more accurate dysplasia classification in BE EOCT images. A total of 812 EOCT images (13 patients, 70 non-dysplastic biopsy sites including 499 images and 38 dysplastic biopsy sites including 313 images) were analyzed. Using only one frame for classification, 95% sensitivity (95% confidence interval (CI) is 82%-100%) and 94% specificity (95% CI is 86%-98%) were achieved. Using three frames per biopsy site and requiring two frames to be positive to classify the site as positive, 100% sensitivity (95% CI is 89%-100%) and 100% specificity (95% CI is 94%-100%) were achieved, and 97% of the sites had at least three frames available. In conclusion, Fourier domain fractal analysis with classification tree achieved more successful classification of dysplasia in BE than our previous CAD algorithm. And making use of multiple images from a single biopsy site can improve the accuracy of classification. CAD has the potential to enable EOCT surveillance of large surface areas of Barrett's mucosa for dysplasia.
Barrett's esophagus (BE) has become a major health care burden because of its association with adenocarcinoma of the esophagus. We have shown that endoscopic optical coherence tomography (EOCT) has a 70% accuracy in the diagnosis of dysplasia (Gastrointest Endosc 2003; 57:AB77). To demonstrate the feasiblity of computer aided diagnosis (CAD) of dysplasia in BE using EOCT digital images, to quantitate/standardize the diagnosis of dysplasia, and to develop algorithms suitable for EOCT surveillance of large areas of Barrett’s mucosa, 106 EOCT images were selected (13 patients from 28 cases) from the clinical study including 68 of non-dysplastic and 38 of dysplastic mucosa. From the digital image stream, the 3 frames immediately preceding impact of the forceps on the tissue were selected to insure close correlation between histology/EOCT image pairs. Computer aided diagnosis by center symmetric autocorrelation (CENS) and principal component analysis (PCA) were used for feature parameter extraction and analysis based on the segmented ROI. Leave-one-out cross-validation was used for classification and finally receiver operating characteristic (ROC) curve was used to evaluate the performance of CAD and the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy were calculated. The result shows that CAD is able to achieve a higher accuracy than humans for identification of dysplasia in EOCT images. CAD may be of assistance in the EOCT surveillance of large surface areas of Barrett’s mucosa for dysplasia.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.