The incidence of cutaneous melanoma is rising faster than any other type of cancer, however, many experts believe this trend may be due to an increase in false positives rather than an increase in the true occurrence of melanoma.¹ This notion is fueled, in part, by the significant mismatch between the disease’s incidence rate and mortality rate, which have increased 120% and 5%, respectively, since 1991.^2,3 In addition, studies have found no statistical differences between the mortality rates of areas that screen for melanoma and those that do not.⁴ These facts raise important questions regarding current diagnostic paradigms and call for new strategies that can help better target this disease.

The current gold standard for melanoma diagnosis is visual or dermoscopic inspection, followed by excision, and microscopic examination of fixed tissue slices ($\sim 5\text{-}\mu \text{m}$ thick) using hematoxylin and eosin (H&E) staining. Unfortunately, histological evaluation suffers from significant drawbacks; for example, it is subjective. As a result, melanoma diagnosis is subject to (1) high discordance rates ($\sim 14\%$) (i.e., pathologists examining the same skin biopsy samples disagree on the diagnosis);⁵ and (2) a “diagnostic drift” (i.e., the same severely atypical dysplastic nevi samples diagnosed as nonmalignant in the late 1980s are now reclassified as malignant).⁶

These problems have motivated the development of molecular imaging techniques that provide additional insight into the disease states.^7,8 In this work, we utilize pump-probe microscopy, which yields information on diagnostically relevant pigments, namely eu- and pheo-melanin, by probing their excited state photodynamics.⁷ The ability to visualize the melanin chemistry with high spatial resolution is potentially useful to aid diagnosis since it provides insight into metabolic states, location of melanocytes, and melanogenesis.⁷ Recently, we have demonstrated the ability to quantitatively differentiate between melanocytic nevi and malignant melanomas based on bulk percent eumelanin, without considering its spatial distribution.⁷ These findings echo those of other studies that use diffuse reflectance spectroscopy⁹ or destructive chemical analyses.¹⁰ Unlike these methods, however, pump-probe microscopy offers additional information based on the subcellular distribution of the melanin content,^7,11 which, if quantified, could have a significant impact on melanoma diagnosis.

In this Letter, we quantify melanin spatial distribution using a mathematical morphology transformation. The approach applies a two-dimensional (2-D) autocorrelation function to the quantitative molecular images and extracts statistical properties from the resulting distribution. The parameters extracted to quantify melanin structure are described and their diagnostic relevance is demonstrated by analyzing pigmented, unstained, thin cutaneous, and conjunctival melanocytic lesions. Results are compared to the diagnosis made by a board-certified pathologist (Dr. M. A. Selim) using H&E stained samples from adjacent slices from the same lesions.

In pump-probe microscopy, two laser pulses are used to probe the ultra-fast dynamic properties of pigmented samples. A detailed description of the instrumentation is given elsewhere.⁷ In brief, the output of a mode-locked Ti:Sapphire laser, tuned to 810 nm, is split into two beams: One is used to pump an optical parametric oscillator that is tuned to 720 nm, and then modulated at 2 MHz using an acousto-optic modulator. The second beam from the Ti:Sapph (probe beam) is sent to a delay-line to control the time of arrival of the probe with respect to the pump. Then the pump and probe beams are recombined and sent to a scanning microscope. This wavelength combination is chosen to increase contrast between eu- and pheo-melanin. When a nonlinear interaction between the two beams occurs in the sample, the pump modulation is transferred to the probe beam, which is then detected using a lock-in-amplifier.

Data collected can be considered a hyperspectral cube, where two dimensions contain the spatial information and the third contains the dynamic properties. To obtain quantitative molecular maps of the melanin distribution, we first remove unwanted contributions from pigments that are irrelevant to the diagnosis of melanocytic lesions by applying phasor analysis.¹² Unwanted pigments may include surgical ink and/or hemoglobin, neither of which is produced by melanocytes. In phasor analysis, clusters of points in a 2-D histogram of the phasors (cosine and sine transformations from the time response of each image-pixel) are indicative of different molecular species, allowing one to digitally remove unwanted species by applying a mask in phasor space. The same mask, as determined by the cumulative phasor histogram of 42 cutaneous lesions,¹² is applied to all images. Finally, we obtain quantitative information by solving an overdetermined set of linear equations (i.e., by matrix inversion), using as a basis set the known pump-probe response for sepia eumelanin and synthetic pheomelanin. Four molecular maps are obtained: eu- and pheo-melanin distribution, total melanin content, and percent eumelanin. (A different class of melanin, identified as iron-loaded eumelanin, was recently found in tissue and in Jurassic-aged fossils,¹³ but its diagnostic value is still under investigation and is thus not specifically accounted for here.)

Once these four quantitative molecular images are obtained, a 2-D autocorrelation is computed for each. This is equivalent to producing a morphological autocorrelation transformation (MAT) which extracts second-order geometrical information (i.e., morphological covariance) and can produce metrics that are invariant to translation, scale, and rotation.¹⁴ (MAT was defined in Ref. 14 as a collection of directional autocorrelation functions, computed for a set of angles $\varphi$. These are equivalent to radial slices of the 2-D autocorrelation. Thus, the 2-D autocorrelation represents a MAT covering a range of $\varphi \in [0,\pi ]$.) The MAT provides information that is not made available by other mathematical morphology methods (e.g., morphological skeleton transformation or pattern spectrum), which are first order and can only describe boundaries and areas, or give a skeletal image representation.¹⁵ Thus, MAT can better depict the complex architecture observed in the pigmented tissue samples and allow comparison of melanin structure across different images and specimens. Built-in MATLAB (Mathworks 2013, Natick, Massachusetts) functions were used for all computations.

Figure 1 shows representative quantitative molecular images (percent eumelanin) along with their MAT. In general, benign samples tend to have pigment that is more organize or structured; in other words, the melanin is bounded to the epidermis or dermal-epidermal junction.¹⁶ This organized structure manifests itself as anisotropy in the MAT; for example, the MAT will contain a “pointy” streak. On the other hand, samples exhibiting higher degrees of vertical growth, a hallmark of invasive behavior,¹⁶ will possess a rounder, more homogenous MAT distribution. To quantify this observation, we start by computing the standard-deviations (second moments, ${\sigma }_x$ and ${\sigma }_y$) along and orthogonal to $R$, a vector from the origin to the center-of-mass of the MAT. Then, we define the normalized MAT anisotropy parameter as $\theta =|{\tan }^{-1}({\sigma }_y/{\sigma }_x-1)|$, which varies from 0 (highly disorganized) to $\pi$ (highly organized); note that $\theta$ is rotation invariant, zero when ${\sigma }_y={\sigma }_x$ and symmetrical about ${\sigma }_y={\sigma }_x$. Further, the magnitude of $R$ can be interpreted as the amount of organized pigmentation along the direction of highest covariance, which is typically along the epidermis. These parameters are depicted graphically in Fig. 1(b). Next, we compute the entropy of the MAT, which provides an additional measure of the organization of the pigment (i.e., spatial heterogeneity). Note that the image entropy and MAT entropy are not equivalent since the MAT is a second-order transformation. Lastly, images with low SNR (below 0 dB) are omitted from the analysis; to estimate the noise from a single image, we use the average difference in intensity between the zero-lag of the MAT and adjacent points.¹⁷

Fig. 1

Representative percent eumelanin images along with their two-dimensional (2-D) morphological autocorrelation transformation (MAT). Because the MAT is symmetric about the origin, only the right half-plane is depicted. (b) Illustrates the parameters extracted from the MAT to quantify image structure, see text for details.

The results for $N=53$ cutaneous lesions are summarized in Fig. 2. For comparison with the MAT structure analysis, we first classify the lesions using bulk pigment information (such as percent eumelanin), neglecting spatial morphology. Figure 2(a) is a scatter plot of percent eumelanin average versus standard-deviation; with decision lines determined using a quadratic discrimination function. Here, the data generally show that as lesions become more eumelanotic, they lose chemical heterogeneity (i.e., the distribution of percent eumelanin becomes narrower). However, this trend is not strong, and we observe that the percent eumelanin alone is a poor predictor of invasive lesions. To quantitatively compare the predictive power of the parameters, the leave-one-out cross-validation (LOOCV) method is used, which provides an unbiased estimate of the predictive error and is a common choice for small sample sets.¹⁸ Also, support vector machine (SVM) is used to train the algorithm with a Gaussian radial basis function kernel.¹⁹ Using these bulk melanin parameters alone to discriminate between invasive and noninvasive lesions, the cross-validation yields a sensitivity (probability of detection) $P_D=69\%$ and specificity (complement of false alarm probability) $1-P_{\mathrm{FA}}=72\%$. Note that with SVM and LOOCV, the user does not have to specify decision lines nor a sensitivity or specificity value. It is also worth emphasizing that we only classify for invasive melanomas since clear cytological and architectural features for this lesion type allows for an indisputable diagnosis using histology and thus provide an appropriate benchmark for the analysis.

Fig. 2

Scatter plot of quantitative parameters. Arrows point to the parameters from samples shown in Fig. 1. SK: seborrheic keratosis (benign), BN: benign nevus, CN: compound nevus, DN: dysplastic nevus, MIS: melanoma in-situ, and IM: invasive melanoma.

Figure 2(b) incorporates the entropy from the eumelanin image MAT, where it is clear that the two parameters provide better discrimination. Quantitatively, the standard-deviation of the percent eumelanin and entropy of the eumelanin image MAT yield an improved $P_D=84.6\%$ and $1-P_{\mathrm{FA}}=85\%$. Adding $R$ and $\theta$ from the eumelanin image MAT [shown in Fig. 2(c)] to the cross-validation further improves the classifier to $P_D=92.3\%$ and $1-P_{\mathrm{FA}}=97.5\%$ (one false negative and one false positive).

These results show dramatically improved sensitivity and specificity even over previous attempts to incorporate spatial information using wavelet analysis and hidden Markov trees, which have achieved $P_D=72\%$ and $1-P_{\mathrm{FA}}=74\%$ (same dataset and statistical analysis were used in this study).²⁰ Further improvements might be possible by incorporating additional MAT-derived parameters (e.g., entropy of total-melanin image MAT). However, each new parameter increases the dimensionality of the classification space, bringing on the so-called “curse of dimensionality”: while our limited training set ($N=53$) adequately samples a low-dimensional space, it is too sparse in a high-dimensional space to reliably estimate decision boundaries.²¹ These effects are expected to lessen as we acquire a larger training set in future work.

The 2-D MAT analysis was also recently applied to conjunctival melanocytic lesions²² (the MAT method was not described in detail there). In this proof-of-principle work, a smaller sample subset was imaged ($N=12$), with three different groups: benign nevi, primary acquired melanosis, and conjunctival melanoma. For these lesions, $\theta$ from pheomelanin images, total melanin MAT entropy, and percent eumelanin standard-deviation provided perfect separation of the three different groups. Although this was a small study, the results further support the importance of the 2-D MAT as a valuable tool to quantify image structure for melanoma detection.

Recent work by Seidenari et al.⁸ using multiphoton laser tomography and fluorescence life time imaging (FLIM) have also shown promising results for diagnosing melanoma. They differentiate between disease states using FLIM spectra along with morphological features. Classification of features, however, was achieved using observer-blinded studies rather than mathematical/computational approaches.

In conclusion, we have presented a method to quantify pump-probe image structure using a 2-D mathematical autocorrelation transformation. The parameters extracted from the MAT, namely $R$, $\theta$, and entropy, seek to quantify different features related to the structure of the molecular images. The method can be implemented using built-in MATLAB functions and all new parameters have been explicitly defined. The combination of these parameters along with bulk chemical properties, yield high sensitivity and specificity for invasive melanoma. Future work will focus on obtaining a larger sample set to enable separation of other lesion types, such as melanoma in-situ and dysplastic nevi, and to look for statistical differences with age and gender.