2.1.

RS System

Measurements were made using a macroscopic RS imaging system for which details were provided elsewhere.⁴⁸ Briefly, the excitation branch of the Raman line-scanning system was composed of a laser centered at 785 nm providing a line-shaped laser excitation and a white light source with illumination between 400 and 700 nm. The laser and white light sources were conveyed through an excitation fiber bundle onto the sample. The Raman signal and the reflected white light were collected by the collection fiber bundle and separated by a dichroic mirror through a galvanometer-based system designed for scanning the tissue surface. An imaging spectrometer generated hyperspectral Raman images, and an RGB (red-green-blue) camera was used to collect white light images. The spatial resolution of the hyperspectral images was $250\mu \mathrm{m}$, and the spectral resolution was $6{\mathrm{cm}}^{-1}$. The Raman spectra were acquired in the fingerprint region from 400 to $1900{\mathrm{cm}}^{-1}$. The measurements were made in non-contact mode at a working distance of 40 mm from the specimen. The FOV of both the Raman hyperspectral images and the white light images was $1{\mathrm{cm}}^2$. The system was controlled using custom software (LabVIEW 2018 version 18.0f2, National Instruments, Austin, Texas, United States) that allowed acquisition parameters to be set by the user, including laser power, exposure time per line, and number of repeated measurements (accumulations) for each image.⁴⁸

2.2.

Patient Selection

Six patients who underwent breast surgery (lumpectomy or mastectomy) following a diagnosis of invasive breast carcinoma (ductal or mucinous) were recruited for this study. All patients recruited in the study were women undergoing breast surgery for the first time. The patients did not receive neoadjuvant therapy and had a cancer grade inferior to four.

Fresh patient specimens were utilized to build an ex vivo dataset of Raman hyperspectral images combined with histopathological and clinical data. Informed consent was obtained before each patient underwent surgery (McGill University Health Center Ethics Committees, approval number 2021-7940). Clinical data available included age, tumor type, and size, and patient demographic details were provided (Tables 1 and 2). Each tumor size was characterized by its largest dimension measured by the pathologist at the time of intraoperative consultation.

Table 1

Clinical and pathological characteristics of all patients undergoing breast surgery who were recruited as part of the study.

Table 2

Demographic and clinical information of the six patients included in the study.

2.3.

Specimen Handling and Ex Vivo Spectroscopic Measurements

Eight specimens were extracted from six patients by a surgeon (S.M. or F.T.). The patients were labeled P1 to P6. In cases where more than one specimen was obtained for the same patient, each was labeled either S1 or S2. A total of eight samples were analyzed that were all subsequently sent to pathology for margins assessment as part of the regular clinical workflow. Prior to this, the specimens were weighed, measured, inked, and sliced into 5 mm sections, as per institutional protocols.

Depending on the size of the tumor, Raman imaging measurements and spatial registration with histology were achieved using either of two methods (Fig. 1). All samples imaged with the system were slices of lumpectomy specimens that had a thickness of $\sim 5\mathrm{mm}$. One method was used for the cases where the largest dimension of a tumor, measured ex vivo by the pathologist, was superior to 1 cm. For those cases, it was possible to cut the specimen without compromising the microscopic margin evaluation and the standard post-surgery clinical diagnostic workflow. A second method was used in cases where the largest dimension of the tumor was inferior to 1 cm. A different method was then required to ensure tissue cutting could be made without compromising the diagnosis.

Fig. 1

Schematic representation of the two specimen handling methods leading to ex vivo spectroscopic measurements: (a)–(f) tumors larger than 1 cm and (g)–(k) tumors smaller than 1 cm. (a) A sample was cut from the specimen slices and (b) placed onto a black-coated aluminum sheet disposed on an EPS sheet. The orientation of the margins was marked on the EPS sheet and a photograph was taken. (c) The EPS sheet was then placed under the Raman imaging system for measurements. (e) The sides of the sample were inked with different colors to indicate the orientation of the margins, and the sample was transferred into a pathology cassette before being put in formalin for fixation. (f) An H&E-stained image of the sample was analyzed by the pathologist. The white light image and the specimen photograph were used to register the position of the Raman measurements with the H&E image. (g) The entire specimen slice was put on a black-coated aluminum sheet (h) that was placed onto an EPS sheet. The orientation of the margins was marked on the EPS sheet, and a 3D-printed grid was placed on top without direct contact with the specimen. A photograph was taken, and the pathologist indicated a square of the grid that contained the structures of interest. (i) The EPS sheet was placed under the Raman imaging system and the grid was used to record the location of the measurement. The grid was removed without moving the specimen slice, and the Raman measurements were made. (j) The specimen slice was then transferred to another EPS sheet where it was pinned alongside the other slices of the specimen, when applicable. The orientation of the margins was indicated on the EPS sheet, and it was placed in formalin for fixation as per standard of care. (k) This resulted in an H&E-stained image of the entire specimen slice where the pathologist indicated the position of the tumor. The white light image and the specimen photograph were used to register the position of the Raman measurement on the H&E image.

For tumors larger than 1 cm, one or two samples were cut from different slices [Fig. 1(a)]. Those samples were selected by the pathologist (L.R.) based on visual inspection with the objective of having, whenever possible, cancer, fat, and normal breast tissue for each patient. The samples were placed on a black-coated aluminum sheet disposed on an expanded polystyrene (EPS) foam sheet. The orientation of the sample relative to the margins of the specimen (superior, inferior, posterior, and anterior) was noted on the EPS sheet, and a photograph was taken [Fig. 1(b)]. The EPS sheet was then placed under the imaging probe, and a white light measurement was taken before initiating the Raman imaging sequence [Fig. 1(c)].

For tumors smaller than 1 cm, the entire slice was left intact [Fig. 1(g)]. A slice selected by the pathologist after visual inspection of the specimen was placed on a black-coated aluminum sheet. The slice was then placed on an EPS sheet, and a note was made of the orientation of the margins. A custom 3D-printed guiding grid, for which all squares were the size of the system FOV, was created. The grid was then placed on top of the slice—without contact with the tissue—and a photograph was taken [Fig. 1(h)]. The pathologist indicated which square of the grid contained tumor tissue. The EPS sheet was placed under the imaging probe and aligned so the square area associated with the region of interest could be imaged. The selected square was identified, and the grid was carefully removed without moving the tissue slice. A white light measurement was taken before initiating the Raman imaging sequence [Fig. 1(i)].

For both methods, the laser power was set at maximum, resulting in $\sim 500\mathrm{mW}$ being delivered to the tissue surface over a line of width $350\mu \mathrm{m}$. The line was scanned over the entire FOV, with each acquisition associated with three repeat measurements that were subsequently averaged to maximize the signal-to-noise ratio. The only exception was the fifth patient where the time allocated for acquisition was limited. Therefore, for that patient, the number of repeat measurements was set to one. Acquisition time was set at 10 s per line: 40 lines, for a total of 20 min when doing three repeat measurements. The exposure time was selected to respect the 1-h timeframe imposed by the pathologist after surgical excision to avoid tissue degradation that could compromise diagnostic accuracy.

After spectroscopic measurements, the orientation of the margins was marked with ink with different colors on each side of the sample. The samples were then placed into a pathology cassette [tumors larger than 1 cm, Fig. 1(e)]. The specimen slices were all pined to a new EPS sheet with the orientation of the margins written on it [(tumors smaller than 1 cm, Fig. 1(j)] as per standard of care. All tissues were then fixed in 10% neutral buffered formalin, embedded in paraffin, and sectioned into slides. They were then stained with hematoxylin and eosin (H&E) as per institutional standards, resulting in one stained image for each sample or one stained image for the entire slice if the tumor was smaller than 1 cm [Figs. 1(f) and 2(k)].

Fig. 2

Average processed Raman spectra from breast tissue with standard deviation shown for each spectral bin. The spectra were classified into three categories: fat identified by model A, and cancer or normal predicted by model B after the exclusion of the fat spectra. The five spectral features used for classification (model A: 940, 1004, $1304{\mathrm{cm}}^{-1}$; model B: 940, 1004, 1129, $1155{\mathrm{cm}}^{-1}$) are identified by vertical lines.

Digitized slides were reviewed by a pathologist (L.R.), who classified the tissue into three categories: (1) cancer (tumor cells, tumor stroma, or necrosis); (2) normal: either normal breast (connective tissue, stroma, fibroblast, collagen) or breast parenchyma (ducts and lobules); and (3) fat (adipose cells). The areas containing cancerous cells were annotated by the pathologist.

2.4.

Registration with Optical Measurements and Histology Labels

Two three-step methodologies were developed to register spatially the histology images to the Raman hyperspectral images. For tumors larger than 1 cm, (1) the H&E-stained images were superposed to the photograph of the sample and rotated to ensure that the contours of the stained tissue sections matched as much as possible the contours seen in the photograph. The colored ink marking the margin orientation on the samples, as well as the irregularities associated with its shape, was used as geometrical landmarks guiding the spatial registration. (2) The white light image acquired with the system [Fig. 1(f)] was superposed to the picture of the sample with the margins annotated [Fig. 1(b)]. Again, margin orientation and visible structures, including blood vessels and regions with high adipose content, were used as tissue-based fiducial markers to ensure an accurate superposition of the white light image with the photograph of the specimen. (3) The histopathology image was then truncated to fit the size of the white light image and rotated 90 deg to compensate for the difference in orientation between white light images and Raman hyperspectral images. The resulting images were then comparable to the Raman hyperspectral image.

Similarly, three steps were followed for tumors smaller than 1 cm. (1) The H&E-stained images were superposed to the photograph of the specimen slice with the guidance grid [Fig. 1(h)] and rotated to ensure that the contours of the stained tissue sections matched as much as possible the contours seen in the photograph. (2) The white light image acquired with the system [Fig. 1(k)] was superposed to the specimen photograph and aligned with the corresponding square of the grid. (3) The histopathology image was then truncated to fit the size of the white light image and rotated 90 deg to compensate for the difference in orientation between white light images and Raman hyperspectral images.

2.5.

Data Processing and Exclusion Criteria

The data processing steps preceding the application of the machine learning models led to the extraction of the inelastic scattering signature for each spectrum within the hyperspectral images. Background sources of signal non-specific to tissue Raman scattering, e.g., intrinsic fluorescence from tissue biomolecules,⁴⁹ were isolated and removed from the spectra. Spectral pre-processing included the following steps: (1) median filter to each spectrum to eliminate cosmic ray events, (2) $\sigma$-axis (wavenumber shift) calibration based on a Raman spectrum acquired in acetaminophen, (3) normalization with a National Institute of Standards and Technology (NIST) Raman standard (SRM 2214) to correct for the instrument response, (4) removal of background signals using the custom algorithm BubbleFill,⁵⁰ (5) removal of the background signal from the chromatic triplet of the system by subtracting an image acquired with nothing in focus, with a large integration time,⁵¹ and (6) standard normal variate (SNV) normalization. The five first columns of the hyperspectral images were excluded. The NIST standard being used to correct the instrument response was smaller than the FOV of the system; therefore, the correction could not be correctly performed for a small portion of the hyperspectral image.

A quantitative spectral quality factor (QF) metric (with a maximum value of 1) was computed for each spectrum of the image. It provided a statistical assessment of the likelihood the SNV-normalized signal was associated with tissue Raman peaks or stochastic noise.⁵² Spectra with a QF metric inferior to 0.5 were excluded from the images. The percentage of pixels associated with a QF factor inferior to 0.5 was computed for each image.

2.6.

Machine Learning Workflow

A study was previously conducted involving 20 patients undergoing breast surgery.⁴⁶ An RS point probe system was used to measure the spectral fingerprint of normal breast and cancer tissue ex vivo. Overall, 238 Raman spectra were acquired in that study consisting of 93 measurements associated with adipose tissue, 87 with cancer, and 58 with non-fat normal breast structures. These categories were labeled fat, cancer, and normal, respectively. This dataset was used to train classifiers using SVM to distinguish fat from normal structures, as well as to detect cancer out of non-fat normal breast tissue. The dimensional reduction was performed to ensure that the models used only a small number of biochemically interpretable spectral bands for classification. This also reduced the likelihood of overfitting and increased the model’s potential to generalize well to new data.

Two models trained on breast tissue spectra acquired with the point probe system were chosen for this study. The first model, model A, distinguished fat tissue from non-fat tissue (either normal or cancer) and was based on three spectral features centered at 940, 1004, and $1304{\mathrm{cm}}^{-1}$. The second model, model B, classified cancer versus normal tissue (excluding fat data) and required four spectral features centered 940, 1004, 1129, and $1155{\mathrm{cm}}^{-1}$. Those SVM models were directly applied to each of the spectra composing the hyperspectral Raman images acquired with the line-scanning system for eight breast specimens. Models A and B returned, for each spectrum, a posterior probability $0\le p\le 1$ that the spectrum belonged to a given class. To obtain predictions from these probabilities, a threshold of classification was needed. This threshold was chosen to optimize sensitivity and specificity during the previous study. A prediction was then obtained for the class of each spectrum (i.e., each pixel) composing the hyperspectral images.

The classification of the hyperspectral images was done in two steps. First, model A was applied to all spectra to identify fat spectra with the hyperspectral images. Then, model B was applied to the spectra identified as non-fat to classify them as either cancer or normal. The results of the classification were expressed through classification maps where each pixel corresponded to the tissue class predicted. The threshold of classification for model A to classify a spectrum as fat was increased to maximize sensitivity. This meant that less spectra were classified as fat since the spectra needed a higher probability $p$ to be classified as fat. This limited the number of spectra falsely identified as fat. Increasing the threshold minimally affected the classification performance in the original point probe dataset (initially sensitivity/specificity was 96%/99% and, after increasing the threshold, it was 100%/95%). Adipose spectra not classified as fat were all classified as normal, which did not impact possible clinical applications as both normal and fat tissues are viewed as healthy tissue by pathologists and surgeons.

3.1.

Spectroscopic Measurements

The application of the Raman imaging protocol resulted in eight hyperspectral images from six patients with co-located histopathology analyses. A total of 11,520 spectra were acquired, with each spectrum corresponding to 1 pixel. After the exclusion of the spectra presenting a spectral QF inferior to 0.5, a total of 8020 spectra remained.

Classification sequentially used models A and B to split the dataset into three categories fat ($n=6008$ spectra), normal ($n=1137$), and cancer ($n=875$). Figure 2 presents the average processed spectra for each category and the standard deviation associated with each spectral bin. The spectral features used for tissue classification by the SVM models are highlighted in Fig. 2. The Standard deviation for fat spectra was smaller than normal and cancer due to a higher signal-to-noise ratio in fat spectra.

3.2.

Machine Learning Models and Biomolecular Predictions

Figure 3 presents classification results for all eight samples considered in this study. Each row corresponded to a different sample that was labeled according to an identification number. An annotation was provided in Fig. 3 (first column) of the microscopic analysis results provided by a pathologist to establish the main tissue constituents. In cases where only adipose tissue could be detected by the pathologist, the label fat was used. The second column in Fig. 3 showed the photograph of the sample region that was imaged with the Raman imaging system, and the third column showed the corresponding histology image (stains: H&E) cropped to match the FOV of the Raman system. The fourth, fifth, sixth, and seventh columns showed the intensity maps for the spectral features 940, 1004, 1129, and $1159{\mathrm{cm}}^{-1}$, respectively. These bands corresponded to the spectral features included in model B trained to detect cancer from single-point RS measurements. The Raman band images were normalized to their maximum to ensure they were all represented on the same colormap to ensure relative intensities could be compared across all samples. The last column in Fig. 3 showed classification results for each spectrum as fat, normal, or cancer. The spectra with a QF below 0.5 were excluded from the classification maps.

Fig. 3

Imaging results for the eight samples considered in the study. Each line is for a different specimen, with the first column presenting an identification number and a label indicating tissue type. The second column presents a photograph of each specimen, and the third column shows the collocated H&E image used for microscopic analysis by a pathologist. The fourth, fifth, sixth, and seventh columns show inelastic scattering contrast for the Raman bands at 940, 1004, 1129, and $1159{\mathrm{cm}}^{-1}$, respectively. All images were normalized to their maximum value, and thus, all have a maximum value of 1 and a minimum value of 0. The eighth column presents the results of applying the machine learning models to each pixel image with the Raman system. Spectra that were classified as fat are in pink, spectra identified as normal or benign tissue are in yellow, and cancer predictions are represented by the color purple.

Spectral images were acquired by summing data over three accumulations (i.e., three consecutive imaging sequences) to increase the signal-to-noise ratio. The same process as above was applied to spectral imaging data associated with only one accumulation, effectively reducing the image time by a factor of three (Fig. 5). The only specimen for which this was not done is P5 for which only one imaging sequence had been performed, i.e., no accumulation was possible.

The first four samples shown in Fig. 3 (P1-S1, P2, P4-S1, P5) contained cancer cells. The histology for P4-S1 showed that the sample was entirely composed of invasive mucinous carcinoma albeit with lower cellularity in the lower part of the image when compared with the upper area. The Raman models predicted that all pixels with QF > 0.5 were cancer. However, the spectral quality of the spectra in the image was very low for this specimen, and only 3% of all spectra remained after exclusion based on the spectral QF. The lower QFs associated with P4-S1 were associated with a larger fluorescence background—relative to the inelastic scattering signal—when compared with the other specimens in the study.

The histology for P2 showed that half of the sample was constituted of invasive ductal carcinoma and the other half was adipose tissue. Model A identified adipose structures in a manner that was consistent with histology. Model B predicted that 74% of the remaining spectra belonged to the cancer category, providing an overall agreement with histology. As shown by the photograph of the sample, P2 was not large enough to fill the FOV of the imaging system. Spectra taken on regions where there was no biological sample (i.e., the black-coated aluminum substrate) were mostly excluded from classification due to low QF. However, spectra taken on the edge of the sample contained tissue signal and traces of black-coated aluminum Raman signal. In fact, classification results showed that spectra surrounding an adipose region were classified as normal instead of fat. This was due to the contamination of the adipose signal by the aluminum signal that contained an inelastic scattering signal around $1004{\mathrm{cm}}^{-1}$. The presence of an artifactual signal from the substrate confounded model predictions around the border of that specimen. This issue appeared on the boundary of all samples that were not large enough to fill the entire FOV of the system, i.e., P2, P1-S1, P4-S2, P3, and P1-S2. The effect of this signal contamination is illustrated in Fig. 4. This effect also resulted in the presence of individual pixels that were predicted as cancer around the periphery of specimens, including P4-S2.

Fig. 4

Classification results for specimen P1-S2 illustrating that the presence of non-tissue artifacts can confound machine learning model predictions: (a) classification map obtained when applying model A (trained to distinguish adipose tissue from either normal/benign or cancer tissue) followed by the application of model B (trained to distinguish normal/benign from cancer tissue) to pixels classified as non-adipose by model A. Pixels in pink are classified as fat, pixels in yellow as normal, and the only pixel classified as cancer is in purple. (b) Three representative Raman spectra are shown as follows: a spectrum from black-coated aluminum, a spectrum acquired at the border between the sample and the substrate, and a spectrum of adipose tissue. The location of each spectrum is indicated by a cross in panel (a) that has the same color as its corresponding spectrum in panel (b). A vertical line is used to indicate the spectral feature at $1004{\mathrm{cm}}^{-1}$ in panel (b).

For P1-S1, histology showed the sample contained invasive ductal carcinoma with high cellularity in the upper part of the image and adipose tissue in the upper right corner of the image. Model A correctly identified a fat region in the same region as the one shown by histology. Sixty-two percent of the remaining spectra were classified as cancer by model B. P5 contained a tumor of 3 mm size that was an invasive ductal carcinoma surrounded by adipose tissue and normal breast. Most of the spectra were classified as fat except for a small region at the bottom of the image that appeared to correspond to the tumor seen on histology, here slightly shifted to the left. After the exclusion of all fat-predicted spectra, model B classified 21% of the remaining spectra as cancer (34 spectra). The intensity map of the $1004{\mathrm{cm}}^{-1}$ band showed a content in the phenylalanine band for the entire tumor region that was like that associated with P4, P2, and P1-S1. The content in proteins, using the band at $1155{\mathrm{cm}}^{-1}$ as a surrogate, also appeared comparable to the other tumors.

The last four samples (P4-S2, P6, P3, P1-S2) contained no cancer cells. P4-S2 contained adipose and other normal breast tissue structures. Predictions of the models for that sample were consistent with histology, with most of the spectra being classified as normal or fat, and less than 1% of all pixels classified as cancer. The eight spectra on the edge of the sample classified as cancer contained a mix of normal tissue and aluminum signal, which resulted in a higher spectral feature around $1004{\mathrm{cm}}^{-1}$ (Fig. 4). The two other spectra classified as cancer had similar spectral features as normal spectra with a phenylalanine peak slightly higher. P6 contained mostly adipose tissue and a thin strip associated with other normal breast tissue structures. Model A classified 80% of all spectra as fat, with all other spectra being classified as normal. The histology for P3 showed mostly adipose and other normal breast tissue. The white light image of that specimen also showed residual black ink used for margin identification by the pathologist. This led to the exclusion of most of the spectra in the upper left part of the image based on low QF. Most of the remaining pixels for that sample were classified as fat. Histology for P1-S2 showed adipose and normal breast tissues. Most of the pixels in that image were classified as fat.

As previously mentioned, P2, P3, P4-S2, P3, and P1-S2 did not fill the entire FOV of the system, and Fig. 3 shows that the spectra at the border between the samples and the substrate were misclassified as normal or cancer. This was attributed to the contamination of the Raman signal by spectral features from the black-coated aluminum substrate, particularly around $1004{\mathrm{cm}}^{-1}$. Figure 4 presents the impact of the aluminum signal in the sample P1-S2. Figure 4(b) shows a spectrum of black-coated aluminum (excluded from the classification due to low QF), a spectrum from the edge of the sample, and a spectrum from adipose tissue. The location of each spectrum was indicated by a cross in the classification map [Fig. 4(a)]. Figure 4(b) shows an aluminum spectral feature around $1004{\mathrm{cm}}^{-1}$ that was present in the spectrum acquired on the edge of the sample, leading to the normal classification. Raman signal contamination by black ink was also observed in P3, and the spectra in the region with black ink were classified as normal. The samples P1-S2, P3, and P4-S2 also showed spectra classified as cancer on the edge of the samples where there was contamination by aluminum signal.

Table 3 reports that, for each Raman image, the percentage of pixels that had a QF inferior to 0.5, the percentage of all remaining pixels that were predicted as fat, and the percentage of pixels that were predicted as cancer. The latter metric was computed based only on the pixels that were predicted by model A not to belong to the fat category. Table 3 also includes the average posterior probability $p$ (and its standard deviation) computed only based on the non-fat pixels. The latter metric could be used as a quantitative cancer burden marker, with larger values being associated with a more important burden. All specimens that contained cancer cells had an average $p$ that was larger than 0.3, whereas all specimens that did not contain cancer had a value of 0.15 or inferior.

Table 3

Statistics for several metrics associated with specimen images acquired with the hyperspectral Raman images. Computed quantities include the percentage of pixels excluded from the analyses because they had a spectral QF inferior to 0.5, percentage of the remaining pixels that were classified as fat (model A), percentage of the pixels predicted as cancer by model B from non-fat measurements, and average and standard deviation (std) of the posterior probability value p that the non-fat pixels were predicted to belong to the cancer rather than the normal category (model B).