2.1.

Step 1: Cell Segmentation and Feature Extraction

Individual biological cells are identified in the input image using a cell segmentation method suited for the tissue type, see e.g., Refs. 24, 53, and 54. For each identified cell, the location $\overrightarrow{X}=(x_1,x_2)$ of its nucleus in the image is stored together with $f$ extracted cell features, e.g., the average marker intensity for each fluorescent color channel, the cells size and shape, etc. The final output of the cell segmentation step is a list of cells with their coordinates $\overrightarrow{X}$ and $f$ features, as shown in Table 1.

Table 1

Example output of the cell identification (Step 1).

Cell segmentation in an fm-IHC laboratory setting is normally performed using established software, e.g., CellProfiler,²⁸ ImageJ,²⁹ inForm (Akoya Biosciences, Inc.), or HALO (Indica Labs), with supervision from an expert in the field, as the process depends on multiple parameters, such as tissue type and tissue stains, and could require manual annotation as training data. This was the setting and scope of our study, but it is worth mentioning that recently published state-of-the-art methods on cell segmentation^30,31 might soon become widespread in such a setting, providing even better results (see Sec. 4.1).

2.2.

Step 2: Assignment of Cells to Target Grid

After choosing a target grid spacing $d$, all identified cells are assigned to the square target grid by binning their original coordinates $\overrightarrow{X}$ to the nodes of the grid using ${\overrightarrow{X}}_g=\lfloor \overrightarrow{X}/d\rceil$, whereas $\lfloor ·\rceil$ denotes conventional rounding to the next integer and ${\overrightarrow{X}}_g$ corresponds to coordinates of the target grid nodes (GNs) (i.e., indices of pixel locations in the final output image). While for most biological tissue types, $d=5\mu \mathrm{m}$ is a good choice, we provide guidelines for how to systematically chose the target grid spacing $d$ in Supplementary Material S.3.

Using this method, any two cells within a distance of $d\surd 2$ may be assigned to the same GN ${\overrightarrow{X}}_g$. To achieve a one-to-one relation of cells to output pixels, we require that each cell be assigned uniquely to a GN and that any GN can hold at most one biological cell. This one-to-one relation is the essential property of Cell2Grid data representation.

Assignment conflicts are resolved successively by applying Munkres’ algorithm⁵⁵ to all cells within the local $3\times 3$ subgrid around a GN with conflicts. If the number of cells $k$ within this window is higher than the number of GNs in the subgrid, the $5\times 5$ subgrid is considered. This process is continued until a maximum window size $w_{\max }\times w_{\max }$ is reached. If the number of cells still exceeds the number of available GNs, $k>{(w_{\max })}^2$, then $k-(w_{\max }{)}^2$ cells are deleted from the data. This process is explained in more detail in Supplementary Material S.1.

After conflict resolution, every cell has either been uniquely assigned to a GN or was deleted due to unresolvable conflicts. Similarly, every target GN either holds exactly one or no cells.

2.2.1.

Grid spacing and assignment conflicts – theoretical analysis

In this section, we present analytical expressions for the expected number of GNs with assignment conflicts and the total expected cell loss after local conflict resolution. These expressions assume a random uniform distribution of cell locations over the entire image and depend on the overall cell density $\rho$, the target grid spacing $d$, and the local conflict resolution window size $w$.

Using results from the ball-into-bin model that describes the occupancy problem,⁵⁶ the random allocation of $n$ cells into $m$ nodes is described by the Poisson distribution $\mathrm{Pois}(\lambda ,X=k)=\frac{{\lambda }^k}{k!}{e}^{-\lambda }$, which is the probability mass function for a single bin (GN) containing $k$ balls (cells) with $\lambda =\frac{n_{\text{cells}}}{m_{\text{nodes}}}$, and $m_{\text{nodes}}$ being the number of GNs of the target grid. Therefore, $\mathrm{Pois}(\lambda ,X\ge 2)$ estimates the fraction of GNs that hold two or more cells (i.e., assignment conflicts) and is only dependent on $\lambda$. For our purpose, $\lambda$ can be defined using the cell density $\rho =\frac{n_{\text{cells}}}{A}$ of $n_{\text{cells}}$ in an image area $A$ and target grid spacing $d$ as $\lambda (d)=\frac{n_{\text{cells}}}{m_{\text{nodes}}}=\frac{n_{\text{cells}}}{(A/d^2)}=\rho d^2$. Therefore, the estimated fraction of GNs with $k$ assigned cells is written as

Eq. (1)

$$\mathrm{Pois}(\lambda (d),k)=\frac{{(\rho d^2)}^k}{k!}{e}^{-\rho d^2}.$$

Using this result, we express the fraction of GNs that hold two or more cells (i.e., nodes with assignment conflicts) as follows:

Eq. (2)

$${\mathrm{GN}}_{\mathrm{conf}}(d)=\mathrm{Pois}(\lambda (d),k\ge 2)=1-\mathrm{Pois}(\lambda (d),k=0)-\mathrm{Pois}(\lambda (d),k=1).$$

The fraction of cells in conflict is the number of cells that are not allocated to one-cell GNs,

Eq. (3)

$${\mathrm{C}}_{\mathrm{conf}}(d)=1-\frac{m_{\text{nodes}}}{n_{\text{cells}}}\mathrm{Pois}(\lambda (d),k=1).$$

Next, we estimate the number of unresolvable conflicts when a local conflict resolution window of size $w$ is used ($w$ being an odd integer $\ge 3$). Using a modified target grid spacing $d_w=w·d$, we can estimate the number of cells in a $w\times w$ subgrid around a GN with conflicts. Subgrids (or “neighborhoods”) that contain more than $w^2$ cells constitute unresolvable conflicts. Because taking the local neighborhood into account is only relevant when the central node actually contains a conflict, we multiply with ${\mathrm{GN}}_{\mathrm{conf}}(d)$. By neglecting conditional probabilities for $k>w^2$ when there are already $k\ge 2$ cells at the central node, we arrive at

Eq. (4)

$$\begin{gathered} {\mathrm{GN}}_{\mathrm{unres}}(d,w)\approx {\mathrm{GN}}_{\mathrm{conf}}(d,w)·\mathrm{Pois}(\lambda (wd),k>w^2) \\ ={\mathrm{GN}}_{\mathrm{conf}}(d,w)·(1-\sum _{k=0}^{k=w^2}\mathrm{Pois}(\lambda (wd),k)). \end{gathered}$$

Finally, we estimate the cell-loss-fraction due to unresolvable conflicts by considering cells that have been assigned successfully using ${\mathrm{C}}_{\text{loss}}=(n_{\text{cells}}-N_1-N_2)/n_{\text{cells}}$. Here, $N_1$ is the number of cells located in “unsaturated” local neighborhoods (all cells are assigned if an area of size $(wd{)}^2$ contains $k\le w^2$ cells):

Eq. (5)

$$N_1=m_{\text{nodes}}\sum _{k=0}^{w^2}\frac{k}{w^2}\mathrm{Pois}(\lambda (wd),k).$$

Note that the summation term contains $k/w^2$, which is the average number of cells per GN when $k$ cells are in an area of $w^2$ GNs. In contrast, $N_2$ is the number of cells that are successfully assigned during conflict resolution in “saturated” areas (in areas of size ${(wd)}^2$ containing $k>w^2$ cells only $w^2$ cells are assigned and $k-w^2$ cells need to be deleted):

Eq. (6)

$$N_2=m_{\text{nodes}}·\mathrm{Pois}(\lambda (wd),k>w^2)=m_{\text{nodes}}(1-\sum _{k=0}^{w^2}\mathrm{Pois}(\lambda (wd),k)).$$

In $N_2$, we simply count the number of GNs that are part of oversaturated areas because each of them will contain exactly one cell after conflict resolution. Put together and simplified, the estimated cell loss after conflict resolution is

Eq. (7)

$$C_{\text{loss}}(w,d)=1-\frac{w^2}{\lambda (wd)}-\frac{w^2}{\lambda (wd)}\sum _{k=0}^{w^2}\mathrm{Pois}(\lambda (wd),k)·(\frac{k}{w^2}-1).$$

In Fig. 2, we visualize $C_{\text{loss}}(w,d)$ for different values of $w$, $d$, and cell densities $\rho$.

Fig. 2

Expected cell loss for different target grid spacings $d$, conflict resolution window sizes $w$ and cell densities $\rho$. Left: fixed target grid spacing $d=5\mu \mathrm{m}$ and vertical dashed line denotes simplest nontrivial conflict resolution size $w=3$. Right: fixed conflict resolution window size $w=3$ and vertical dashed line denotes target grid spacing chosen for all presented experiments ($5\mu \mathrm{m}$). Cell density of our dataset is $\rho \approx 0.012{\mu \mathrm{m}}^{-2}$ (orange dashed line).

Note that for comparison with empirical data that use an adaptive local conflict resolution window $w$ (see Supplementary Material S.1), using $w=3$, provides the best approximation.

2.3.

Step 3: Image Creation

In the last step of Cell2Grid, a low-resolution output image is created. Each node of the target grid stores the $f$ extracted features of its assigned cell, like pixels in conventional RGB images storing three color values. This final data structure is a tensor of size $K_{x_1}\times K_{x_2}\times f$ where $K_{x_1}$ and $K_{x_2}$ are the number of GNs in $x_1$ and $x_2$ dimension. The GNs without cells are assigned zero as a default value for every cell feature. Because a square grid was used, this tensor can be converted directly into an image with $f$ color channels, e.g., using the multipage supporting tagged image file format (TIFF). This multipage image, which we term Cell2Grid image, is the final output of the image creation step and the whole Cell2Grid algorithm (see Figs. 1 and 8 for examples).

3.1.

Data for Experiments

Surgically removed tumor tissue samples from $N=54$ patients were collected from the local biobank affiliated to the university hospital where patients had been treated (Ethics Committee number: 28-342 ex 15/16). Each patient’s tumor recurrence status after 5 years was labeled as either relapse (17 patients) or healthy (37 patients), serving as the variable to be predicted by classification CNNs.

To obtain fm-IHC images (see Fig. 1 for an example), one formalin-fixated paraffin-embedded tumor tissue sample per patient was stained with fluorescence-conjugated antibodies targeting CD3, CD8, CD45RO, PD-L1, FoxP3, Her2, and DAPI. Of each whole-slide scan, several ROIs covering an area of $674\times 506\mu \mathrm{m}$ were selected by a human expert and were recorded with $20\times$ magnification ($1344\times 1008\mathrm{px}\sim 0.5\mu \mathrm{m}/\mathrm{px}$ resolution) using a Vectra 3 microscope (PerkinElmer, Inc.). Spectral unmixing of raw images was performed using inForm software (Akoya Biosciences, Inc.). In total, $n=1353$ images ($\approx 25$ per patient) were recorded.

3.2.

Grid Spacing, Assignment Conflicts, and Processing Time

We first investigated the empirical algorithm runtime and compression ratio of Cell2Grid and if real cell loss numbers follow our analytical expressions. Cell segmentation (step 1) was performed using inForm software (Akoya Biosciences, Inc.) with parameters set by an IHC expert. Cell nuclei were identified using DAPI stain, and cell membranes and cytoplasm were subsequently identified using remaining markers. On average, 4131 cells per image were identified (cell density $\rho =0.0121\mu {\mathrm{m}}^{-2}$). For each cell, we extracted $f=6$ features using the average marker intensity across the entire cell area of each fluorescent color channel (excluding DAPI).

We then investigated how different target grid spacing values influenced Cell2Grid, using a maximum local conflict resolution size $w_{\max }=7$, see Figs. 3 and 4. Algorithm runtime remained flat below $d=5\mu \mathrm{m}$ and increased steeply afterward. As expected, image compression ratio increased with the square of the target grid spacing $R={(d[\mu \mathrm{m}]/0.5\mu \mathrm{m})}^2$ with $0.5\mu \mathrm{m}$ being the original image resolution.

Fig. 3

Mean algorithm runtime per image and image compression ratio (double-log plot) for different target grid spacings. Vertical dashed line represents our final choice of $d=5\mu \mathrm{m}$.

Fig. 4

Cell assignment properties for different target grid spacings averaged over all images of our dataset. Left: percent of GNs with conflicts before conflict resolution. Right: initial percent of cells in conflict and cell loss after conflict resolution. Vertical dashed lines represent our final choice of $d=5\mu \mathrm{m}$.

Figure 4 shows that the percentage of GNs with conflicts and cell loss due to unresolvable conflicts followed the expected behavior [Eqs. (2), (3), and (7)]. However, the percentage of cells in conflict before conflict resolution was lower than expected below $d=5\mu \mathrm{m}$ and higher for larger values of $d$ due to non-uniform distributions of cells in real biological tissue. More importantly, cell loss remained neglectable below $d=5\mu \mathrm{m}$ and increased for higher values.

Based on these findings, we decided to use $d=5\mu \mathrm{m}$ as the target grid spacing of Cell2Grid for our following experiments. Using our dataset, this value provided a compression ratio of $R=100$, average cell loss of 2.06 cells (0.05%) per image, and processing time of 1.56 s per image (excluding cell segmentation).

To put the processing time of our method into perspective, we compared it with three conventional image rescaling methods and with the creation of a cell graph using cell segmentation data. Cell graphs were created using $k$-nearest-neighbor algorithm ($k=5$, see Fig. 8(c) for an example). Figure 5 shows the mean data processing time for each method but does not include time for cell segmentation required for cell graph creation and Cell2Grid. The comparably high variance in Cell2Grid is a result of the variance in cell densities and subsequent conflict resolutions per image, the latter being the main driver of computation time in Cell2Grid.

Fig. 5

Mean and 95% confidence interval for data processing times per sample shown for image rescaling with bilinear (bil) and cubic (cub) interpolations and Lanczos (lanc) sampling, cell-graph (cgr) creation and Cell2Grid (c2g) representation, the latter two starting from cell segmentation data and do not include cell segmentation processing time. All presented times include read and write operations, multipage-TIFF-file unpacking and stacking (bil, cub, and lanc), and necessary metadata extraction.

3.3.

Phenotype Counts and Spatial Features

Next, we investigated if conventional spatial features calculated from cell segmentation output tables were altered due to the local spatial approximations performed in Cell2Grid. For every individual image of our colon cancer relapse dataset, we calculated the total number of $T$ cells (CD3+) and regulatory $T$ cells (CD3 + FoxP3 + CD8−) to investigate the potential change in a dense and a rare cell population, respectively. Additionally, we investigated a potential change in the average distance between $T$ cells and their nearest PD-L1 positive cell as well as the number of $T$ cells around PD-L1 positive cells within a radius of $50\mu \mathrm{m}$ using the phenoptr $R$ package.³⁴ Marker thresholds for these phenotypes were set manually. Figure 6 compares these feature values calculated with original cell segmentation coordinates and coordinates as approximated by Cell2Grid. Regression plots (Pearson $R>0.99$, $p<0.0001$) showed strong agreement between both methods, which confirms that the small number of deleted cells introduces only insignificant changes in the data. Additionally, Bland–Altman plots, offering an alternative way to test the same hypothesis, show that at least 95% of the method differences fall within the predefined interval of $\pm 1.96$ times the standard deviation (SD), which is a sign of good agreement between the two measurements.⁵⁷ In particular, no more than three $T$ cells were deleted in any given image. Notably, no regulatory $T$ cell was deleted in any image, indicating that rare cell populations are less likely to be affected by cell loss simply due to their lower abundance. Furthermore, the mean distance between $T$ cells and their nearest PD-L1 positive cell was altered by $<\pm 1.6\mu \mathrm{m}$ for 95% of images, and the count of $T$ cells within a radius of $50\mu \mathrm{m}$ around PD-L1 positive cells was altered by a count of $<\pm 5\text{cells}$ in $>97\%$ of images. Results for additional features, including cell marker distributions within the cells and nucleus and cell shape properties, are shown in Supplementary Material S.4.

Fig. 6

Comparison of conventional spatial features calculated using original cell segmentation coordinates versus Cell2Grid coordinates using Pearson correlation (left column, showing correlation coefficient and $p$-value) and the same data visualized using Bland–Altman plots (right column, showing percentage of measurements lying within an interval of $\pm 1.96$ times the SD around the mean). Top row to bottom: (a) number of $T$ cells per image, (b) number of CD3 + FoxP3 + CD8 cells (regulatory $T$ cells) per image (no change, SD = 0), (c) mean distance between $T$ cells and nearest PD-L1 positive cell, and (d) count of $T$ cells within $50\text{-}\mu \mathrm{m}$ radius around PD-L1 positive cells.

3.4.

Neural Network Image Classification

Finally, we investigated how different CNN architectures performed on Cell2Grid images, and if given a fixed image size, Cell2Grid data representation improved performance compared with standard image rescaling. For the retrospective colon cancer dataset introduced in Sec. 3.1, the goal was to predict the patients’ 5-year tumor recurrence status (relapse/healthy) from histologic images in a supervised training setting. For that purpose, we trained two different CNN architectures (VGG and small interpretable network (SIN), explained below) on Cell2Grid images and on raw images rescaled with three conventional image rescaling methods (bilinear (bil) interpolation, bicubic (cub) interpolation, and Lanczos (lanc) sampling),⁵⁸ all using a final resolution of $5\mu \mathrm{m}/\mathrm{px}$.

To investigate the effects of additional cell shape features in Cell2Grid, we also trained a CNN on Cell2Grid images that contain, apart from the six default mean marker channels, three additional image channels describing the cell shapes (nucleus size, cell size, and cell axis ratio). We refer to these image set as c2gShape. Additionally, we trained a GNN on a cell graph created from the same cell segmentation data used for Cell2Grid.

Figure 7 shows the entire experiment setup, and a side-by-side comparison of all data modalities is shown in Fig. 8. A description of interpolation-free data augmentation schemes required for Cell2Grid images is presented in Supplementary Material S.5.

Fig. 7

Model configurations used in our experiment. Models using our method highlighted in gray.

Fig. 8

Comparison of image representation methods using false color images of six marker channels. Scale bar $100\mu \mathrm{m}$. (a) Original fm-IHC ($0.5\mu \mathrm{m}$ resolution), (b) conventional bil rescaling ($5\mu \mathrm{m}$), (c) cell segmentation output (yellow dots) and cgr (white lines), and (d) Cell2Grid image ($5\mu \mathrm{m}$ target grid) using the same cell segmentation data as shown in panel (c). Image best viewed on screen.

3.4.5.

Results

To assess model performance, we report the balanced classification error rate ${\mathrm{err}}_{\mathrm{bal}}=1-{\mathrm{acc}}_{\mathrm{bal}}$ calculated from the balanced image classification accuracy ${\mathrm{acc}}_{\mathrm{bal}}$ of model predictions compared with the known ground truth label (5-year tumor recurrence status). Results of all models and ten repeated runs each are summarized in Table 3 and Fig. 9. VGG models showed higher variance in error rates compared with SIN, regardless of image processing method. As shown in Supplementary Material S.6, different training settings for VGG did not improve its performance. We found that conventional rescaling methods bil and cub resulted in models with comparable performance on the test set with lanc falling slightly behind. Despite similar validation set error rates, Cell2Grid models maintained lower test set error rates compared with bil, cub, and lanc models, especially for the SIN architecture, indicating better generalization properties and less overfitting. Figure 10 shows a comparison of the training curves of the Cell2Grid-SIN and bil-SIN models and test set receiver operating characteristic (ROC) curves for all ten repeated model runs.

Table 3

Balanced error rates for all models and ten repeated runs. See Fig. 9 for a visual representation.

Model	Balanced error rate median (min–max)		Relative test error rate compared with bil model (lower is better)	Relative improvement compared with bil model
	Validation set	Test set
bil-VGG	0.092(0.060–0.500)	0.105(0.064–0.500)	1.000	—
cub-VGG	0.121(0.084–0.151)	0.108(0.080–0.168)	1.028	—
lanc-VGG	0.140(0.102–0.176)	0.152(0.102–0.183)	1.448	—
Cell2Grid-VGG	0.088(0.046–0.296)	0.086(0.050–0.297)	0.819	18.1%
cgr-GNN	0.163(0.122–0.200)	0.172(0.143–0.178)	—	—
bil-SIN	0.063(0.053–0.100)	0.099(0.078–0.113)	1.000	—
cub-SIN	0.076(0.062–0.095)	0.104(0.055–0.151)	1.051	—
lanc-SIN	0.062(0.050–0.086)	0.089(0.071–0.113)	0.899	10.1%
Cell2Grid-SIN	0.065(0.050–0.083)	0.074(0.059–0.097)	0.748	25.3%
c2gShape-SIN	0.049(0.038–0.057)	0.078(0.042–0.106)	0.788	21.2%

Fig. 9

Balanced validation and test set error rates for all models (lower is better), ten repeated runs summarized by a single boxplot each (two outliers for bil-VGG and one for Cell2Grid-VGG with error rates above 0.2 not shown for clarity, see Table 3). Datasets indicated on the left and network architectures in the middle in bold. Models trained with our method (Cell2Grid and c2gShape) are highlighted in gray.

Fig. 10

Smoothed training curves for loss function and balanced error rate as well as test set ROC curve with mean area under the curve for all ten SIN model runs. (a) SIN trained on bil rescaled images (bil-SIN) and (b) SIN trained on Cell2Grid images (c2g-SIN).

Additional cell shape features, as used in c2gShape-SIN, led to the best validation set error rates, but did not improve upon Cell2Grid-SIN (which use only fluorescent marker features) on the test set across multiple model runs. We attribute this to c2gShape containing more information but also being more prone to overfitting due to its additional input features. Nevertheless, the single best performing model run was a c2gShape-SIN model (3.8% and 4.2% balanced error rate on the validation and test set, respectively). The GNN model used in our experiment showed higher test set error rates than all CNN models. As shown in Table 3 and Fig. 9, models trained on Cell2Grid data led to an 18.1% and 25.3% relative reduction of test set error rate for VGG and SIN models, respectively, compared with conventional bil rescaling.

As an additional experiment, we investigated if higher input image resolution in the bil-SIN models could improve model performance compared Cell2Grid-SIN. We therefore rescaled our data to 2.25 and $1.0\mu \mathrm{m}/\mathrm{px}$ (images sizes $300\times 255\mathrm{px}$ and $675\times 506\mathrm{px}$, respectively) using bil and used the same training setup as before.

Figure 11 shows that test set error rates of bil-SIN decreased with increased image resolution, but remained above Cell2Grid-SIN, even at $1\mu \mathrm{m}/\mathrm{px}$. However, the larger images sizes at higher resolutions led to longer training times due to the increased number of network parameters in the top layers and slower data augmentation. Specifically, per-epoch training time of Cell2Grid-SIN at $5\mu \mathrm{m}$ was decreased by 85% compared with bil-SIN at $1\mu \mathrm{m}$ while simultaneously achieving lower test set error rates.

Fig. 11

Balanced test set error rates and per-epoch training times for bil-SIN models at different resolutions (5, 2.25, and $1\mu \mathrm{m}$) compared with Cell2Grid-SIN ($5\mu \mathrm{m}$). Our method highlighted in gray.

4.1.

Limitations and Future Work

The research scope of this work focused on the investigation of an alternative data representation of high-resolution histologic images to facilitate their use with CNNs. Therefore, the first step of our algorithm directly relied on the output provided by commercial software available in most laboratories and provided a standardized scenario on which to experiment. Moreover, in our current implementation, an IHC expert had to choose which method and parameters were best suited for the biological tissue. Because our method strongly depends on this cell segmentation step, this set of constraints constitutes an important limitation. Our research scope did not include an investigation of the effects of different cell segmentation algorithms, but fully automated cell segmentation is an active field of research^{24,30,31,53,54} that we intend to explore in future work with recently published methods showing promising results.^30,31

Additionally, Cell2Grid requires additional processing time compared to conventional image rescaling. As shown in Sec. 3.2, data preprocessing using Cell2Grid is slower than conventional image rescaling even without accounting for cell segmentation time. Furthermore, cell graph creation was significantly faster than our method, both starting from cell segmentation data. This additional data processing time needs to be weighed against any potential gains of our method for downstream applications, like a potential reduction in CNN training time or classification accuracy.

In our experiments, we focused on a small set of cell features to construct Cell2Grid images, i.e., the mean marker expression within each cell. This enables a direct comparison with conventional image rescaling methods as the number and type of image channels remains the same. To add additional information to Cell2Grid images, other features, such as marker distribution properties across the cell and cell shape features, may be used as additional image channels in Cell2Grid output, as demonstrated by our experiments (see results for c2gShape and Sec. S.4 in the Supplementary Material). However, extracellular information or cell-based information that is either not extracted during cell segmentation or is not used as a dedicated image channel in Cell2Grid is inevitably lost from the data. This is a limiting factor for our method, especially for applications where a pathologist—or an image classification model—requires this information for accurate assessment of the images.

Regarding the assignment of cells to the target grid (step 2 of Cell2Grid), care needs to be taken, as a large target grid spacing $d$ may lead to deletion of individual cells. Even if in small numbers, cell deletion may be undesired if tissue areas with high cell density are of particular interest, e.g., immune cell clusters in insulitis.⁷⁹ In this case, a smaller target grid spacing $d$, a larger local conflict resolution size $w_{\max }$, or both are required, potentially increasing processing time. In our method, the precise location of individual cells is approximated to fit the target grid during cell assignment. However, because cells are moving in live tissue, their exact position in the final tissue section depends on the time of sampling as well as the location and angle of tissue sectioning. We estimate that the net effect of these factors is greater than the local shift in cell positions introduced by Cell2Grid, which for most cells is smaller than the target grid spacing $d$.

Even though we demonstrated the benefits of Cell2Grid using fm-IHC images of size $674\times 506\mu \mathrm{m}$, a WSI of a tissue section can be up to 30 times bigger in each spatial dimension. In our experiments, we did not yet investigate the utility of Cell2Grid when applied to WSI data. Additionally, we only considered image rescaling instead of image compression methods (such as JPEG2000⁸⁰) as reference data processing methods CNN training, because the latter only reduce image storage size and not memory size once decompressed and loaded into memory for model training.⁸

Our study is limited using only a single fm-IHC dataset of colorectal cancer patients that comprises only 54 patients and 1,353 images. These numbers are small compared with the large number of CNN parameters (${10}^7$ and ${10}^4$ for VGG and SIN architectures, respectively). For a more comprehensive analysis of our method, future experiments on different datasets with larger cohorts are required to identify potential weaknesses and demonstrate robustness of our algorithm for different tissue types. Additionally, the utility of our method when applied to other image modalities, such as conventional H&E-stained images, needs to be investigated in future work.

In addition, we want to build on our current work and develop further two areas of interest.