2.1.

System Configuration and Optimization Roadmap

A treatment system consisting of a binocular vision system and an image progress unit was used for three-dimensional (3-D) reconstruction of lesion areas; this reconstructed model will serve as the objective basis to optimize the direction of the irradiation treatment.^24,25 As shown in Fig. 1(a), the proposed model for binocular digital illumination comprised a camera, digital projector, and computing unit. A camera (Nikon D3200, Tokyo, Japan) with $3008\times 2000\text{pixels}$ was used to capture images during the process. A projector (Sony VPL-DX120, KōnanMinato, Tokyo, Japan) with $1024\times 768\text{pixels}$ was applied as a space modulator in 3-D reconstruction and as a light device in the treatment. In the experiment, two head models painted with red color were used to represent the patient with PWS lesions.

Fig. 1

(a) System setup and (b) flow chart of optimization.

The majority of lesions should receive the required irradiance dosage during treatment. The distribution of light irradiance was optimized by dividing the lesions into small patches and mapping them to the corresponding pixels on the projector. This map was used to control the irradiance on the lesion by adjusting the gray value of the video signals to the projector.

The software for optimization included preprocessing of the point cloud of lesions, calculating the irradiance angle, incorporating weight coefficients in the optimization function, and evaluating irradiation [Fig. 1(b)]. The flow path is introduced in detail in the following sections.

2.2.

Parameters for Optimization

After the camera-projector vision system was calibrated, 3-D modeling of the head was conducted through a gray-coded structured light method based on ray-plane triangulation. The pixels in the camera image and the 3-D points share one-to-one mapping. Digital color images of the skin were used to segment the lesions, and 3-D lesion coordinates were indexed from the two-dimensional projector image.^24,25 Although previous studies specified that the point clouds obtained using structured light measurements with MeshLab exhibited high reconstructive accuracy,²⁴ error points may still exist because of incorrect lesion extraction. Processing the point clouds and calculating the characteristic parameters of the lesion surface are important in optimization to accurately characterize the surface features of lesions.

2.2.3.

Triangularization of lesions

The Delaunay method was applied to triangulate the point cloud of lesions and acquire the statistic of the illumination area. The area of each triangle patch was individually calculated and totaled to approximate the area of lesions. When the projector was used as the treatment light source, the corresponding pixel $P_p$ in the projector image that matched the parameters of the binocular system was used to control irradiation.

The difference in the irradiance received by the small triangle patches can be identified based on the irradiation angle. In the method described in Sec. 2.2.2, the normal vector for each point in the lesions can be calculated. The surface normal vector of the triangle patch was also set according to the fitting of the corresponding normal vectors of the triangle’s vertices.

As shown in Fig. 2, the irradiation angle $\theta$ is the angle between the incident ray $O_{p}P$ and the surface normal $\overrightarrow{n}$ of the patch. $O_p-X_p{Y}_p{Z}_p$ represents the coordinates of the projector, and $P_p$ is the control pixel in the projector image for the irradiance of the patch. The cosine of $\theta$ is defined in Eq. (7):

Eq. (7)

$$\cos \theta =\frac{\overrightarrow{P{P}_p}\times \overrightarrow{n}}{|\overrightarrow{P{P}_p}||\overrightarrow{n}|}.$$

Fig. 2

Irradiation angle of a patch.

2.3.

Optimization

An iterative optimization method with weight coefficients is proposed to optimize the irradiation and achieve the maximum lesion area in a single session. The weight coefficients were established according to the distribution area of irradiation. The optimization function was defined in terms of the weight coefficients and normal vectors of the lesions. The location of the light source was updated based on the optimized vector.

2.3.3.

Updating the position of the light source

The projector was located at a specific distance from the patient. Suppose the point set of lesions is

$$\mathrm{\Omega }=\{p_1(x_1,y_1,z_1),p_2(x_2,y_2,z_2),\dots ,p_n(x_n,y_n,z_n)\}.$$

At the beginning of optimization, the projector was translated to enable its optic axis to pass through the initial point for the direction optimization $G(x_0,y_0,z_0)$ with

Eq. (11)

$$\{\begin{array}{c}{x}_0=\frac{x_{\min }+x_{\max }}{2}\\ y_0=\frac{y_{\min }+y_{\max }}{2}\\ z_0=\frac{z_{\min }+z_{\max }}{2}\end{array},$$

where

Eq. (12)

$$\{\begin{array}{l}{x}_{\min }=\min (x_1,x_2,\dots ,x_n),x_{\max }=\max (x_1,x_2,\dots ,x_n)\\ y_{\min }=\min (y_1,y_2,\dots ,y_n),y_{\max }=\max (y_1,y_2,\dots ,y_n)\\ z_{\min }=\min (z_1,z_2,\dots ,z_n),z_{\max }=\min (z_1,z_2,\dots ,z_n)\end{array}.$$

In optimization, the optic axis of the projector passed through the initial point. In iteration, the position of the light source was updated according to the optimization results to irradiate the lesions in a new irradiation direction as shown in Fig. 3. $O_p-X_p{Y}_p{Z}_p$ and $O_p^{\prime }-X_p^{\prime }{Y}_p^{\prime }{Z}_p^{\prime }$ represent the original and updated positions of the projector, respectively. $O(u,v)$ is the optical center of the projector, and $f_p$ is the focal length of the projector that remains constant to avoid recalibration and reduce complexity. The distance between $G$ and the optical axis is $L$, and $\overrightarrow{v}$ is the optimized vector. The irradiation angles are $\theta$ and ${\theta }^{\prime }$ when the projector irradiated the lesions at the original and updated positions, respectively. The updated position $O_p^{\prime }(X,Y,Z)$ can be calculated using Eq. (13):

Eq. (13)

$$\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\left[\begin{array}{c}{x}_0\\ y_0\\ z_0\end{array}\right]+\beta \overrightarrow{v},$$

where $\sqrt{(\beta \overrightarrow{v}){(\beta \overrightarrow{v})}^T}=L+f_p$.

Fig. 3

Three-dimensional diagram of the updated position.

3.1.

Reconstruction of Lesions

PWS lesions were represented by the two head models painted on the face and neck, where PWS frequently occurs. The regions of interest in the head models are shown in Figs. 4(a) and 4(b).The point clouds of lesions were also extracted and shown in Figs. 4(c) and 4(d).

Fig. 4

Reconstruction of lesions in the two experiments: region of interest reconstruction in experiments (a) I and (b) II, and the point cloud of lesions in experiments (c) I and (d) II.

3.2.

Processing Point Clouds

The point clouds processed with filtering and smoothing are shown in Fig. 5. In smoothing, the order of the basis function was 4 and the Gaussian function was used as the weighting function. Comparison of the processed and original data in experiments I [Figs. 4(c) and 5(a)] and II [Figs. 4(d) and 5(b)] revealed that the error points were removed. The smoothing degrees of the model data were also significantly improved.

Fig. 5

Preprocessed point clouds in experiments (a) I and (b) II, and the triangle mesh model of lesions in experiments (c) I and (d) II.

The point clouds were triangulated as discussed in Sec. 2.2 and shown in Figs. 5(c) and 5(d). The numbers of triangles in the two experimental lesions were 69,756 and 44,442, with total lesion areas of 105.7 and $42.5{\mathrm{cm}}^2$, respectively. The area, normal vector, and location of the center of gravity of each triangle were stored and used as basic parameters for optimizing the irradiation direction.

3.3.

Optimizing Irradiation Direction

In each iterative process, the cosine value of the irradiation angle of each triangle irradiated in the previous direction was calculated. This value may be negative because the curvature radii varied in different patches of the complex curved lesion surface. The cosine values were then divided into 11 intervals, and data with non-negative numbers were classified into one interval.

The optimized vector was calculated using Eqs. (9) and (10), and the irradiation angle for each patch irradiated in the optimized direction was determined. The sum of the products of the cosine values and area were calculated to assess irradiation and reveal the optimization result. When $e$ is closer to the area of the lesions, the optimization is more effective and the lesions can receive more irradiance. The values of $e$ during the five iterations in experiments I and II are shown in Fig. 6.

Fig. 6

Evaluation value in each optimization in experiments (a) I and (b) II.

Statistical analysis of $e$ indicated that the optimization result can achieve a steady state after five iterations despite the occurrence of a slight fluctuation. The value of $e_0$ was evaluated at the original irradiation, and the second $e$ value of $e_1$ was calculated by adjusting the optical axis of the projector to pass through the initial point and thus optimize the original irradiation direction. Although this value may be lower than $e_0$ as shown in Fig. 6(b), it can also reach its best value in the following optimization steps. The optimized vectors in the fifth and fourth optimization represented the optimal irradiation direction vectors for experiments I and II, respectively.

3.4.

Contrast Analysis

The irradiance received by a small patch is proportional to the cosine value of the irradiation angle when the optical power is constant. Given this relationship, this study used the area of the lesion patches within the same interval to estimate the distribution of the illumination area. The areas were compared before and after optimization to determine the effect of the proposed optimization method. The directions of the original and optimized irradiation are shown in Fig. 7. The sum of the area of the triangles within the same interval before and after optimization is shown in Fig. 8.

Fig. 7

Irradiation of the original and optimal directions in experiments (a) I and (b) II.

Fig. 8

Illumination area with different cosines before and after optimization in experiments (a) I and (b) II.

As indicated in Figs. 8(a) and 8(b), the majority of the irradiation angle of the lesions decreased and the area with higher irradiance was improved, which revealed that the projector can effectively irradiate lesions in the optimal direction in each treatment session.

4.1.

Optical Parameters of the Human Skin

This study aimed to decrease the irradiation angle of lesions and improve light irradiance in each patch. Optical parameters (e.g., absorption coefficient, scattering coefficient, and photosensitizer absorption) of the human skin were not considered because of the limitations in the experimental conditions and clinical trials. The light absorbed by the photosensitizer was regarded as the effective light dose for PDT. This dose was unequal to the product of the optical density and irradiation time because of the varied complex light propagation in tissues and the optical coefficients of various tissues in different individuals.^33–35 The effective absorbed dose of lesions should be compensated when the individual optical properties of the tissue can be measured to improve the efficacy of irradiation. Thus, the optimization results can be evaluated in succeeding treatment sessions.

4.2.

Light Source Requirement

The required irradiance was $\sim 100\mathrm{mW}/{\mathrm{cm}}^2$ for general PWS treatment. The power requirement of a light source can be estimated using Eq. (14) without considering the optical parameters of the lesion tissue.²⁴

As indicated in Figs. 9(a) and 9(b), the required power of each patch was equal to the standard when the light source irradiated the lesions at the optimal direction in both experiments; as a result, the patches would receive the standard irradiance value. The 3-D irradiance map can be used by doctors to adjust the power of the light source and achieve uniform light distribution on lesions.

Fig. 9

Power required when lesions received the desired irradiance in experiments (a) I and (b) II.

In the two experiments, the projector also irradiated the lesions by using different powers and revealed the illumination area with an irradiance satisfying the required dosage (Fig. 10). At a power of $\sim 8\mathrm{W}$, the ratio of the effective area to the entire lesion for treatment could reach 98% in experiment I. At 5.7 W, the ratio could reach 99% in experiment II. Uniform light distribution can be achieved by modifying the gray value of the projection pixel that corresponds to the small patches on the lesions.

Fig. 10

Area with irradiance that satisfied the dosage requirement in experiments (a) I and (b) II.

4.3.

Severities of Lesions

PWS typically progresses from light pink to dark purple as the patient ages. These colors have been used as clinical indicators of PWS severity. The areas with different colors in the head model can be separately clustered via a superpixel method.²⁴ The severity of lesions in a patch can also be considered as another weight coefficient in the proposed optimization function. Severely affected regions can receive increased irradiance when the irradiation distribution is improved in the optimized direction within a specific power. After the expected light dosimetry of different lesions was established based on severities, and the required power was estimated using Eq. (14), a 3-D power map can guide doctors in adjusting the power of the light source to enable lesions with varying severities to receive optimum irradiance.

4.4.

Factors Influencing Optimization

Control of irradiation direction is a complex optimization process affected by several factors. In the proposed optimization method, holes may exist in triangle mesh models because of defective data on the 3-D scanning point cloud. The defect area in the entire area of lesions was $<1\%$ in both experiments, so the holes minimally affected the optimization. Thus, the hole-fitting process was omitted to simplify the proposed method and reduce calculation requirements.

4.5.

Feasibility Analysis of the Optimization Method

The point cloud processing was written in C++, and the optimization processing was written in MATLAB® on Microsoft Windows 7 32-bit platform. The execution performances of the two experiments are illustrated in Table 1. In terms of iterations and run-time, the proposed method can be applied in a treatment device to guide irradiation. A novel therapy device combined with the proposed digital illumination via a binocular treatment system incorporated with a robot arm could be an object of further research. The intelligent device that can perform individual therapy exhibits a significant potential in both research and clinical applications.

Table 1

Execution performance of the optimization algorithm.