2.1.

Traceability

Within this technique, traceability is achieved to the standard of length through the measurement of the pixel pitch of the display screen, as shown in Fig. 1.

Fig. 1

Traceability chain for calibration method.

For the purpose of this investigation, the technique was applied to both liquid crystal display (LCD) and electronic paper display (EPD) screens. A ZEISS Axio Imager. M1m microscope with AxioCam HRc was used to verify the screen manufacturer pixel pitch specification and provide traceability.

The microscope was calibrated for measurement in ZEISS AxioVision Rel 4.8 software using a $100\times 0.1=10\mathrm{mm}$ Pyser-SGI Ltd. PS1R stage micrometer graticule, which in turn was within the validity of its calibration, completed in accordance to UKAS requirements to give an expanded measurement uncertainty of $\pm 0.5\mu \mathrm{m}$ at $20.0\pm 1°\mathrm{C}$. This was achieved by performing three repeated measurements using a laser interferometer of each 1 mm graduation on the scale of the graticule from 1 to 10 mm. A single value for combined standard uncertainty was calculated using the root sum square of the standard deviations of each set of repeated measurements. The expanded measurement uncertainty value of $\pm 0.5\mu \mathrm{m}$ at $20.0\pm 1°\mathrm{C}$ was then calculated using a coverage factor, $k=2$ to provide a confidence level of 95%.

The laboratory in which the pixel pitch measurements were performed was temperature controlled at 21°C. It was verified that the effect of thermal expansion of the glass plate of the graticule was insignificant in relation to the expanded measurement uncertainty by calculating that a temperature change of $\pm 1°\mathrm{C}$ would induce a length change in the glass of $\pm 90\mathrm{nm}$ over the length of the graticule (using a coefficient of thermal expansion for glass of $9\times {10}^{-6}$ per °C at 20°C).¹²

Measurements of pixel clusters were taken at 12 regularly spaced intervals around the LCD and EPD screens (each cluster defined as one group of red, green, and blue subpixels for the LCD screen and one black pixel for the e-ink screen). The LCD screen was removed from the casing and deconstructed until the color filter layer was revealed. Measurements were completed on this layer because it is assumed to most accurately represent the fixed dimension of each subpixel used to display an image.

The EPD screen was not deconstructed because it does not have the same layer construction as LCD screens. Therefore, pixel pitch measurements were carried out on the fully assembled screen, concentrating on the ink capsule layer, Fig. 2.

Fig. 2

Schematic view of one capsule from within the ink capsule layer of an E-ink screen.

The repeatability of the operator’s measurements was assessed by measuring each set of pixels using $10\times$, $20\times$, and $50\times$ objective lenses, respectively. This range of lenses was selected because it represents both the smallest and largest magnifications where defined pixel edges are visible and the distance between at least two pixel clusters can be measured. For each image, a horizontal ($X$-axis) and vertical ($Y$-axis) sequence of pixels was counted and measured using the AxioVision Release 4.7 software package. Each measurement was then divided by the number of pixels counted to give the average pixel pitch for each magnification. Final pixel pitch values were then obtained by averaging the pixel pitch in the $X$ and $Y$ directions across all magnifications, Table 1.

Table 1

Final pixel pitch values.

An example image of pixels as viewed under the microscope is shown in Fig. 3.

Fig. 3

Close-up of the pixels of the different display types.

2.2.

Pixel Pitch Measurement Uncertainty

The overall accuracy of the pixel pitch measurements is a combination of both the calibrated hardware/software and operator accuracy.

The hardware and software accuracy is $0.635\mu \mathrm{m}$. The value is taken directly from the calibration certificate. This is a summation of the calibration’s variation of measurement and uncertainty of measurement, averaged over both graticules used.

Operator accuracy was determined from the difference in measurements of the same pixels at different magnification. The uncertainty in this process stems from the variability in manually selecting the same point on each pixel in the image due to the lack of clearly defined edges of the screen pixels. Operator accuracy was determined by averaging the maximum variation in pixel pitch of the same pixel cluster for each magnification case. This assumes that the same pixel cluster is being measured for each of the magnifications. The average was then taken and multiplied by the microscope’s camera image pixel dimension at each given magnification and divided by the average number of pixel clusters measured at each magnification to give an overall user accuracy presented in Table 2.

Table 2

Microscope measurement uncertainty—user.

The combined overall uncertainty is quantified in Table 3.

Table 3

Microscope measurement uncertainty—overall.

2.3.

Test Set-Up

Tests were performed in a temperature-controlled laboratory at 21°C. DIC data capture used a pair of Allied Vision Technologies Dolphin F201-B 2 megapixel, $1/{1.8}^{\prime \prime }$ sensor cameras with 12-mm focal length Schneider C-mount lenses at an $f/11$ aperture. This combination of lens and sensor demonstrates reasonable optical distortions (see $\kappa$-values in Table 4) and the aperture was chosen to provide a large depth of field for the experiment, while minimizing the effects of diffraction. For each test, the DIC system was set up in a conventional manner, with stand-off distance approximately twice the camera separation distance to give a camera angle of $\sim 30\mathrm{deg}$, which was accurately confirmed to 28.645 deg as parameter $\beta$ from Table 4. The field of view was optimized to match the screen dimensions as closely as possible to utilize as many camera image pixels as possible.

Table 4

Typical two- and three-dimensional camera calibration parameters for a calibration score of 0.021 px presented for repeatability. Camera model and parameter naming follows standard convention.13

A system calibration was performed, using Correlated Solutions’ image acquisition software Vic-Snap 8 and postprocessing software Vic-3D 7, for application to all data analysis. The system calibration builds a model of the physical system in the software using intrinsic characteristics such as the focal length and distortion of the cameras, extrinsic characteristics such as the distance and angles between the cameras, and the geometry of a calibration board that consists of a number of control points with a fixed pitch. For these tests, a calibration board with 9.02 mm spacing between control points was used. Thirty image pairs of the calibration board at different angles and orientations were taken within a 200-mm deep control volume of the stand-off distance with the cameras at an exposure time of 41.5 ms. These images were then imported into the DIC postprocessing software and analyzed with respect to the 9.02 mm target used. This method achieved a calibration score of $<0.022$ px for both LCD and EPD screen experiments. This score corresponds to the average distance in camera image pixels between a theoretical point predicted by the model and its actual position in the image. A detailed overview of typical camera system calibration parameters is given in Table 4.

An ideal reference speckle image was sourced from an image database supplied by the Metrology Workgroup of the French CNRS research network 2519 “Mesures de Champs et Identification en Mecanique des Solides/Full-Field Measurements and Identification in Solid Mechanics.”¹⁴ The scale and resolution of this image were then cropped using Photoshop CS5 to provide an “optimal” 3-5 camera image pixels per speckle for the resolution of each display screen. After this, the resolution of the speckle image was fixed to match that of the display screen. This speckle pitch has been shown to give optimal results in terms of spatial resolution and measurement accuracy.¹⁵ The displacement images were then created by applying integer number of screen pixel shifts to this image in the $X$ and $Y$ directions.

Prior to photographing the digitally translated speckle patterns with the stereo DIC system, the success of the numerical screen pixel translations between images was verified by subtracting the relevant columns/rows of the translated images from the reference speckle image. The images were converted into 0–255 gray level matrices and subtracted from each other to verify that the subtraction result was a null field all zero entries which was the case for all translations.

Three shift cases were generated for the $X$- and $Y$-axes, each with 1, 5, and 10 screen pixel step sizes, as shown in Table 5.

Table 5

Shift cases in screen pixels.

These cases were chosen to be representative of the various test envelopes performed during structural testing of aerospace components, portraying maximum displacements of $\sim 2$ mm, 10 mm, and 20 mm, for each case, respectively.

For each shift case, a sequence of 21 images was generated. Each starts with the reference image which is then shifted by the step size 20 times to the maximum shift value. Sample shifted speckle images of 0 px, 5 px, and 10 px image shifts can be seen in Fig. 4.

Fig. 4

Sample shifted speckle images.

Tests were also conducted to investigate the effect, if any, of image persistence or retention, an occurrence on LCD and EPD screens wherein a remnant of the previous image may be present in the following image. All shift cases were recaptured with a fully saturated white image presented in between each shift image to erase any trace of the previously presented image. This was found to have a statistically insignificant effect on the LCD screen results but significant effect on the EPD screen. Therefore, all shift cases on EPD screens were captured using a fully saturated white image between each shift image.

Furthermore, an investigation into the use of the backlight built into LCD and EPD screens during the experiment was also conducted. All cases of shifts were recaptured with full screen backlight brightness and minimum possible backlight brightness for each screen. These cases were then postprocessed and the overall error of the measurements compared. From the data, the use of backlight was found to have an insignificant effect on the LCD screen results but significant effects on EPD screen results with up to 0.07 screen pixels greater maximum measurement error observed in results using no backlight versus results using full backlight brightness.

During all tests, the screen was supported so that it remained physically stationary and perpendicular to the DIC system throughout the data capturing process. For each of the three $X$ and three $Y$ shift cases, the shift images were displayed in order from the zero pixel shift reference image to the maximum shift, a total of 21 images. The shift images were presented using a slideshow format to ensure there was no movement of the screen by the operator between shifts. DIC data capture software was used to capture each shift image three times giving a total of 63 images and each shift case was captured three times over three days to ensure repeatability. The time each screen was on before capture was found not to influence the measurements, which is expected since temperature of the screens remain close to ambient. A temperature fluctuation of $<1°\mathrm{C}$ was measured for the EPD during extended operation, both with and without backlight. Temperature change effects are thus considered negligible. It was also ensured that ambient lighting did not change significantly during the capture process.

2.4.

Postprocessing

The images were initially postprocessed using an area of interest and coordinate system (set using three point selection) chosen as shown in Fig. 5. The origin and $x$- and $y$-axis points of the coordinate system translation were selected at the same pixel cluster in each analysis. It was calculated that a small misalignment between the coordinate system assigned in the postprocessing software and that of the display screen/shift images had a negligible effect on the resultant data. This can be illustrated as follows: assume that during selection, the second point establishing the $x$-axis deviates by 10 camera image pixels from the actual $x$-axis of the screen. This is a very conservative estimation, since a typical deviation in selecting points on a sharp image is 1 to 2 pixels. For the EPD screen, the shorter of the two targets, with $\sim 1000$ camera image pixels between the two picking points, this results in a misalignment angle $\alpha ={\tan }^{-1}\frac{10\mathrm{px}}{1000\mathrm{px}}\approx 0.01$. From straightforward trigonometry, the difference in the measured and actual displacement would be $1-\cos \alpha \approx 0.005\%$. This error is very much negligible when compared with the $\approx 0.1\%$ to 1% uncertainty of pixel pitch measurement.

Fig. 5

Vic 3-D area of interest—LCD screen.

This was additionally investigated by repeating the analysis of a set of shift images and changing the particular pixel chosen for the origin, $x$- and $y$-axis points by a few camera image pixels in each direction in each analysis. Furthermore, the default best plane fit coordinate system was also evaluated using the same comparison method. By comparing the produced error and standard deviation results in both investigations, it could be observed that the small changes in the selected coordinate system had an insignificant consequence on the values. As a result, all subsequent measurements were postprocessed using the default best plane fit coordinate system.

The subset size, which determines the size of the data facet that is tracked across time, was set at $25\times 25$ camera image pixels so as to be sufficiently large to contain a distinct speckle pattern while still computing results in a reasonable time. Equally, the step size, which determines how many data facets are tracked, was set at five camera image pixels as a compromise between providing an acceptable amount of data points and calculation time, yielding $\sim \mathrm{55,000}$ data points on each image of each translation case. In addition, a seed point was added in the analysis area, at approximately the same point on the speckle for each analysis, in order to ensure the correlation ran smoothly with the larger pixel shifts. Although not required for successful correlation, the seed point was also added to the analysis of the smaller shifts in order to keep consistency between tests.

Data were exported in the form of a mean and standard deviation of displacement in only $X$ or $Y$ depending on which parameter was shifted for each image. These are mean and standard deviation values over the whole area of interest of each correlation. Exported data were then averaged across all repeated tests to achieve a single value for each $X$ and $Y$ shift. These values were then converted from meters (from camera calibration) to screen pixels by dividing them by the appropriate $X$ or $Y$ pixel pitch value from Table 1. This method was also used to achieve a single standard deviation value, in screen pixels, for each shift. Error, normalized error, and standard deviation of measurement for each shift case were then graphically displayed to assess the overall error of the system.

Error and normalized error were calculated according to Eqs. (1) and (2), respectively. Normalization has the benefit of presenting the results in a manner that clearly portrays the relationship between the magnitude of the error and the magnitude of the measurement.

Fig. 6

Sample sigma—LCD screen.

3.1.

Difference Between Screen Axes

Figure 7 shows the error for large displacements of the DIC measurement at each imposed pixel shift for the two different screens. Pixel shifts are imposed along the long ($X$) and short ($Y$) axes of the screen (see also Fig. 5). The error is calculated according to Eq. (1) and as discussed in Sec. 2.4. Several important observations can be made based on the figure.

Fig. 7

The error in pixels is linear with the imposed pixel shift, which suggests constant normalized error in pixels. The two axes of the LCD screen behave similarly. The short axis on the E-Ink (EPD) screen shows a 1-px step in the error at 40, 120, and 200 px.

First, a linear relationship between the imposed shift and the error is evident for both the EPD and LCD screens. This is consistent with a constant normalized error (error per pixel) as discussed further in Sec. 3.3.

The error along $X$ and $Y$ for the LCD screen is approximately equal. This suggests little difference either between the properties of the screen along the axes or the performance of the DIC setup and algorithm. However, the error along the $y$-axis of the EPD screen is much larger than along the $x$-axis, due to steps one screen pixel at pixel shifts of 40, 120, and 200. On further inspection, including repeating the test with a rotated screen to ensure no optical effect outside of the screen is responsible, it is concluded that this error is a result of the screen construction, which is slightly different between the $X$ and $Y$ directions.

The well-defined behavior of the bias (1 px for every 80 px, starting at 40) suggests that a more involved methodology (e.g., not shifting the image from the edge, correcting for the known bias, etc.) for displacing the image on the screen might resolve the issue. However, in its current form, the method still allows for unbiased validation of displacements up to 40 screen pixels, or $\sim 4\mathrm{mm}$.

Since the LCD screen behaves similarly in both principal directions, all further results displayed are for displacement along the long ($X$) axis of the screen unless explicitly specified.

3.2.

Standard Deviation

A clear sinusoidal dependence of the standard deviation of measurement on the imposed shift can be observed in Fig. 8 for all three different steps. A similar sinusoidal pattern of one camera image pixel period is observed in standard deviation of error in DIC due to the periodicity of the properties of the camera sensors used in the image discretization process.¹⁶ (While standard deviation of error and standard deviation of measurement are distinct statistical properties, they behave similarly if the sample size is sufficiently large.) However, the sinusoidal patterns observed on the figure have different periods for the different step sizes. This is explained with the very close size of camera pixel and imposed displacements. The aliasing effect observed is similar to sampling a wave at sampling frequency lower than the Nyquist frequency of the wave. Depending on the sampling frequency, the wave appears to have different (lower) frequencies. On Fig. 8, the step size is equivalent to the sampling rate.

Fig. 8

The standard deviation of the measurement shows a sinusoidal dependency, with a frequency dependent on the sampling step. The sinusoidal behavior dominates at small displacement, while an additional linear dependence is apparent at larger shifts.

A linear dependence appears to be added to the sinusoidal behavior. Its distribution among different sources might represent interest for further investigation, but is beyond the scope of the current discussion. Nevertheless, the fact that standard deviation of measurement exhibits discretization error behavior shows that measurements taken from a screen do not introduce large unexpected random error which is hidden in the rest of the results by the averaging process.

3.3.

Difference Between Screens

Figure 9 shows the main results of the evaluation of the method. The normalized error is calculated as detailed in Eq. (2). Two different length scales are shown to illustrate the short-range and long-range behaviors of the error.

Fig. 9

For large displacements, the normalized error is approximately constant, with E-Ink (EPD) screen having an order of magnitude smaller error than the LCD. At very small displacements, LCD screens provide smaller error, but EPD error decreases very quickly with increasing displacement.

The plot on the right-hand side of Fig. 9 shows the behavior of the normalized error (error per pixel, in percent) at large displacement scales. The constant nature of the phenomenon is expected, since it is consistent with error in the pixel pitch measurement, which has relatively large uncertainty.

The size of the normalized error, on average about 0.5% for the LCD and 0.05% for the EPD, is well within the minimum necessary to perform a useful validation, which suggests that both screens can be used in practice. However, since similar a setup was used to evaluate both types of screens, it can be reasoned that the EPD will add much smaller traceable uncertainty to the measurement, despite the larger screen pixel size.

The small-scale displacement plot on the left-hand side of Fig. 9 shows the normalized error at small scales, up to a pixel shift equivalent to $\sim 2\mathrm{mm}$ of displacement. The EPD normalized error starts higher than the LCD error. The reasons behind this can be numerous, but should in general be attributed to the type of screen, as the LCD uses a similar setup but has much smaller normalized error. Nevertheless, the error of the EPD falls quickly with the imposed shift and becomes comparable with LCD for displacements larger than $500\mu \mathrm{m}$.

Figure 9 shows that all sources contributing to the error of the DIC measurement in the current study add up to $\sim 0.1\%$ to 1%, depending on the configuration. Since part of the error is setup dependent, the error introduced by using a screen with relatively large pixel pitch uncertainty is at most within the same range.