Two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy rely on either deep models or physical models. Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of sufficient physical constraints. In contrast, deep-learning methods have strong problem-solving abilities, but their generalization ability is often questioned because of the unclear physical principles. In addition, conventional deep models are difficult to apply to some specific scenes because of the difficulty in acquiring high-quality training data and their limited capacity to generalize across different scenarios. To combine the advantages of deep models and physical models together, we propose a hybrid framework consisting of three subneural networks (two deep-learning networks and one physics-based network). We first obtain a result with rich semantic information through a light deep-learning neural network and then use it as the initial value of the physical network to make its output comply with physical process constraints. These two results are then used as the input of a fusion deep-learning neural work that utilizes the paired features between the reconstruction results of two different models to further enhance imaging quality. The proposed hybrid framework integrates the advantages of both deep models and physical models and can quickly solve the computational reconstruction inverse problem in programmable illumination computational microscopy and achieve better results. We verified the feasibility and effectiveness of the proposed hybrid framework with theoretical analysis and actual experiments on resolution targets and biological samples.
Due to the lack of pixel level structural matching data, thermal infrared grayscale images are more difficult to color than visible and near-infrared grayscale images. Therefore, this paper proposes a unsupervised learning method based on CycleGAN. On the basis of CycleGAN, a pre trained edge monitor is introduced to calculate the edge feature map before and after image transformation, and the edge similarity loss function is calculated as the basis for optimizing the neural network parameters. The experimental results show that the proposed method effectively reduces the loss of effective edge information during the coloring process and suppresses the generation of abnormal edge information during the coloring process.
In view that the sun is a main stray light source for space-based camera with wide filed as high-intensity light, the influence on the image quality of the camera was studied. Through the design of the camera’s lens hood, the number of stray lights scattering on the surface of the lens is reduced. Then the stray light which can reach the detector is diminished. Compare the image quality of the flying surveillance camera with which before the lens hood designed, the maximum value of PST(θ) outside the critical angle 55° was reduced to 1.1099×10-9. Stray light was significantly inhibited, and the image quality was effectively improved.
KEYWORDS: Polarization, Cameras, Speckle, 3D metrology, 3D modeling, Digital image correlation, Image fusion, Calibration, 3D image processing, Optical testing
The measurement of the external topography and deformation of large industrial equipment is important. Photoelectric measurement technology can quickly obtain three-dimensional (3D) topography information of components and equipment. Among them, stereo digital image correlation (stereo-DIC) is a 3D measurement method which combines binocular stereo vision system and digital image cross-correlation matching technology. This method has many advantages such as non-contact, wide measuring range, high speed and single frame reconstruction. However, this passive optical measurement method is prone to the interference of ambient light and the impact of high reflective surface, resulting in data loss, which limits the application scenarios of this method. This paper proposes a method of 3D reconstruction against ambient light based on stereo polarization digital image correlation (stereo-PDIC). In this method, speckle is generated by a laser beam irradiating a ground glass and is then projected onto the surface of the target. The polarization speckle image in single frame and multiple polarization channels is collected by binocular polarizing cameras. The influence of the reflected stray light and ambient light is eliminated and the contrast of speckle image is improved. Stereo matching of the speckle images with the corresponding angle of polarization captured by the binocular polarizing cameras is carried out, and finally the 3D reconstruction of the target is completed. The feasibility of this method is verified by both simulation and experiment.
For small-aperture aspheric mirror surface measurement, the traditional measurement method phase measuring deflectometry (PMD) requires multi-step phase shifting, and the result is not high in accuracy and complicated in operation. The digital image correlation (DIC) that can be used for transient measurement is mostly used for diffuse reflection. surface. At present, a speckle deflection technique combining the former two methods has appeared, but its optical system is based on the traditional triangulation method, which has problems of angle occlusion and defects. This paper improves the speckle deflection algorithm and optical path system, replacing the traditional triangulation optical path system with Coaxial normal incidence speckle deflectometry system (CNISDS), adding a plate beam-splitter and a telecentric camera. On the one hand, it can avoid the changes in the shape of the captured image caused by perspective, and on the other hand, it can effectively avoid angle occlusion and missing information. Among them, the DIC method is used for image matching, the PMD method is used to calculate the displacement distribution. Then use the slope data to calculate to reconstruct the surface shape. The improved method can measure absolute surface shape and deformation at the same time, and has the advantages of simple measurement steps, fast response speed, and low cost. Finally, the correctness and feasibility are verified by simulation and actual measurement with deformable mirror. Compared with the phase-shifting operation which has complex steps or the low-precision mirror profile measurement results, the improved method is effective and superior, and provides a new idea for measuring small-aperture aspheric mirrors.
With the rapid development of computer vision, archaeology, medicine, reverse engineering and other fields, optical 3D measurement, as one of its crucial technologies, has been utilized more and more widely. In actual measurement, due to the limitation of the measurement range of the measuring equipment and the occlusion of the measured object, it is difficult to obtain the complete shape of the object through single measurement, thus requires multiple measurement from different perspectives and registration of the point cloud data obtained from each perspective together. To realize the registration and stitching of two point clouds with relative low overlap rate, this paper proposes a method based on curvature features and direction vector threshold. In the registration step, the curvature feature of the point cloud data is utilized to achieve accurate matching, and the Kdtree nearest neighbor search method is used to improve the matching points searching speed. In order to further reduce the registration error, the wrong point pairs are eliminated with the direction vector threshold method. The OpenMP multi-threaded parallel calculation method is used in the process of calculating the direction vector to improve the efficiency and speed. Subsequently, the rotation matrix R and the translation vector t between two point clouds are obtained by singular value decomposition method. Finally, the obtained transformation matrix is used to realize the rigid body transformation between the point clouds. Experimental results show that the proposed algorithm can effectively improves the registration accuracy and time efficiency of point cloud data with low initial overlap rate.
Polarized skylight sensor can calculate the heading angle by detecting the polarization patterns of skylight and overcome many inherent defects of the conventional navigation methods. This paper develops a real-time bionic polarized skylight sensor. In order to eliminate the sensor’s hardware errors, an indoor calibration experiment is conducted. We also propose an image processing method to enhance the sensor’s robustness in the urban environment. The comparative experiment shows that both calibration experiment and image processing algorithm can achieve good effects.
Three-dimensional digital image correlation (3D DIC), which combines binocular stereo vision and digital image cross-correlation matching technology, can be used to restore the three-dimensional and deformation information of the object under test. 3D DIC can be accomplished by matching the subset in the left(right) image with that in the right(left). The size of the matching window is found to be critical to the measurement accuracy. Nevertheless, when the subset is small, the measurement accuracy and resolution are high but very sensitive to noise. In contrast, for large subset the measurement accuracy and resolution are lower, while the measurement is more robust to noise. To combine the advantages of high precision and robustness, the Spatio-temporal cross-correlation method is proposed in this paper. A set of speckle patterns are projected onto the objects under test. Instead of the way constructing subsets from spatial neighbor points, the way used in conventional DIC, both spatial and temporal neighbor points are utilized to construct subsets with rich information and strong characteristics. To implement the proposed scheme, we use a mechanical galvanometer to realize the projection of sequential speckle patterns and construct a stereo vision system to realize the three-dimensional reconstruction. The sub-pixel matching algorithm is used to improve the accuracy of stereo matching and 3D reconstruction. Simulations and experiments are carried out to verify the feasibility and success of the proposed scheme and system.
In fringe projection profilometry, phase unwrapping has long been a critical issue. Unwrapping methods can be classified as spatial unwrapping methods and temporal unwrapping methods. The spatial unwrapping method applies only to the surface of continuous objects. The temporal unwrapping method is more widely used and can be used on discontinuous or isolated objects. Nevertheless, the temporal unwrapping methods are suffering from time-consuming, such as the large number of pictures required (standard temporal unwrapping method, phase shift plus Gray code method), and high algorithm complexity (such as periodic encoding method, Fourier transform Method) Etc. Therefore, we propose a temporal unwrapping method using only three projection patterns. In this method, two linear gray scale increasing and decreasing pictures are used to obtain the cores global phase map and uniform illumination background. Another sine fringe image and the above uniformly illuminated background image are used to obtain the wrapped phase. Then the absolute phase can be achieved with the coarse global phase distribution and the wrapped phase. Experimental results prove that this method can measure three-dimensional scenes containing isolated objects.
With the development of machine vision technology, in the process of visual navigation with images, it is necessary to match the local geometric features or global features of the images; however, the matching of local geometric features is low in accuracy and difficult to be used in tracking. In contrast, template-based global feature matching can directly use the information of the entire image, and it has high robustness to illumination variations and occlusions, so it has attracted widespread attention. At present, the classical matching algorithms based on templates mainly include Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), Normalized Cross Correlation (NCC), and Mutual Information (MI). In order to make it more reasonable to evaluate and compare the performance of the algorithms, in this paper, we decided to compare Mean Absolute Differences (MAD), Mean Square Differences (MSD), Zero-mean Normalized Cross Correlation (ZNCC), and Normalized Mutual Information (NMI). During the experiment, the Gaussian noise, illumination variations and occlusion were applied to the current image to simulate complex navigation scenes, and then matched it with the template images. The matching values obtained by the above four matching algorithms in different scenes were collectively called as alignment metric values. The matching effects of the four algorithms were evaluated from the following aspects including the smoothness of the metric value, the number of local extremums and whether the best position was in the correct alignment position. The results showed that the accuracy of MSD was greatly affected by noise and was not suitable for scenes interfered by noise, the number of local extremums of ZNCC changed greatly under the conditions of noise, illumination changes, and occlusion, the alignment metric values became unsmooth. In comparison, the NMI showed good robustness and accuracy in different conditions.
Imaging through scattering layers plays an important role in the field of optical imaging. Because of its characteristics, we can observe some targets that are invisible or unobservable. Now, it is a simple and effective way to process images of scattering layers by autocorrelation. However, due to the memory effect and the limitation of the acquisition environment, imaging through scattering layers still lacks the ability to accurately detect unknown objects. In this paper, we analyzed the influence of memory effects and actual acquisition environment on speckle correlations imaging. By controlling the various variables of the experimental device and the image processing, different experimental images and restoration results of the images are obtained. The memory effects control the optical thickness of the scattering layer, the size of the target, and the distance from the target to the scattering layer. There must be appropriate experimental parameter settings to meet the memory effect requirements. In addition, the selection of the position of the image acquisition device determines the degree of dispersion of the speckle. Image processing is mainly for the filtering of space domain and frequency domain, and for changes in constraints in Hybrid Input-Output algorithms. Finally, comparing the influence of all the parameters on the final restored image, the reasonable acquisition scheme and image processing scheme for different targets and scattering media can be obtained. It has reference and guiding significance for the application of imaging through scattering layers via speckle correlations.
Advanced professional courses (APCs) in the senior year will lay the foundation for further graduate study. Meanwhile, they are summaries and applications of the learnt fundamental professional courses (FPCs). Thus APCs form a connecting link between the preceding and the following studies. For example, Principles and Design of Optoelectronic Instruments (PDOI) is a lecture-based APC aiming at familiarizing students with the operating principles and basic design methods of commonly used optoelectronic instruments. Students will be able to describe the operating procedure of the instruments, distinguish the structure and function of each part, and present preliminary results of both overall design and parameter design. Problem-based approach with the following implementation is a good choice for such APCs. An assignment of system design is announced as the problem at the beginning of the semester. Students are asked to (1) describe the basic working principle, (2) do the overall design and draw the schematic diagram of the system, (3) do the module devision as well as the budget, and (4) finally analyze a critical parameter of the system. Then during the explanation of corresponding chapters, four times of in-class practices are arranged to help the students finish the assignment question-by-question with the help of textbook, internet and the teacher. Compared with straightforward explanation of the chapters and leaving the assignment as a homework at last, the proposed problem-based approach helps improving the motivation and achievement of the students.
Fourier ptychography microscopy (FPM) is a recently developed computational imaging approach which surpasses the resolution barrier of a low numerical aperture (NA) imaging system. It is a powerful tool due to its ability to achieve super resolution of complex sample function, pupil aberration, LED misalignment, and beyond. However, recent studies have focused more on the optimization algorithms and set-ups instead of its theoretical background. Although some imaging laws about FPM have already been set forth, the formulas and laws are not fully defined, and the connection between diffraction theory and Fourier optics has a gap. Therefore, there exist a need for comprehensive research on physical and mathematical basis of FPM for future applications. Keeping this goal in mind, this manuscript utilizes scalar field diffraction theory to rigorously study the relationship between wavelength, the propagation mode, illumination direction of the incident wave, sample structure information and the direction of the output wave. The theoretical analysis of diffraction imaging in FPM provides a clear physical basis for not only the FPM systems, but also for the ptychography iterative engine (PIE) and any other coherent diffraction imaging techniques and systems. It can help to find the source of noise and therefore improve image quality in FPM technique and systems.
Image enhancement technique is utilized to emphasize the overall or local characteristics of pictures and widely used in aerospace, and machine vision application. However, most of these techniques are mathematical algorithms based on captured pictures instead of the imaging process. Fourier ptychographic microscopy (FPM) is a recently developed computational imaging approach which stitches together low-resolution images acquired under different angles of illumination with the same intensity in Fourier space to produce a wide-field, high-resolution complex sample image. In this article, a theoretical model about the illumination intensity is proposed. The effect of uneven illumination intensity can be reduced significantly based on our model. Furthermore, the quality of the reconstructed image can be enhanced by adjusting the intensity of the illumination light corresponding to the high frequency components of the original spectrum.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.