In the past 25 plus years, machine vision has grown from a high priced solution looking for a problem to solve, to a multibillion dollar industry playing a crucial role in todayís high demand production environment. Early machine vision systems consisted of dedicated, special architecture processors, some with hundreds of individual processors, and price tags approaching a hundred thousand dollars or more. There were large array boxes that required high end workstations to communicate with, dedicated units with control like interfaces the size of small refrigerators, and special low level languages understood only by the most dedicated programmer. Today, a full featured vision processor will fit into a standard PC box, often as a plug in card, and the fast, dedicated purpose systems will fit in the palm of your hand, connecting to any PC over an internet connection. The 3D vision system has likewise made great strides, though it remains just a step or two behind its 2D cousin. Early 3D systems were notoriously slow, taking the good part of an hour on temper mental equipment, producing complicated clouds of data with not good way to use the information. Today, high quality 3D data can be obtained from rugged, even portable units with simple to use interfaces, producing inspection information based upon part tolerances and CAD models in a matter of seconds. None of these systems are the mystical iRobots that can seeî envisioned in those early days. But what machine vision is today, and what it is becoming, is a technological tool on its way to becoming as common place as the computer, not only in production environments, but potentially in our every day lives. This paper will look at what 2D and 3D vision can do today, where and how it is being used, and where it may be going in the future.

Illuminators used in machine vision applications typically produce non-uniform illumination onto the targeted surface being observed, causing a variety of problems with machine vision alignment or measurement. In most circumstances the light source is broad spectrum, leading to further problems with image quality when viewed through a CCD camera. Configured with a simple light bulb and a mirrored reflector and/or frosted glass plates, these general illuminators are appropriate for only macro applications. Over the last 5 years newer illuminators have hit the market including circular or rectangular arrays of high intensity light emitting diodes. These diode arrays are used to create monochromatic flood illumination of a surface that is to be inspected. The problem with these illumination techniques is that most of the light does not illuminate the desired areas, but broadly spreads across the surface, or when integrated with diffuser elements, tend to create similar shadowing effects to the broad spectrum light sources. In many cases a user will try to increase the performance of these illuminators by adding several of these assemblies together, increasing the intensity or by moving the illumination source closer or farther from the surface being inspected. In this case these non-uniform techniques can lead to machine vision errors, where the computer machine vision may read false information, such as interpreting non-uniform lighting or shadowing effects as defects. This paper will cover a technique involving the use of holographic / diffractive hybrid optical elements that are integrated into standard and customized light sources used in the machine vision industry. The bulk of the paper will describe the function and fabrication of the holographic/diffractive optics and how they can be tailored to improve illuminator design. Further information will be provided a specific design and examples of it in operation will be disclosed.

This paper describes a 3-D imaging technique developed as an internal research project at Southwest Research Institute. The technique is based on an extension of structured light methods in which a projected pattern of parallel lines is rotated over the surface to be measured. A sequence of images is captured and the surface elevation at any location can then be determined from measurements of the temporal pattern, at any point, without considering any other points on the surface. The paper describes techniques for system calibration and surface measurement based on the method of projected quadric shells. Algorithms were developed for image and signal analysis and computer programs were written to calibrate the system and to calculate 3-D coordinates of points on a measured surface. A prototype of the Dynamic Structured Light (DSL) 3-D imaging system was assembled and typical parts were measured. The design procedure was verified and used to implement several different configurations with different measurement volumes and measurement accuracy. A small-parts measurement accuracy of 32 micrometers (.0012”) RMS was verified by measuring the surface of a precision-machined plane. Large aircraft control surfaces were measured with a prototype setup that provided .02” depth resolution over a 4’ by 8’ field of view. Measurement times are typically less than three minutes for 300,000 points. A patent application has been filed.

A solution for industrial inspection purposes will be shown. The conception is based on digital colour images taken from numerous perspectives around the object to be controlled. The process of image orientation is simplified and improved by the use of spatial reference objects. In contrast to conventional planar targets they are visible from all views and allow a better concatenation of the images. In addition, they can introduce a scale information what simplifies preparatory actions to be done within object space and can contribute with their spatial shape. For the inspection the object will be marked by a laser projection system with point or line patterns. The main aspects of the solution will be shown together with first test results.

The performance of a 3-D inspection process as measured in terms of accuracy and repeatability of point cloud data obtained through the process is of great importance for proper utilization of data. This paper attempts to characterize effect of different operating conditions on system performance and to estimate a set of best operating conditions for the Capture Geometry Inside system CSS-1000. The system consists of a milling machine to cut the part one layer at a time and a vision system made up of a line-scan camera and associated software to capture images of the cut surfaces. These images are stacked to construct a 3-D point cloud data of the part. The mill-head rotation speed, milling feed rate and part orientation are some of the operating conditions studied. Accuracy is measured by comparing measured dimensions of the part with dimensions obtained from the process while spread of point cloud data from the best-fit plane on a surface provides a measure of repeatability. Tests were done on parts made up of two different materials: Aluminum and Delrin®, under two different sets of operating conditions. The results show improvement in accuracy under the new set of operating conditions for aluminum parts while repeatability improves for Delrin parts.

We have developed a new optical sensor that measures the width and the surface profile of a steel plate or strip. The key feature of the new multifunction sensor is that it can measure the width and the height of the object at the same time. To achieve this, we applied laser scanning and a custom-made photo diode (PD) module which consists of dual line PD array with a layout in which 1 array is shifted by half an element pitch. We have constructed test equipment to verify the principle of the new multifunction sensor. It consists of a laser scanning unit and a laser detection unit. The scanning unit has a polygon mirror which scans a collimated laser beam and irradiates it onto the steel plate. A laser detection unit receives the laser beam which is reflected from the surface of the steel plate and images it to the PD module, while the beam spot scans on the plate. In this paper, we show the principles of the new method of sensing width and height, and afterwards we describe the evaluation results of the new multifunction sensor by using the test equipment.

Volume holographic imaging (VHI) utilizes the Bragg selectivity of volume holograms to achieve 3D optical slicing. The depth resolution of VHI degrades quadratically with increasing object distance like most 3D imaging systems. We have devised an imaging scheme that takes advantage of the superior lateral resolution of VHI and a-priori surface information about the object to build a profilometer that can resolve 50 μm features at a working distance of ≈ 50 cm. We discuss the scheme and present experimental results of surface profiles of MEMS devices.

One of the approaches to fabrication of MEMS involves surface micromachining to define dies on single crystal silicon wafers, dicing of the wafers to separate the dies, and electronic packaging of the individual dies. Dicing and packaging of MEMS accounts for a large fraction of the fabrication costs, therefore, nondestructive evaluation at the wafer level, before dicing, can have significant implications on improving production yield and costs. In this paper, advances in development of optoelectronic holography (OEH) techniques for nondestructive, noninvasive, full-field of view evaluation of MEMS at the wafer level are described. With OEH techniques, quantitative measurements of shape and deformation of MEMS, as related to their performance and integrity, are obtained with sub-micrometer spatial resolution and nanometer measuring accuracy. To inspect an entire wafer with OEH methodologies, measurements of overlapping regions of interest (ROI) on a wafer are recorded and adjacent ROIs are stitched together through efficient 3D correlation analysis algorithms. Capabilities of the OEH techniques are illustrated with representative applications, including determination of optimal inspection conditions to minimize inspection time while achieving sufficient levels of accuracy and resolution.

The inspection of plastic film flatness is approached by utilizing the light specular reflection off the film. A grating image formed by the reflection provides the slope information of the film surface. The wedge test shows the specular reflection-based flatness tester can achieve the slope accuracy of 0.01°. The film test shows the capability of the tester on the surface with a variety of slopes resulting in RMS and PV values of the slope. The film appearance in terms of the amount of interference rings is correlated to the RMS value of slope derivative.

Vision-based measurement methods were used to measure bubble sizes in this sonoluminescence experiment. Bubble imaging was accomplished by placing the bubble between a bright light source and a microscope-CCD camera system. A collimated light-emitting diode was operated in a pulsed mode with an adjustable time delay with respect to the piezo-electric transducer drive signal. The light-emitting diode produced a bubble shadowgraph consisting of a multiple exposure made by numerous light pulses imaged onto a charge-couple device camera. Each image was transferred from the camera to a computer-controlled machine vision system via a frame grabber. The frame grabber was equipped with on-board memory to accommodate sequential image buffering while images were transferred to the host processor and analyzed. This configuration allowed the host computer to perform diameter measurements, centroid position measurements and shape estimation in "real-time" as the next image was being acquired. Bubble size measurement accuracy with an uncertainty of 3 microns was achieved using standard lenses and machine vision algorithms. Bubble centroid position accuracy was also within the 3 micron tolerance of the vision system. This uncertainty estimation accounted for the optical spatial resolution, digitization errors and the edge detection algorithm accuracy. The vision algorithms include camera calibration, thresholding, edge detection, edge position determination, distance between two edges computations and centroid position computations.

Hitherto, machine vision systems have been designed as integrated units, in which one or more cameras, a dedicated image processing engine and control/image display console are connected together and hence are all located in close physical proximity to one another. Myriad is a web-based framework that allows these components to be separated from one another and enables complex hierarchies and meshes of inter-communicating cameras and image processing elements to be built. Any component of a given type can be interchanged with any component of the same type, wherever they are located. Myriad networks can employ single- or multiple-stream processing. Several users may be active at a given moment and an unlimited number of users can passively observe what another one is doing, provided that the appropriate permissions have been granted. Myriad networks can encompass intelligent programmed controllers, and/or human controllers, working through a standard web browser. A sub-system can record its observations in a standard database, which can then be interrogated in the normal manner.

A significant limitation of the application of 3D structured light systems has been the large dynamic range of reflectivity of typical parts such as machined parts. The advent of digital cameras have helped this problem to some extent by providing a larger dynamic range of detection, but often parts must still be coated with white paint or powder to get a good enough return for 3D measurement techniques such as structured light. This paper will present an overview of methods that have been used to minimize the range of light reflections from many parts including polarization, multiple exposure, multiple viewing and masking techniques. Also presented will be methods of analysis such as phase analysis techniques which can provide improved robustness. Finally, we will discuss the pros and cons of these options as applied to the application of 3D structured light techniques to machined metal parts.

This paper describes a range-finder that uses two feedback loops. Each feedback loop is respectively used to control the phase and the modulation-amplitude in a sinusoidal phase-modulating interference signal. It enables us to perform unambiguous measurement and real-time processing on the interference signal. Also external disturbance superimposed on the interference signal is eliminated by the feedback control. A distributed-Bragg-reflector laser diode that has wide range in wavelength-tuning allows us to improve the measurement accuracy.

We propose a two-wavelength interferometer that is used to a stroboscopic step height measurement. Different from most two-wavelength interferometers, in present experiment, two slightly different wavelengths are simultaneously oscillated by currently and thermally controlling a laser diode to work at mode hop region. By use of this two-wavelength laser source, a Twyman-Green interferometer, whose reference arm and object arm have known step height r and unknown step height h, respectively, is constructed. Three independent interference patterns corresponding to different OPDs are formed and they can be simultaneously taken by a CCD camera. Furthermore, tilting the reference, spatial frequencies are introduced into the interference patterns. Taking the Fourier transform of these patterns, three fringe amplitudes are obtained and their expressions can be solved for the unknown step height. As we can capture clear image of the interference patterns in a very short time by use of the high speed shutter function of the CCD camera, the error induced by the external disturbance is farthest reduced.

The depth from focus measurement principle relies on the detection of the optimal focusing distance for measuring the depth map of an object and finding its 3D shape. The principle is most effective at microscopic ranges where it is usually found implemented around a z-controlled microscope and sometimes named multifocus 3D microscopy. As such, the method competes with many other 3D measurement methods showing both advantages and disadvantages. Multifocus 3D microscopy is presented and compared to chromatic aberation, confocal microscopy, white light interferometry. Then, this paper discusses two applications of multifocus 3D microscopy for measuring wood respectively metallic parts in the sub-millimeter range. The first application aims at measuring the topography of wood samples for surface quality control. The wood samples surface topography is evaluated with data obtained from both confocal microscopy and multifocus 3D microscopy. The profiles and a standard roughness factor are compared. The second application concerns the measurement of burrs on metallic parts. Possibilities and limits of multifocus 3D microscopy are presented and discussed.

3D image correlation is a robust method for measuring full-field displacements and strains using a calibrated pair of video cameras. Underlying principles and benefits are reviewed, and the method is compared to both 3D ESPI and 2D image correlation. Several applications combining image correlation photogrammetry with stroboscopic illumination and/or high-speed video cameras are presented. Operational strains in ionic polymeric muscle samples and electro-restrictive actuators are determined. The use of short-duration white light pulses to study automobile tires on road wheels at speeds up to 150 miles per hour is demonstrated. Initial work measuring strains on an 18" flywheel in a spin pit at up to 35,000 rpm is described. A notched rubber dogbone sample is pulled to failure at 125% strain in 38 milliseconds, and hundreds of full-field strain maps are captured. This paper includes discussion of sample preparation methods and special lighting systems, including pulsed arc lamps and pulsed lasers. A matrix of capability using available high speed cameras is included.

Sheet metal manufacturers are under constant and increasing pressure to improve and document quality, while reducing cost. Furthermore, OEMs are shifting responsibility for quality inspections to suppliers, adding extra burdens. Exciting, shapely product designs are placing greater demands on both quality assurance and development departments. Deep drawing and other new advanced forming methods push materials to their limits. There is a new stamping quality control tool available for easy, effective and reliable determination of shape, strains and thinning. Full-field optical vision systems, based on the well-known principles of circle grid analysis and photogrammetry, provide automated analysis and quantitative color maps for every square inch of complex parts. Quality results are displayed on a 3D computer model, using the actual measured dimensions of the real part, allowing it to be viewed from any angle. One of the key features of this system is a dynamic link between the forming limit diagram and the strain/thinning color map. When a point is clicked on either display, a second crosshair automatically highlights that same point on the other display, and a detail box presents all measured and calculated quantities. Critical points can be identified at a glance so that corrective action can be taken. Examples shown include before and after die optimization, and analysis of a 1.2 meter long B-pillar stamping.

With the cost of image acquisition and processing hardware decreasing substantially, consumer applications utilizing machine vision are becoming more feasible. Automotive vision systems represent one emerging application area and offer the potential of significant enhancements to automotive safety. However, the relative lack of lower-cost and higher-performance cameras limits the use of vision technology in cars. Camera acquisition speed, sensitivity and dynamic range issues are especially critical due to the totally unconstrained illumination for this type of application. A successful vision system must be highly reliable under direct sunlight and near-total darkness. Conditions of extreme contrast occur primarily during the day when deep shadows are cast across part of a scene being imaged by the camera. This paper provides a survey of existing camera hardware and discusses the limitations of existing hardware. Performance criteria requirements for different automotive applications will also be presented.

The SKIPSM (Separated-Kernel Image Processing using Finite-State Machines) paradigm was originally developed about 1990 as a technique for increasing the speed and versatility of pipelined image-processing hardware. As general-purpose computers became faster (although still much slower than dedicated hardware), it became clear that the most important application of SKIPSM would be for speeding up software image-processing programs running on PCs. This paper therefore concentrates on software implementations written in the C language. Because the SKIPSM paradigm is radically different from conventional image processing algorithms, the paper begins with a general overview of the method. This is followed by some examples selected from the wide range of available SKIPSM operations. Finally, execution-speed comparisons for some of these examples are presented.

A novel 3D sensing system with real-time 3D processing has been developed which is capable of scanning an object at very high speeds (greater than 500,000 3D points/second) and creating high-resolution 3D surface maps. Laser triangulation is used in conjunction with a high-resolution camera, a laser diode, and processing electronics all incorporated into a small sensor package that traverses linearly or rotates from a fixed position to scan an object. Processing is done on board the instrument and the resultant 3D data is transmitted to a PC. This results in rapid scans, with 3D images produced as the instrument is scanning. The sensing system was developed for the NASA Mars rover program and for the inspection of Shuttle Thermal Protection System (tiles), radiator, and structures.

Myriad is a framework for building networked and distributed vision systems and is described in a companion paper in this conference. Myriad allows the components of a multi-camera, multi-user vision system (web-cameras, image processing engines, intelligent device controllers, databases and the user interface terminals) to be interconnected and operated together, even if they are physically separated by many hundreds, or thousands, of kilometres. This is achieved by operating them as Internet services. The principal objective in this article is to illustrate the simplicity of harmonising visual control with an existing system using Myriad. However, packing of 2-dimensional blob-like objects is of considerable commercial importance in some industries and involves robotic handling and/or cutting. The shapes to be packed may be cut from sheet metal, glass, cloth, leather, wood, card, paper, composite board, or flat food materials. In addition, many 3D packing applications can realistically be tackled only by regarding them as multi-layer 2D applications. Using Myriad to perform 2D packing, a set of blob-like input objects ("shapes") can be digitised using a standard camera (e.g. a "webcam"). The resulting digital images are then analysed, using a separate processing engine, perhaps located on a different continent. The packing is planned by another processing system, perhaps on a third continent. Finally, the assembly is performed using a robot, usually but not necessarily, located close to the camera.

In applications like 3D surfacic reconstruction, it can be interesting to use an active stereovision system. We are working on such a system consisting of a colour camera and a LCD projector. The use of colour is a way to differentiate the patterns emitted towards the scene by the projector. By using the LCD projector to project an image with colour patterns onto the scene, the 3D position of the patterns can be reached: one have to match the location of the patterns in the emitted image with their position in the acquired image. This step needs the knowledge of the geometric calibration parameters of the stereovision system. The LCD projector is connected to the computer and the resulting image is projected onto a graduated plane for the object referential. So, a set of 2D cursor positions p_i (in the referential of the emitted image) can be associated to a set of 3D points Pi (in the object referential). Because the plane is motorized, the set of 3D points can cover all the working space while the 2D positions cover all the screen of the emitted image. Then, the half of the couple of points is used to calculate the calibration parameters by using the Faugeras and Toscani method. Finally, the 3D points which belong to the second half of the couples of points are used. They allow to calculate their theoretic corresponding 2D points in the referential of the emitted image, by using the calibration parameters. The comparison with their practical values, obtained during the acquisition step, shows low errors.

Color-encoded digital fringe projection is a recently developed technique for high-speed 3-D shape measurement. In this technique, only a single image is needed for reconstructing the 3-D surface shape of an object. Thus the measurement speed can be increased up to the frame rate of the camera. However, previous experimental results showed that the measurement accuracy of this technique was lower than that of the traditional grayscale technique due largely to the color coupling and imbalance problems. In this research, we propose an improved system with a high brightness LCD projector and a 3-CCD video camera to address these problems. A Look-Up-Table (LUT) method is developed to compensate for errors caused by color coupling and imbalance. In addition, a nonlinearity compensation method is used to further reduce measurement errors. Experimental results showed that the measurement error was significantly reduced.