This PDF file contains the front matter associated with SPIE Proceedings Volume 10644, including the Title Page, Copyright information, Table of Contents, and Conference Committee listing.

Results using a neural network classifier for LWIR HSI target detection are presented. Detection performance using two different single layer networks with 10 and 20 nodes in the hidden layer are presented. Additionally, a two-layer network with 20 nodes in the first layer and 10 in the second is also tested. The larger networks generally performbetter, as expected, but this is not necessarily true for all targets. Neural networks significantly outperformACE across several detection metric for eight different targets. Even when ACE is applied to atmospherically-corrected emissivity datacubes, neural networks applied without atmospheric compensation still performbetter. Finally, it is found that incorporating atmospheric metadata into the network inputs can improve detection performance but is again target-dependent.

Hyperspectral imagers are often used in an airborne platform and have shown utility in material detection and identification. In these scenarios the sensor is flown over a scene and the motion of the aircraft and/or the scanning action of a gimbal causes motion blur in the collected imagery. The only solution to this problem is to use expensive pointing gimbals to stabilize the imager and allow for step scanning. The effects of motion blur are dependent on scan speed and integration time of the imager. A Headwall visible hyperspectral imager collected data of a knife edge target to experimentally show the effect of motion blur on spatial resolution in the along scan direction. A data cube was then used to simulate single-pixel and sub-pixel targets. A simulated step scanned scene and continuously scanned scene were then compared using target detection results. The continuous scanned scene showed a twenty five percent drop in performance over the step scanned scene.

We describe and compare two approaches to solid subpixel target detection in hyperspectral imagery. The first approach requires explicit models for both the target and the background, and employs a generalized likelihood ratio in order to obtain a detector that is optimized to those specific models. When this approach is most successful, a closed-form solution is obtained that permits the detector to be efficiently applied. A specific example of this approach is outlined in some detail, leading to the elliptically-contoured finite-target matched filter (EC-FTMF), a variant of the classical FTMF algorithm that uses a multivariate t-distribution instead of a Gaussian as the model for the background. The second approach also requires an explicit model of the target, but does not need a model for the background. In this second approach, matched pairs of data samples are created: for each pixel in the original hyperspectral image, a corresponding pixel is generated by implanting the target into the original pixel. These matched pairs are used as training data for a machine learning algorithm to classify pixels as either non-target or target. Here we use a support vector machine, but the matched pair machine learning (MPML) framework does not restrict the choice of classifier type. Detectors using both approaches are applied both to simulated data (with Gaussian and with multivariate fit distributed backgrounds) and to real hyperspectral data with known, referenced targets.

We describe the development and implementation of plume detection algorithms under severe bandwidth and processing constraints imposed by a CubeSat architecture. In particular, two ideas will be presented: one employs onboard processing to reduce the data that is downlinked, and one employs the Sparse Matrix Transform (SMT) to speed up the onboard computation of an approximate Mahalanobis distance.

We revisit the replacement target problem in the spectro-optic detection of opaque sub-pixel materials, but with a more general distribution model. For pure target or background pixels we assume independent elliptically contoured distributions with covariance matrices related by a scale factor. An unknown sub-pixel mixing ratio defines a composite hypothesis testing problem, whose generalized likelihood ratio test cannot be expressed in closed form for the usual distribution models. Instead we derive closed-form detectors based on a clairvoyant fusion technique that effects a renormalization of the target probability density functions. These robust algorithms prove in some cases to be nearly indistinguishable from the intractable generalized likelihood ratio test solutions.

Real-time, standoff detection of trace chemicals on surfaces in the presence of unknown interferent contaminants using active IR spectroscopy poses significant challenges. The measurement time and computational burden can be prohibitive due to the number of spatial pixels, hundreds of potential wavenumbers, and size of the chemical library containing thousands of signatures. Therefore it is advantageous to optimally sample a small subset of the possible wavenumbers, where optimality is meant in the sense of detection and classification performance. Our approach accomplishes this by selecting wavenumbers which maximize the information about chemical identity. This is done by using submodular optimization, a technique which guarantees near-optimality at vanishingly low computational burden. Therefore, the methods shown here lend themselves to the time and resource constrained problems of data acquisition. This is in contrast to more traditional dimensionality reduction approaches such as lowering spectral resolution, random sparse sampling, and principal component analysis which degrade detection performance. In this work we describe methods for optimal illumination wavenumber selection to address the time constraints while addressing the challenges imposed by hardware and environmental artifacts (e.g., atmospheric effects).

Land-use classification from multiple data sources is an important problem in remote sensing. Data fusion algorithms like Semi-Supervised Manifold Alignment (SSMA) and Manifold Alignment with Schroedinger Eigenmaps (SEMA) use spectral and/or spatial features from multispectral, multimodal imagery to project each data source into a common latent space in which classification can be performed. However, in order for these algorithms to be well-posed, they require an expert user to either directly identify pairwise dissimilarities in the data or to identify class labels for a subset of points from which pairwise dissimilarities can be derived. In this paper, we propose a related data fusion technique, which we refer to as Semi-Supervised Normalized Embeddings (SSNE). SSNE is defined by modifying the SSMA/SEMA objective functions to incorporate an extra normalization term that enables a latent space to be well-defined even when no pairwise-dissimilarities are provided. Using publicly available data from the 2017 IEEE GRSS Data Fusion Contest, we show that SSNE enables similar land-use classification performance to SSMA/SEMA in scenarios where pairwise dissimilarities are available, but that unlike SSMA/SEMA, it also enables land-use classification in other scenarios.

Hyperspectral images often contain hundreds of spectral bands. Man-made and natural materials usually exhibit variability in their reflective and emissive response across these bands, which is exploitable via target detection algorithms. The high-dimensional nature of hyperspectral data has driven studies that explored ways to reduce spectral dimensionality without adversely affecting spectral target detection. Recently, spatial-spectral feature extraction techniques have demonstrated additional discrimination capability versus spectral-only approaches in VNIR, SWIR, and LWIR hyperspectral imagery. When spatial descriptors are applied to spectral bands within a hyperspectral image, the length of a resulting spatial-spectral feature vector can be several times that of the original spectrum. While numerous efforts to reduce the dimensionality of hyperspectral imagery have been undertaken, they have not been commonly extended to the spatial-spectral domain. In this work, we address the relatively new problem of spatial-spectral dimensionality reduction through a strategy designed to remove features that neither negatively affect a target detection algorithm's capability to detect targets nor detract from that algorithm's ability to discriminate between targets in an exemplar signature library.

This paper explores the use of SVD-based column subset selection (SVDSS) algorithms for endmember extraction in hyperspectral unmixing. We are using a version of the column subset selection problem that selects the subset of columns that better predicts the remaining columns in the matrix. It has been shown that this problem is related to the problem of selecting a maximum volume sub-matrix of a matrix. This is related to the problem of finding the spectral signatures from the image that provide the simplex of maximum volume that can be inscribed within the hyperspectral data set, which is the base of some endmember extraction algorithms like N-FINDR. In this paper, we present a numerical comparison between SVDSS and several endmember extraction algorithms such as N-FINDR, PPI and VCA. Simulated and real hyperspectral imagery from the HYDICE Urban Scene are used in numerical experiments to determine which algorithm extracts the endmembers with the highest simplex volume. Results show that SVDSS outperforms other approaches for endmember extraction.

In the 2015 NASA ROSES solicitation, NASA has expressed strong interest in improving the accuracy of Mars surface characterization using satellite images. Thermal Emission Imaging System (THEMIS), an imager with a spatial resolution of 100 meters, has 10 infrared bands between 6 and 15 micrometers. Thermal Emission Spectrometer (TES), an imager with a spatial resolution of 3 km, has 143 bands between 5 and 50 micrometers. While both imagers have a variety of applications, it would be ideal to generate high-spatial and high-spectral resolution data products by fusing their respective outputs. We present a novel approach to fusing THEMIS and TES satellite images, aiming to improve orbital characterization of Mars’ surface. First, the THEMIS bands must undergo atmospheric compensation (AC) due to the presence of dust, ice, carbon dioxide, etc. A systematic AC procedure using elevation information and spectrally uniform pixels has been developed and implemented. Second, a set of proven pan-sharpening algorithms has been applied to fuse the two sets of images. The pan-sharpened images have the spatial resolution of THEMIS images and the spectral resolution of TES images. The results of extensive experiments using THEMIS and TES data collected near the Syrtis Major region (one of the final 3 candidate landing sites for the Mars 2020 rover) clearly demonstrate the feasibility of the proposed approach.

We focus on the problem of spatial feature correspondence between images generated by sensors operating in different regions of the spectrum, in particular the Visible (Vis: 0.4-0.7 μm) and Shortwave Infrared (SWIR: 1.0-2.5 μm). Under the assumption that only one of the available datasets is geospatial ortho-rectified (e.g., Vis), this spatial correspondence can play a major role in enabling a machine to automatically register SWIR and Vis images, representing the same swath, as the first step toward achieving a full geospatial ortho-rectification of, in this case, the SWIR dataset. Assuming further that the Vis images are associated with a Lidar derived Digital Elevation Model (DEM), corresponding local spatial features between SWIR and Vis images can also lead to the association of all of the additional data available in these sets, to include SWIR hyperspectral and elevation data. Such a data association may also be interpreted as data fusion from these two sensing modalities: hyperspectral and Lidar. We show that, using the Scale Invariant Feature Transformation (SIFT) and Optimal Randomized RANdom Sample Consensus (RANSAC) algorithm, a software method can successfully find spatial correspondence between SWIR and Vis images for a complete pixel by pixel alignment. Our method is validated through an experiment using a large SWIR hyperspectral data cube, representing a portion of Los Angeles, California, and a DEM with associated Vis images covering a significantly wider area of Los Angeles.

Multi-modal data fusion for situational awareness is of interest because fusion of data can provide more information than the individual modalities alone. However, many questions remain, including what data is beneficial, what algorithms work the best or are fastest, and where in the processing pipeline should data be fused? In this paper, we explore some of these questions through a processing pipeline designed for multi-modal data fusion in an autonomous UAV landing scenario. In this paper, we assess landing zone identification methods using two data modalities: hyperspectral imagery and LIDAR point clouds. Using hyperspectral image and LIDAR data from two datasets of Maui and a university campus, we assess the accuracies of different landing zone identification methods, compare rule-based and machine learning based classifications, and show that depending on the dataset, fusion does not always increase performance. However, we show that machine learning methods can be used to ascertain the usefulness of individual modalities and their resulting attributes when used to perform classification.

We investigate an anomaly detection framework that uses manifold-based distances within the existing skeleton kernel principle component analysis (SkPCA) manifold-learning technique. SkPCA constructs a manifold from the an adjacency matrix built using a sparse subsample of the data and a similarity measure. In anomaly detection the relative abundance of the anomalous class is rare by definition and in practice anomalous samples are unlikely to be randomly selected for inclusion in the sparse data subsample. Thus, anomalies should not be well modeled by the SkPCA-constructed model. Here, we consider alternative distance measures based on viewing spectral pixels as points in projective space, that is, each pixel is a 1-dimensional line through the origin. Chordal and geodesic distances are computed between hyperspectral pixels and detection performance leveraging these distances is compared to alternative anomaly detection algorithms. In addition, we introduce Ensemble SkPCA which utilizes the ensemble of mean, normalized detection scores corresponding to multiple randomly generated skeletons. For acceptable false alarm tolerances, the ensemble detection score derived from chordaland geodesic-based methods achieves higher probability of detection than Euclidean distance-based Ensemble SkPCA or the benchmark RX algorithm.

The ability to search for radiation sources is of interest to the Homeland Security community. The hope is to find any radiation sources which may pose a reasonable chance for harm in a terrorist act. The best chance of success for search operations generally comes with fielding as many detection systems as possible. In doing this, the hoped for encounter with the threat source will inevitably be buried in an even larger number of encounters with non-threatening radiation sources commonly used for many medical and industrial use. The problem then becomes effectively filtering the non-threatening sources, and presenting the human-in-the-loop with a modest list of potential threats. Our approach is to field a collection of detection systems which utilize soft-sensing algorithms for the purpose of discriminating potential threat and non-threat objects, based on a variety of machine learning techniques.

We explore the representation capabilities of scattering transforms for the classification of hyperspectral images. We examine several types, including a recently developed technique called the Fourier scattering transform. This method is naturally suited for the representation of hyperspectral data because it decomposes signals into multi-frequency bands and removes small perturbations such as noise. We test on four standard hyperspectral datasets, and the results indicate that the Fourier scattering transform is effective at representing spectral data. We also present a spatial-spectral scattering transform that combines Wavelet and Fourier representations, and this method obtains significantly higher classification accuracies.

We propose a novel unsupervised learning algorithm that makes use of image fusion to efficiently cluster remote sensing data. Exploiting nonlinear structures in multimodal data, we devise a clustering algorithm based on a random walk in a fused feature space. Constructing the random walk on the fused space enforces that pixels are considered close only if they are close in both sensing modalities. The structure learned by this random walk is combined with density estimation to label all pixels. Spatial information may also be used to regularize the resulting clusterings. We compare the proposed method with several spectral methods for image fusion on both synthetic and real data.

A deep learning convolutional network of fiber filters is investigated for the spectral analysis of hyperspectral imagery for purposes of material classification and identification. Analogous to convolutional neural networks that apply spatial filters to color imagery for purposes of spatial object classification, a network of convolutional filters is applied spectrally to the large numbers of bands in hyperspectral imagery for purposes of material classification and identification. A convolutional filter has a volume NxNxM, where N is the spatial pixel size (length and width) of the filter and M is the filter depth. For spatial convolutional networks, the filter applied to the first (bottom/input) layer is often N = 5 and M = 3 (e.g., corresponding to each layer of RGB color imagery). For the proposed network, the convolution filters in each layer have a volume with the dimension of N = 1 and M > 5, which we refer to as fiber filters. We investigate the ability of this kind of architecture to learn discriminating features for purposes of material classification and identification as compared to a fully connected neural network. The choice of appropriate architecture depth is investigated, which is the number of layers in the network, not to be confused with the filter depth. Various values the filter depth, M, are also investigated. Aerial collections of hyperspectral imagery are used for training and the validation experiment.

Nonlinear graph-based dimensionality reduction algorithms have been shown to be very effective at yielding low-dimensional representations of hyperspectral image data. However, the steps of graph construction and eigenvector computation often suffer from prohibitive computational and memory requirements. In the paper, we develop a semi-supervised deep auto-encoder network (SSDAN) that is capable of generating mappings that approximate the embeddings computed by the nonlinear DR methods. The SSDAN is trained with only a small subset of the original data and enables an expert user to provide constraints that can bias data points from the same class towards being mapped closely together. Once the SSDAN is trained on a small subset of the data, it can be used to map the rest of the data to the lower dimensional space, without requiring complicated out-of-sample extension procedures that are often necessary in nonlinear DR methods. Experiments on publicly available hyperspectral imagery (Indian Pines and Salinas) demonstrate that SSDANs compute low-dimensional embeddings that yield good results when input to pixel-wise classification algorithms.

Hyperspectral imaging systems are typically characterized in a laboratory environment to determine band centers and bandwidths associated with the collected data. This procedure is commonly referred to as spectral calibration. Generally, this wavelength information is assumed to be accurate and spatially-invariant. Exploitation of hyperspectral data utilizes this information for atmospheric compensation and/or resampling of library data for use in detection and identification applications. The spectral information can be inaccurate due to system aberrations, such as spectral smile, or due to spectral calibration error. This work examines the impact of spectral smile on hyperspectral exploitation.

Recently a novel Fresnel zone light field spectral imager was developed that provides snapshot spectral imaging using no moving parts or scanning. This system combines a Fresnel zone plate as the primary optic to conduct both imaging and dispersion with a microlens array configured like a plenoptic camera. This encodes spectral information onto the detector array like a traditional plenoptic camera encodes range information, and both systems require post-processing to produce final images. While algorithms will significantly affect final performance, the ability to judge the optical performance of a particular hardware design before post processing is important to set a base line for algorithm comparisons as well as establish how components in the imaging chain impact performance. For remote sensing scenarios we propose a figure of merit based on shifts of the images formed by each microlens of a point source and derive design equations linking this to system parameters. This talk examines how the Fresnel zone plate and microlenses affect the blur of the point source images and how the sampling of the images by the detector array impacts the figure of merit. For remote sensing scenarios an image shift corresponded to a particular source wavelength. The sampling by the detector array added uncertainty to image shift measurements providing a measure of the spectral resolution due to the hardware. The image shift figure of merit describes how spectral information is encoded in the raw data by the hardware and can be used to estimate performance prior to post-processing.

Over the years, data from Terra and Aqua MODIS (The Moderate Resolution Imaging Spectroradiometer) has provided invaluable information about Earth's atmosphere, land and oceans. Both MODIS sensors have exceeded their designed lifetime of 6 years and are still in operation. Therefore, it is important to understand the on- orbit performance of both sensors. The thermal emissive bands (TEB) on MODIS are comprised of 16 spectral bands with wavelengths from 3.7 to 14.4 μm. The TEB are calibrated on orbit on a scan-by-scan basis using a blackbody calibration source. Since the mission beginning, a steady increase in electronic crosstalk has been observed in Terra MODIS bands 27-30. The MODIS Characterization Support Team (MCST) at NASA/GSFC has recently derived a set of correction factors that correct these bands for the entire mission. These corrections are being applied in Terra Collection 6.1. In this study, the effectiveness of this crosstalk correction is assessed. First, the observed brightness temperatures over Dome C for Terra with and without the crosstalk correction are compared. Then, the calibration consistency and stability between the TEB of Terra, with the crosstalk correction, and Aqua MODIS are also assessed. Finally, the relative bias between the two MODIS instruments is evaluated using the near-surface temperature measurements from an Automatic Weather Station (AWS).

The Visible Infrared Imaging Radiometer Suite (VIIRS) onboard the Suomi National Polar-orbiting Partnership (SNPP) satellite has operated successfully since its launch in October, 2011. The calibration using regular solar diffuser (SD) observations for the 14 reflective solar bands facilitates the generation of high-quality calibrated products, known as sensor data records (SDR). As SNPP VIIRS nears six years on-orbit, monitoring the on-orbit calibration performance using multiple techniques is vital. One such technique, using the deep convective clouds (DCC), has been widely used to monitor the on-orbit calibration performance of earth observing sensors, such as the MODIS instrument on Terra and Aqua spacecrafts. In this study, the DCC technique is utilized to evaluate the stability of SNPP VIIRS SDR reflectance product for 10 moderate resolution bands (M-bands, M1–M5 and M7– M11) and three imaging bands (I-bands, I1–I3). An empirical Bidirectional Reflectance Distribution Function (BRDF) correction for the long-term reflectance measurements over DCC is formulated and implemented. The fluctuation in the long-term trending is analyzed to evaluate the uncertainties. The BRDF-corrected reflectances over DCC are used to evaluate the stability of the response versus scan-angle (RVS) and its effects in the SDR products. The RVS effects are analyzed based on the difference in mode reflectances over DCC among six different frame zones. Results indicate that the RVS effects should be monitored for an improved VIIRS RSB calibration in the future.

Modeling a space borne imaging system is key in predicting mission utility and exploring the sensor design trade space. It is important to capture critical real world phenomena in the modeling as accurately as possible to optimize design parameters. As a step toward optimal design of spectral imaging systems, this work presents simulation techniques that were used to model a panchromatic imaging system and predict well-known image quality metrics for a range of values of a key optical design variable - the effective focal length (EFL).We designed a at desert scene that included 7% and 15% reflectance panels and generated simulated images for a range of EFLs. The panels were used to calculate a sensor signal-to-noise ratio (SNR). The simulation incorporated a summer atmosphere with a collection time and geometry set to produce zenith solar and nadir collection angles. Platform motion and height with a given integration time for known detector parameters were also incorporated to produce images with changing EFL. A point spread function (PSF) of a typical optical system was incorporated that was scaled according to the EFL for a constant aperture diameter in order to capture the optical resolution changes. The PSF along with smear from the platform motion and integration time introduced realistic image blur to enable the relative edge response (RER) for the system to be estimated from the simulated images. The simulation used the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model that incorporates ray-tracing techniques and physics based radiation propagation modules. The SNR, RER, and GSD calculated from the synthetic images were used to estimate the National Imagery Interpretability Rating Scale (NIIRS) rating for a range of EFLs. An optimum EFL was found by this process which included the compensating effects of resolution, sampling, and noise as the EFL changed. This technique is currently being expanded to assess trades for sensor design in order to optimize optical payload designs for multispectral and hyperspectral imaging systems.

For long range targeting, the limited focal length and aperture size associated with compact imaging sensors for airborne operation limit both the spatial resolution and the image brightness. This presents a serious challenge to the identification and tracking of targets. Algorithms that derive target shape and track movement through a scene require a resolved image and use pixel contrast to discriminate the target image from the background. This is of limited use when practical deployment demands the use of compact imaging systems with necessarily limited spatial resolution.

To address this we consider a 2D mosaic filters sampling scheme to acquire an incomplete multispectral data cube on a single frame readout from a focal plane array. Specifically, the sparse data cube contains 4 x 4 spatial cells and 16 wavebands with each waveband sampled once per cell; this corresponds to a 1/16 undersampling of the data cube. Complete multispectral images are then computed using compressed sensing protocols.

Results obtained using hyperspectral datasets from AVIRIS and Stanford University (SCIEN) are presented to demonstrate image reconstruction using 16 wavebands in the visible and near infrared. The function of the mosaic filter is mimicked by sampling the full dataset according to the design of a theoretical mosaic filter. This allows us to investigate different sampling strategies and, in particular, make a direct comparison between random and regular sampling. Our results show that the reconstruction error is strongly dependent on both the colour content and the sampling strategy in the test images, and that very good reconstruction can be achieved approaching the spatial resolution of the original image. Our results can be applied to both the MWIR and LWIR where the lower spectral resolution means that a smaller number of wavebands is likely to be sufficient for identification and tracking. The concept can also be extended to polarimetric imaging with a suitable polarimetric filter mask to provide a dual-mode polarimetric-multispectral imaging capability. This paper presents an overview of the technical approach and the general conclusions.

Accurate estimation of surface emissivity from long-wave infrared (LWIR) hyperspectral imaging data acquired by airborne or space-borne sensors is necessary for many scientific and defense applications. This process consists of two interwoven steps: atmospheric compensation (AC) and temperature-emissivity separation (TES). AC uses an estimated atmospheric model to estimate the ground radiance from the at-aperture radiance, and then TES produces a temperature and emissivity estimate using this ground radiance. Hyperspectral TES algorithms assume that the emissivity spectra for solids are smooth compared to the atmosphere. The quality of TES estimates depend on the accuracy of the atmospheric estimates and their band-averaging to the sensor spectral resolution. The objective of this paper is to study the sensitivity of TES results to errors in atmospheric estimates. There are several errors associated with TES output including temperature errors, and shape, bias, and smoothness errors in the emissivity. The effects of atmospheric errors are studied by using MODTRAN to generate models with varying temperature and water vapor amounts. One model is used to simulate at-aperture radiances while the others are used for AC and TES on the simulated spectra. The mismatch between the models affects the TES results. While most errors cause TES to fail providing rough emissivity estimates, magnitude and shape errors provide smooth but incorrect emissivities. Understanding how atmospheric errors affect the TES estimates and what is required of atmospheric estimates to provide smooth emissivity estimates will help in developing algorithms to find the correct atmosphere for a scene.

Discussions with intelligence analysts indicated there were issues with dynamic range of remote sensor imagery to assess the best details. Further, hyperspectral image analyses require better image data undistorted by the atmospheric propagation due to unknown levels of ozone, carbon dioxide, humidity, aerosols, etc.. These airborne constituents can selectively absorb or scatter various wavelengths of spectral content through the column of space between the objects under view and the sensor collecting the image. To determine an atmospheric transmittance transfer function, we have developed the concept of an operational calibration panel that is composed of calibrated reflectors and emitters with a local witness sensor to validate short range calibrated radiance. The panel is placed in the area of collection and the calibrated emissions are collected at the same time as those by the remote sensor. In this manner, ground truth for pixels in an image can be utilized to determine the atmospheric transfer function and this information can be applied to improve intelligence collected from imagery over a larger area of collection.

Extremely thick haze caused by air pollution is observed in many satellite images of the earth, and in particular over eastern China. Standard image display software typically provides satisfactory visualization of the ground through automated or user-driven scaling to enhance contrast; however, it does not perform well with these highly polluted scenes, where the haze is spatially non-uniform. Furthermore, estimation of surface reflectance using standard atmospheric correction software is highly problematic under these conditions due to very low visible transmission of the haze coupled with lack of knowledge of its optical properties, which may not conform to the haze or aerosol models in the software. In this paper we show that a version of the empirical Quick Atmospheric Correction (QUAC) algorithm, adapted for spatially dependent scattering, produces visually satisfying imagery of the entire ground in multispectral satellite scenes containing thick haze, and that the output reflectance spectra appear to be realistic enough for performing basic surface classification. The QUAC algorithm is applicable to multispectral and hyperspectral imagery with any number of wavelength bands, including true color (RGB) imagery, and does not require radiometrically calibrated data.

Within the last few years, several commercial long-wave infrared (LWIR) hyperspectral imaging (HSI) systems have been developed for remote sensing of the ground from aircraft. While much less expensive and more practical to operate than sensors such as SEBASS and MAKO, which have been developed primarily for research and Government use, the commercial systems have poorer signal-to-noise and/or spectral resolution. We investigate the utility of three commercial systems—the Telops Hyper-Cam, SPECIM AisaOWL, and ITRES TASI-600—for quantitative retrieval of surface temperature and emissivity spectra. Atmospheric retrieval, correction and temperature-emissivity separation are performed on example data from these sensors using FLAASH-IR, a first-principles algorithm that incorporates radiation transport calculations and atmosphere models from MODTRAN. The results from the commercial sensors are noisy compared with SEBASS but otherwise appear to be reasonable. Applying a noise suppression algorithm to the radiance data yields better temperature retrievals and much cleaner emissivity spectra, with minimal loss of information, and should benefit scene classification applications.

This paper compares four different bad pixel replacement algorithms for LWIR hyperspectral imagery representing both physics-based unmixing and statistics-based methods. Testing is performed on a measured dataset using detection performance as a comparison metric and synthetic dataset with reconstruction error and detection performance used to compare. It is found that a statistics-based covariance matched filter method generally performs best of the four methods tested but at significantly greater computational cost. A simple plate metaphor interpolation is the fastest technique but struggles to correct pixels where sharp spectral or spatial difference are present. Meanwhile, an endmember-based unmixing approach provided a balance between the two in terms of computational complextity and reconstruction performance.

The Air Force Institute of Technology, in collaboration with Pratt and Whitney and Test Devices Inc., made midwave infrared (IR) measurements of a well-characterized JetCat turbine engine using a hyperspectral imager. The Telops hyperspectral imager used is an infrared Fourier transform spectrometer (IFTS) that utilizes a 320 by 256 indium-antimonide (InSb) focal plane array (FPA) sensitive in the short and mid-wave portion of the IR. The imager is calibrated to give a spectral distribution of radiance for each pixel within the image. Measurements were also made using a Santa Barbara Focal Plane IR imager (InSb, 320 by 256 array) that reported integrated radiance values in the mid-wave IR. These measurements were made at varying engine RPM and different nozzle orientations to determine temperature and concentrations of the flow. The different orientations allow for the determination of the degree of asymmetry in the flow in two dimensions. Corresponding local temperature and pressure readings were taken in order to corroborate the radiometric data. The spectral information from these measurements are used to determine temperature and species concentrations of carbon dioxide and water vapor produced by the combustion of the Jet-A fuel. These data provide detailed information about turbulent eddies of the exhaust gases, and in the construction of a volumetric representation of the flow.

Hyperspectral imaging in the longwave infrared region (LWIR) offers a unique day and night sensor capability currently not available in other spectral ranges. Current proliferation of the technology is however often limited by the size, weight, power and cost (SWaP-C) requirements of the available instruments. This paper presents a low-cost spatial Fourier Transform LWIR hyperspectral imaging camera, based on a corner cube Michelson interferometer and an uncooled microbolometer. In addition to conventional spatial pushbroom scanning, e.g. provided by a moving platform, the current interferometric setup can use temporal scanning of a stationary field-of-view due to a spatially offset stepper motor controlled corner cube retroreflector. The design and calibration and a characterization of the instrument are presented. Applications and future modifications are also discussed.

This paper presents stand-off measurements with an interferometric LWIR hyperspectral camera, which is based on a corner-cube Michelson interferometer and an uncooled micro-bolometer. Spectrally resolved image data may provide a more robust discrimination and, moreover, the ability to also classify and identify materials. This is however dependent on three parameters: instrument, source intensity and vibrational bands. Strong vibrational bands in LWIR can be found in minerals (solid, crystalline substances), textile fibers (polymers) and hazardous substances like toxic chemicals and explosives (smaller molecules). Spectral bands measured e.g. in textile polymers may be used in applications such as discrimination and identification of camouflage. Experimental setups and measured vibrational bands are presented, along with interpretations of vibrational frequencies in LWIR.

Change detection normally involves one reference image and one test image. The objective is to detect changes that are not caused by illumination, atmospheric interferences, and mis-registration and parallax between the two images. Conventional methods can alleviate these issues to some extent. Since there may be some applications where there are multiple reference images collected over time, it would be ideal to incorporate multiple reference images to further improve the change detection performance. In this paper, we present a new approach to change detection, which can explicitly incorporate multiple reference images into account. Extensive experiments using actual hyperspectral images clearly demonstrated the performance of the new approach.

Change detection using hyperspectral images is important in surveillance and reconnaissance operations. The process involves two images: one reference and one test. Many algorithms such as chronochrome (CC) and covariance equalization (CE) were proposed in the past. In this paper, we will present a new nonlinear change detection framework for hyperspectral images. The idea was motivated by the band rationing concept. First, image segmentation is applied to the reference image. For each segmented subimage in the reference image, the bands with the most and least variations are found. Then new images are formed by dividing the two bands. Similarly, the new band ratioed images are formed in the test images. Second, we propose to apply CC or CE to generate residual images. Finally, anomaly detection algorithms are applied to detect changes. Actual hyperspectral images have been used in our studies. Receiver operating characteristics (ROC) curves were used to compare the various options. Results showed that this approach can achieve excellent detection performance.

The detection of temporal changes or new activity in large areas of satellite imagery have been of great interest in many applications ranging from meteorology, environmental-sensing, to defense and disaster monitoring. Automated change detection algorithms play a significant role in analyzing large volumes of satellite data, which otherwise would be a manually laborious task. This paper deals with the evaluation of various change detection methods for detecting both structured and unstructured changes in multi-spectral satellite imagery. A comparison of the algorithm performance on both real and simulated satellite imagery is presented here. The real-world dataset (WorldView-2) required accurate geometric registration and manual ground-truth labelling, while on the other hand, the simulated satellite imagery (generated with the RIT DIRSIG tool) has the advantage of prompt availability and included ground-truth. A variety of structural spatial and spectral changes were incorporated into the simulated images according to the required complexity and the ground truth generated together with the imagery. An array of algorithms were evaluated and compared, including conventional methods such as differencing, ratioing, image transformation techniques; statistical methods such as chronochrome, covariance equalization; linear/non-linear distribution-based methods such as point density, simplex volume/complexity, mean-shift/outlier metrics, and non-linear graph-based methods. The Receiver-Operating-Characteristic (ROC) curve and Area-Under-Curve (AUC) metrics were used to evaluate the algorithms. Finally, the performance of various algorithms on simulated vs real satellite images is also discussed, which may be helpful in further applications of the simulated imagery, such as for training deep-learning networks, where real imagery is not readily available.

In this paper, we develop and evaluate change detection algorithms for longwave infrared (LWIR) hyperspectral imagery. Because measured radiance in the LWIR domain depends on unknown surface temperature, care must be taken to prevent false alarms resulting from in-scene temperature differences that appear as material changes. We consider two strategies to mitigate this effect. In the first, pre-processing via traditional temperature-emissivity separation (TES) yields approximately temperature-invariant emissivity vectors for use in change detection. In the second, we adopt a minimax approach that minimizes the maximal spectral deviation between measurements. While more computationally demanding, the second approach eliminates spectral density assumptions in traditional TES and provides superior change detection performance. Examples on synthetic and measured data quantify computational complexity and detection performance.

Image registration is used by the remote sensing community to align images for the purposes of examining changes in a scene. The application in this paper involves finding anomalies associated with human activity for the purpose of detecting underground nuclear explosions. This paper presents a non-rigid image registration algorithm that can be easily implemented using publicly available tools such as python, numpy, scipy, openCV and SIFT. SIFT is used to find feature correspondences between images. An approach based on Mahalanobis distance is used find a subset of robust correspondences. Comparisons are made to the RANSAC algorithm. The imagery was collected by DigitalGlobe’s Worldview-II satellite. One image pair is orthorectified. A second image pair is only geo-registered. Both image pairs were collected over mountainous desert regions, the second image pair has much rougher terrain and presents a challenging situation. The non-rigid property of the image registration algorithm allows for robust registration in mountainous terrain under different viewing geometries. Image differencing of the PAN-chromatic band is used to find changes, some of which are shown in detail for both sets of images. Overall registration improvement is quantified by using the standard deviation of the difference image.

The non-rigid warping map was also applied to the multispectral bands of the DigitalGlobe data. This dataset made use of a multivariate change detection algorithm that incorporates the spectral properties of each pixel.

Extracting well distributed control points (CPs) is a very challenging task for remote sensing image registration, particularly for large high-resolution images over heterogeneous landscape. Based on image analysis such as edge detection, corner detection, and information theory, a new CP detection approach is proposed to select high- quality, evenly distributed CPs. The Entropy-Block-Based variant of the Harris Corner Detector (EBB-HCD) is achieved by dividing the image into blocks and by allocating the number of CP's based upon the entropy of each block. While the block-based strategy improves the CP balance problem, a factor calculated from entropy avoids overdetection. We conducted a comparison study utilizing the well-known Harris Corner Detector (HCD) and an implementation of the Block-Based Harris Corner Detector (BB-HCD). Experimental results indicate that using EBB-HCD to find the CPs improves the overall alignment accuracy during registration compared with HCD or BB-HCD.

The Selden Map of China, an early 17th century wall map of East Asia was rediscovered in 2008. Recently, the map was imaged using a VNIR hyperspectral imaging system while in the collection at the Bodleian Library, Oxford University. The goal of the collection of the hyperspectral image (HSI) of the Selden Map was to help historians understand the material diversity of its composition and potentially the methods used in the creation of the map. The Selden map has been named one of the ”Treasures of the Bodleian” and it poses many questions such as the diversity of pigments used to create the map. In this research, we extract visually common pixels (here, the green pigments) from the Selden Map and estimate the material diversity of the green pixels. Previous pigment analysis on the HSI of the Gough Map^{1, 2} used an endmember based approach, the Gram Matrix technique^{,3, 4} to understand the number of distinct materials in a scene and then used spectral angle mapper (SAM) to classify all the pigments. Here, we use the same Gram Matrix technique, but due to the complexity of the Selden Map data, instead of using SAM, we use two spectral unmixing techniques, NNLS (nonnegative linear least squares)⁵ and FUMI (functions of multiple instances)⁶ to determine the weights of all the endmembers for each data point to study the within-material diversity. Results show that the Selden Map is composed of at least 6 kinds of dominant green pigments with a particular spatial pattern. This research provides a useful tool for historical geographers and cartographic historians to analyze the material diversity of HSI of cultural heritage artifacts.

Precision agriculture (PA) consists of agricultural practice modulation according to intra-field crop variability. Multispectral sensors have standing use in remote sensing, onboard aircraft and satellites for mapping plant cover, biomass, chlorophyll content etc. Absolute reflectance estimation is mandatory to compare data from different weather conditions, time of the day and geographical locations. Real world reflectance estimation requires detailed knowledge of two sensors: one for vegetation radiance imaging; another for measurement of ambient irradiance. If the latter is straightforward with well-tuned Ambient Light Sensors (ALS) and interferential filters, the former requires a prior knowledge of the relationship between irradiance, acquisition parameters and digital number (DN). In the absence of an a priori model of the imaging sensor, its modelling requires different temperatures, irradiances, gains and exposure times to fully explore its behavior. After exploration and modelling of the entire measurement space, a realistic subset of acquisition parameters is chosen for automated calibration on the camera manufacturing line using an integrating sphere and a reference spectrophotometer. Imaging of a reflectance target completes the calibration of the entire system. A targetless workflow has the potential to enable PA for a much larger public.

Results from Parrot SEQUOIA will be presented to illustrate the hurdles of multispectral camera characterization and calibration at an industrial scale. The consequences of the difference between the calibration test bench geometry and the on-the-field use will be discussed opening the road to a targetless workflow for multispectral cameras.

The new geostationary satellites, G-16 and Himawari-8 carry high-resolution advanced baseline imagers, ABI and AHI. The ABI onboard G-16 provides imagery in two narrow visible bands (red, blue), while ABI’s twin sensor AHI onboard Himawari-8 also has a green band, which allows the direct production of true color (RGB) images for AHI. Since natural color images are easier for both meteorologists and the public to interpret, it is important to provide true color imagery from geostationary orbit. In this paper we present a method to estimate green band for ABI from available visible and near-IR bands by building a statistical predictor trained on AHI data. Simple approaches such as look-up-table or simple linear regression on the multi-spectral input parameters may produce satisfactory results globally, but will fail to correctly estimate green band in some cases due to the underlying non-linearity of the data. We will present an approach which uses piecewise multi-linear regression on the multi-spectral input to train the green channel predictor. Our predictor is built from the combination of a classifier followed by a multi-linear function. Based on the values from the ABI bands, the classifier assigns each pixel to a class. Each class as an associated set of coefficients determining a multi-linear predictor mapping the ABI multi-spectral values to a predicted green value. This combination of a categorical classifier with per-class multi-linear function combines the efficiency of linear map while still preserving flexibility and accuracy by adjusting the number of classes.

Multispectral imaging can be used as a multimodal source to increase prediction accuracy of many machine learning algorithms by introducing additional spectral bands in data samples. This paper introduces a newly curated Multispectral Liquid 12-band (MeL12) dataset, consisting of 12 classes: eleven liquids and an "empty container" class. Multispectral images in this dataset have been captured using the PCO Ultraviolet, Grasshopper3 12.3 MP Color USB3 Vision, Mil-Rugged-High Resolution Snapshot Short Wave Infrared 1280JS, FLIR Medium Wave Infrared A6750sc and FLIR Long Wave Infrared T650sc cameras. Each of the classes initially results in a 640 × 480 × 12 data cube, where the 12 × 1 vector for each spectral pixel spans the spectral bands observed using the above-mentioned cameras and seven add-on bandpass optical filters. The usefulness of multispectral imaging in classification of liquids is demonstrated through the use of a support vector machine on MeL12 for classification of the 12 classes. The reported results are both encouraging and point to the need for additional work to improve liquid classification of harmless and dangerous liquids in high-risk environments, such as airports, concert halls, and political arenas, using multispectral imaging.

A three-mirror off-axis optical system was designed as a fore optics of a hyperspectral imager. The secondary mirror (M2) has an obscuration hole in the middle for the slit assembly. Despite of the disadvantages of having a slit hole in the mirror surface such as stray light defect and mirror surface fabrication difficulty, the configuration has great advantages of maximizing telecentricity while maintaining the wide field of view (FOV). [1] To evaluate the integrity of the optical system design, the stray light effect was analyzed including the spectrometer channel and confirmed that it has little effect on the image performance. Also, the RMS wave front error (WFE) of M2 is measured to be 20.12 nm exceeding our expectation including the edge of the hole. As a result, the optical system is aligned to have WFE less than 90 nm RMS in all fields. [1]

Recent interest in building an imaging system using diffractive optics that can fit on a CubeSat (10 cm x 10 cm x 30 cm) and can correct severe chromatic aberrations inherent to diffractive optics has led to the development of the Fresnel zone light field spectral imaging system (FZLFSI). The FZLFSI is a system that integrates an axial dispersion binary diffractive optic with a light field (plenoptic) camera design that enables snapshot spectral imaging capability. This system suffered from poor resolution and a modified FZLFSI based on full resolution light field rendering has been built and tested. The modified FZLFSI shifts the optical elements to different positions which change the way the light field is encoded on the focal plane array. The new encoding increases the available spatial information at the expense of some spectral information. The system was tested for different internal system parameters, at a range of wavelengths, and the resulting tradeoffs between spatial and spectral performance were studied. The performance of the modified FZLFSI was compared to that of the conventional FZLFSI and optimal internal system parameters identified for different imaging scenarios.

Imaging spectroscopy is of great importance for applications such as environmental sensing and astronomy observations. These devices rely on the acquisition and storage of vast amounts of spectral and spatial information. As experimental techniques are becoming more sophisticated, there is an increasing amount of data to collect and process. Here, we show that spectral information can be undersampled and fully reconstructed at nm resolution, with the aid of a multiple scattering material. We measure spectrally- and spatially-dependent speckle patterns, over a broad wavelength range, which are calibrated and stored in a spectral intensity transmission matrix. A compressive sensing technique is used to limit data collection at the acquisition stage during both the calibration and the reconstruction process. This enables fast reconstruction with minimal data storage needed, and offers a solution for light-weight, compact hyperspectral imaging systems.

The use of optical frequency combs (OFCs) for multi-heterodyne spectroscopy has enabled unprecedented measurement capabilities for spectroscopic sensing, including rapid acquisition speed, high resolution and high sensitivity^1,2. Development of field deployable OFC sources that are widely tunable in the important chemical fingerprint region in the long-wavelength infrared (LWIR) is a major research challenge. In this paper, we report our recent efforts towards developing LWIR comb source for SILMARILS (Standoff ILluminator for Measuring Absorbance and Reflectance Infrared Light Signatures) program by IARPA. LGS has developed fiber optic sources producing spectral combs in the SWIR (1.52 to 1.56 μm and 1.7 to 2.0 μm) and in the LWIR (7.7 to 12.1 μm) regions. The spectral combs in the LWIR are generated by difference-frequency mixing one OFC centered around 1.54 μm with another OFC, whose center wavelength is tunable between 1.7 and 2.0 μm, in a nonlinear optical crystal. Average power of the generated LWIR is 1.2-12 mW and its instantaneous spectral breadth of the combs is > 80 cm^-1, sufficiently broad to cover multiple molecular absorption peaks. We demonstrate standoff sensing of chemical targets having concentration as low as 12 μg/cm² by measuring LWIR transflectance spectra using the comb source.

For spectral cameras, the spatial resolution and coregistration of bands are important performance characteristics. The paper discusses how these can be quantified in a way that is relevant for the spectroscopic processing step which is at the beginning of most spectral image analysis methods. It is argued that the most appropriate measure of resolution may be the ensquared energy of the mean point spread function (PSF) within the specified pixel field of view. For spatial coregistration, it is argued that a previously proposed coregistration metric is well suited, by characterizing the full shape and position differences between PSFs of different bands. A relatively simple method for imaging the PSF is demonstrated, based on tomographic reconstruction from line spread functions measured in different directions. The method is used to characterize two presumably comparable commercial hyperspectral cameras, a SpecIm PFD dating from 2012 and a HySpex VNIR-1600 dating from 2016. The measurements reveal significant differences, but it is pointed out that the present measurements by themselves do not constitute a proper comparison of the camera types. The main point made in the paper is that full characterization of the PSF in all bands is possible with moderate experimental effort, and provides clarifying measurements of actual camera performance.

A miniaturized, lightweight turn-key hyperspectral sensor package incorporating a single, monolithic spectrograph, telescope and navigation system is being built for airborne applications on small, Unmanned Aircraft Systems (UAS). The sensor is based on Corning’s existing MicroHSI 410 Vis/NIR Selectable Hyperspectral Airborne Remote sensing Kit (SHARK) currently used for airborne agricultural monitoring. Under DOE sponsorship, we are extending the approach to cover the full spectral range from 0.4-2.5 microns with a single spectrograph. This will enable rapid aerial surveys of vegetative mass, quality, and carbon sequestration. Other applications include mineralogy, agriculture, and intelligence/surveillance/reconnaissance (ISR).

The sensor features an Offner-type spectrograph machined from a single transmissive block. The monolithic construction provides an unprecedented combination of high performance, low cost and low size, weight, and power. It has an f/1.4 aperture, 5 nm resolution, and measures only 46mm x 60mm x 76mm. The spectrograph block is coupled to a sterling-cooled, back-thinned, HgCdTe FPA covering 0.4-2.5 micron spectral range. The flight package, including spectrograph, camera, telescope, and navigation system weighs less than 2.4kg and can fit on group 1 UASs.

In this paper, we present the design and optical performance of the sensor, and a detailed physical model of detection performance in standard, airborne hyperspectral sensing applications. At 100 Hz data rate, the sensor will achieve shotnoise limited performance with SNR > 250 from 0.4-1.7 microns and SNR<100 between 2-2.3 microns. Operating procedures for airborne monitoring of vegetative properties are also discussed. Initial test flights on a UAS are scheduled for next summer.

The BRDF describes optical scatter off realistic surfaces. The microfacet BRDF model assumes geometric optics but is computationally simple compared to wave optics models. Previously, densely-sampled MERL BRDF data for several materials was analyzed using a novel variation of a microfacet BRDF that used a polarization factor in place of the cross section conversion and geometric attenuation terms, demonstrating improved accuracy. This paper extends that analysis to examine high-fidelity grazing angle BRDF data measured in-plane with the novel BRDF modification. Results indicate that for many materials the novel BRDF modification is more accurate than the Traditional Cook-Torrance BRDF at near grazing angles. We show as much as an order of magnitude improvement in the fit error using this novel BRDF modification. These results are expected to lead to more accurate BRDF modeling for remote sensing, computer graphics, and scene generation.

Advanced combat uniforms with multiple tonal (blended) colors and highly detailed camouflage patterning require additional test methodologies for color evaluation. Here the apparent color, which is the combination of all visible wavelengths (380-700 nm) of light reflected from large (≥1m²) fabric sample sizes for a given standoff distance (25-100 ft), is evaluated as a potential test method. Camouflage patterns lose resolution with increasing standoff distance, and eventually all colors within the pattern appear monotone (the “apparent color” of the camouflage pattern). This paper presents a case-study analysis using apparent color for the purpose of model development relevant to evaluating color in camouflage fabrics.

In the field of Defence and Public Security, standoff detection is a useful tool to identify many threats, such as chemical warfare agents, toxic industrial compounds and other chemical substances under various physical states (gas, liquid and solid). Over the years, Defence Research and Development Canada (DRDC) - Valcartier Research Centre has built an extensive expertise in the field of hyperspectral standoff detection. Although the technology behind the detection instruments is well known, the interpretation of spectral signatures of materials can sometimes be difficult. In reflectance infrared spectroscopy of solids, the same molecule can yield different signatures depending on sample surface roughness and particle sizes as opposed to the transmission and absorption infrared spectroscopies for which the signatures are significantly less influenced by these parameters. In principle, molecules containing one ionic bond, such as NaCl, must not exhibit infrared peaks, since only covalent bonds are supposed to generate active bands in the thermal infrared region. However, experimental results show significant spectral features in reflectance infrared spectroscopy. In fact, in reflectance spectroscopy, the volume diffusion phenomenon can generate spectral features even if no fundamental vibration occurs in the molecule. An effort has been undertaken at DRDC to understand the phenomenology associated with these variations. This paper summarizes experimental results which emphasize the roles of surface roughness and particle size in the interpretation of reflectance infrared spectra of solid materials.

The problem of spectral reflectance retrieval of surfaces via remote hyperspectral imaging is challenging even in benign scenarios, and becomes dramatically more difficult under complex illumination conditions. Shadows, reflections from nearby structures, and atmospheric scattering can all severely impact the observed radiance from ground-level surfaces. In order to study this problem, MIT Lincoln Laboratory recently conducted an airborne data collection experiment that included hyperspectral, laser radar, and pan-chromatic modalities. A comprehensive ground truth data set and extensive efforts directed at sensor characterization makes this data set ideal for the development of hyperspectral exploitation algorithms.

This paper presents a method for hyperspectral image classification that uses support vector data description (SVDD) with the Gaussian kernel function. SVDD has been a popular machine learning technique for single-class classification, but selecting the proper Gaussian kernel bandwidth to achieve the best classification performance is always a challenging problem. This paper proposes a new automatic, unsupervised Gaussian kernel bandwidth selection approach which is used with a multiclass SVDD classification scheme. The performance of the multiclass SVDD classification scheme is evaluated on three frequently used hyperspectral data sets, and preliminary results show that the proposed method can achieve better performance than published results on these data sets.

A series of measurements of keystone and smile is required to assemble and aligning hyperspectral imagers. An efficient measurement method was proposed to calculate the keystone and the smile in a single measurement image by implementing an additional tool called the Field Identifier (FI) [1]. The measurement method is simple enough to make it possible to measure the wave front error (WFE) making minimum change from each measurement setup. To evaluate the accuracy of the measurement method, the positional data of 380 points were collected and 3σ was calculated along spectral and spatial axis. The measurement error calculated as 3σ is to be less than 1/10 of the performance goal to prove the effectiveness of the measurement method.

The acceleration of the acquisition of spectral images and their processing is important for the acceptance of these measurement methods in quality assurance and inspection. A frequently used preprocessing step is the Principal Component Analysis (PCA). It is used in variations, for example, for segmentation, spectral decomposition or data compression. The presented implementation calculates the PCA for the 12 spectral image channels of a filter wheel camera parallel to image acquisition. This includes the determination of the covariance matrix, the calculation of the main components and the transformation of the data. The parallel processing during the sequential imaging acquisition is performed on a System-on-a-programmable-chip (SoPC) Xilinx Zynq-7000 directly within the camera. The algorithm is partitioned into hard and software components and implemented in the field programmable gate array (FPGA) fabric as well as the ARM processor core firmware of the SoPC. In order to ensure the steps of the image acquisition chain in addition to the calculation, the system was implemented as an asymmetric multiprocessing system (AMP) with individual processors. For additional acceleration under static conditions (e.g. continuous testing in the manufacturing process), the feature vector can be stored as a calibration value. The calculation is reduced to the transformation of the data.

A passive Fourier transform infrared (FTIR) spectrometer is an instrument that can detect and identify chemical contaminants. An FTIR spectrometer exploits the infrared radiation of the surrounding terrain as a light source and receives a mixed signal of background signal, gas signal, and noise. The performance of most detection algorithms for detecting gaseous plumes, such as the normalized matched filter (NMF) and adaptive subspace detector (ASD), deteriorates due to the noise generated by an FTIR spectrometer. In this paper, a noise reduction algorithm based on the maximum noise fraction (MNF) transform to improve the performance of hazardous gas detection algorithms is proposed. We apply the MNF transform to the measured spectra and remove the high noise fraction component. Then the noise-reduced spectra are restored by conducting the inverse MNF transform. The experimental results show that the proposed algorithm reduces the noise and enhances the gas detection performance.

Traditionally, material surface classification has relied on reflectance spectrum measurement and pixel-wise comparison. For best results, the measurement usually requires measurement of the full spectral reflectance which can be the time-consuming issue and error-prone. Our previous works have proved that convolutional neural networks could learn the Bi-directional Texture Function (BTF) data feature hierarchy from pixels to the classifier. In this paper, we trained a wide-dense neural network for BTF data. The dense neural network was developed from the residual network (ResNet), which made a residual mapping for these stacked layers instead of directly fitting a desired underlying mapping. This shortcut idea has excellent performance in deeper layers. The dense structure reduced the redundancy within the feature maps of the individual layers and increased training speed. The narrow size of the dense net layers simplified the feature map for the whole network. The wide structure of the network was applied due to the resolution of our BTF data. In this paper, we generated BTF angular maps for material classification. Our data consisted of 151 lighting and 151 viewing directions. This angular resolution is higher than input data of conventional pre-trained dense network model. By adding a wide net structure, training time is reduced and classification performance improved. Training our own networks for specific data requires a large training dataset, therefore we augmented angular BTF image to improve the robustness of the training model. Within the over 30,000 BTF angular maps, we obtained a more reliable training model. Finally, we compared the improvement of our wide-dense network with other pre-trained neural networks and other feature extraction network.

The Spectral-Polarimetric Hypersensor (SPH) is a new compact, solid state, video rate imaging system with no moving parts capable of capturing up to 16 bands of both spectral and polarimetric content simultaneously. For these imagers, which are an adaptation of a plenoptic system, all elements for a given super pixel sample the same point in the scene, thus providing the ability to accurately measure point sources. Various scenes are imaged using two systems that span the VNIR (0.4 – 0.95nm) and SWIR (0.9-1.7m) wavebands. For each imager, twelve spectral components and four polarization states are captured simultaneously. The VNIR system captures full data cubes of 512x512 (spatial pixels) x 12 (spectral bands) x 4 (polarization states) at frame rates up to 30 Hz while the SWIR system captures full data cubes of 320x256 (spatial pixels) x 12 (spectral) x 4 (polarization states) at frame rates up to 30 Hz. From the acquired images, we compute the Stokes vector components, calculate DoLP, polarization angle, and obtain a spectral signature of objects in the scene. This data is then used to carry out target identification and clutter suppression in real-time, i.e. video frame rates.

This study describes a parametric model of diffuse reflectance for the purpose of simulating the spectral response of near-infrared (NIR, 0.7-0.9 μm) and shortwave infrared (SWIR, 0.9-1.7 μm) absorbing dyes for tailoring the NIR-SWIR reflectance of fabrics. This model combines diffuse-reflectance theory with reference spectra for simulating absorption in fabric of NIR/SWIR absorbing dyes whose absorption spectra span the NIR/SWIR spectral range. Fabric samples consisting of a cotton blend were used as the test substrate for NIR/SWIR dye application. The results of this study demonstrate application of the parametric model for simulating NIR/SWIR spectral responses corresponding to variable dye and dye blend concentrations in fabrics.

Over the years, data from different satellites has provided invaluable information about Earth's atmosphere, land and oceans. The thermal emissive bands (TEB) on the Moderate Resolution Imaging Spectroradiometer (MODIS) are comprised of 16 spectral bands with wavelengths ranging from 3.7 to 14.4 μm. MODIS TEB are calibrated on orbit on a scan-by-scan basis using an on-board blackbody (BB). Sentinel-3 Sea and Land Surface Temperature Radiometer (SLSTR), launched on 16 February 2016, has 11 spectral bands with wavelengths from 0.55 to 12 μm. In this study, we compare the observed brightness temperature from MODIS bands 31 and 32 and SLSTR bands S8 and S9 over Dome C using a 20 × 20 km region of interest (ROI) centered at (75.102 °S ,123.395 °E). A total of 2989 scenes for Terra, 2963 for Aqua and 1961 for SLSTR from November, 2016 to January 2018 are analyzed. The relative bias between MODIS and SLSTR is evaluated using the near-surface temperature measurements from an Automatic Weather Station (AWS).

This paper evaluates the detection performance of the three subpixel target detection algorithms based on the spectral signature of a target. Three subpixel target detection algorithms, Adaptive Coherence Estimator (ACE), Spectral Matched Filter (SMF), and Constrained Energy Minimization (CEM) are evaluated and compared using the Principal Component Analysis (PCA) spaced RIT Avon12 hyperspectral dataset. The performance of the three detectors is evaluated by generating the Receiver Operating Characteristic (ROC) curve. The ROC curves are generated by uploading the detection statistics image produced by the three detectors to the Data and Algorithm Standard Evaluation ( DASE) Website of IEEE Geoscience and Remote Sensing Society(GRSS) . Finally, we note the Area Under Curve (AUC) as the proposed utility metric value to evaluate the performance of the three detectors. The AUCs of the ROC curve produced by the ACE, CEM, and SMF are 94.0 %, 93.9 %, and 87.2 % respectively.

Two-dimensional digital flattening was implemented on distorted 3D documents for the Rochester Cultural Heritage Imaging, Visualization, and Education (R-CHIVE) database, with the purpose of exploring new techniques to enhance usability among historians and analysts. This process was investigated with a minimal cost RGB scanner for implementation into a future hyperspectral system. Results from the flattening improved readability of the distorted document when compared to a simple nadir view. The flattening process was then combined with pan-chromatic sharpening to produce a high spatial and spectral resolution image removed of projective distortion. The effectiveness of processing order was investigated relative to spectral, spatial, and flattening accuracy. Flattening and then sharpening proved more accurate both spectrally and spatially, with 60% the spectral error and 80% less FWHM of its line spread function. Two flattening methods were investigated and again the flattening first method outperformed the sharpening then flattening method, this time by 38.7%. This suggests, for simple flattening requirements, sharpening the image first, applying it to the 3D model, and then flattening the model provides the most accurate results.

The reflective solar bands (RSB) of the MODIS instruments on the Terra and Aqua spacecraft support a variety of science applications. The RSB are primarily calibrated using regularly scheduled solar diffuser (SD) and lunar observations. As the instruments continue to operate beyond their designed life, significant degradation of the instrument gain is observed, particularly at the short wavelengths. In recent years, the onboard calibrators are insufficient in accurately characterizing the response versus scan-angle (RVS) for the RSB. Therefore, the MODIS Characterization Support Team (MCST) has implemented an enhanced approach whereby the on-board measurements are supplemented by observations of the pseudo-invariant calibration sites in Northern Africa to better characterize the on-orbit RVS changes in Level 1B Collection 6 and Collection 6.1 for select short and near-infrared bands of both MODIS instruments. This paper explores the use of alternative Earth-scene targets to characterize the on-orbit RVS. In particular, an alternative version of the on-orbit RVS, with the desert data substituted with the measurements from deep convective clouds (DCC), has been formulated and implemented. Results from this version are compared with the operational C6.1 algorithm. The long-term reflectance trends from the Dome Concordia (Dome C) site in Antarctica are derived using both versions and evaluated for long-term stability for independent assessments. Due to the Earth-scene saturation from the high-gain bands, the paper only focusses on the comparisons for bands 1, 3 and 4 of both MODIS instruments. This study finds that DCC can be used as alternative invariant ground targets in MODIS RVS characterization for these bands.

Fast and precise target detection is an issue in underwater object detection applications such as search and rescue missions. Hyperspectral imaging could be effective in determining different elements in a region thanks to its ability to provide detailed spectral information. Utilization of this rich spectral information calls for the analysis of spectral differences of objects on water surface (dry/wet) and underwater conditions.

In our study, we investigated the effect of water column to the target spectrum with two water column correction models and evaluated the results obtained with these models in two detection algorithms. The results show that the methodology enhances underwater object detection performance. The materials used for the study are selected to be typical in search and rescue missions such as metal, cotton fabric with different colors and denim to evaluate the effect of these cases on the detectability of targets for such scenarios.

Cultural Heritage Imaging is a rapidly expanding area of research utilizing multi and hyperspectral imaging systems to study historical artifacts of significance. Current imaging systems are based on colored LED illumination and select filter wheels to produce imagery in reflective, transmissive, and fluorescent modalities. The advent of new, smaller, hyperspectral systems provides an opportunity to re-think the imaging system for this particular application. This project is to assist in the design and development of a new hyperspectral imaging system for imaging historical documents. Specifically, this project will conduct a band selection study for the imaging system to use for this application, which leads to decisions such as which illumination sources, sensors, and filters should be available on the new imaging system. This paper will present spectral measurements of representative parchments, inks, and reagents. Measurements will be made using a point spectrometer covering 0.4 - 2.4 um. Fluorescence will be measured using a UV illumination LED in a darkened room. Measurements will be made of both bare parchment as well as parchment with ink and reagent applied.

Sparse Representation (SR) has received an increasing amount of interest in recent years. It aims to find the sparsest representation of each data capturing high-level semantics among the linear combinations of the base sets in a given dictionary. In order to further improve the classification performance, the joint SR that incorporates interpixel correlation information of neighborhoods has been proposed for image pixel classification. However, joint SR method yields high computational cost. To improve the performance and computation efficiency of SR and joint SR, we propose a seeded Laplacian based on sparse representation (SeedLSR) framework for hyperspectral image classification, where a hypergraph Laplacian explicitly takes into account the local manifold structure of the hyperspectral pixel in a spatial-type weighted graph. Given the training data in a dictionary, SeedLSR algorithm firstly finds the sparse representation of hyperspectral pixels, which is used to define the spectral-type affinity matrix of an undirected graph. Then, using the training data as user-defined seeds, the final classification can be obtained by solving the combination of spectral and spatial hypergraph Laplacian quadratic problem. To assess the efficiency of the proposed SeedLSR method, experiments were performed on the scene data under daylight illumination. Compared with SR algorithm, the classification results vary smoothly along the geodesics of the data manifold.

Recently superpixel-based approaches have been proposed for dimensionality reduction (DR) in hyperspectral images. The basic assumption of this approach is that the superpixel over-segmentation segments the image into small homogeneous areas. A low-dimensional (LD) image representation is obtained by using the average of the superpixels, which are then used in other image processing tasks up the processing chain. Due to superpixel-segmentation algorithm limitations, the region inside a superpixel may not be homogeneous. Therefore, the average may not be an adequate representation for the superpixel, leading to inaccuracies in the low dimensional representation resulting in errors in the image processing tasks or analysis. Here we present an enhanced superpixel-based dimensionality reduction approach that incorporates homogeneity testing of superpixels. Homogeneous superpixels are represented by their mean but heterogeneous superpixels are represented by multiple representative signatures selected using the SVDSS column subset selection algorithm. The representative signatures for the homogeneous and heterogeneous superpixels provide an improved low-dimensional representation for the hyperspectral image that better captures the image structure. We present experiments applying the proposed enhanced and the conventional superpixel dimensionality reduction approaches to unmixing using the constrained non-negative matrix factorization (cNMF). A subset of data from the Washington DC Mall HYDICE image is utilized. In the experiments, the enhanced superpixel-based dimensionality-reduction approach results in better unmixing results compared to the conventional approach and to unmixing using the full data set.

The development of a platform to mix active imaging sensors into an Augmented Reality Environment. The system makes use of a 2D LiDAR modified into a 3D LiDAR on a mesh network. The 2D LiDAR, alongside a custom designed mount gives the ability to provide a slow rotation rate 3D LiDAR scan of the environment in conjunction with 3D active imaging and sensors. The data can then interpreted by the Unity game engine to be used in applications with multiperspective features in Augmented Reality. The goal of this work is to explore this collected data and apply it to immersive technology applications developed as means towards ongoing open source development.

Kernel-based methods for anomaly detection have recently shown promising results - surpassing those of model-based statistical methods. This success is due in part to the distribution of the non-anomalous data failing to conform to the distribution model assumed by model-based statistical methods. Alternatively, the skeleton kernel principle component analysis anomaly detector (sKPCA-AD) assumes that a better background model can be learned by constructing a graph from a small, randomly sampled subset of the data (a skeleton). By definition, anomalies are rare and thus the sampling is assumed to be comprised chiefly of non-anomalous samples and correspondingly the learned graph models the background. Error magnitudes in the models' representation of data from the full data set are used as an anomaly measure. Additionally, the smaller skeleton sample makes kernel methods computationally feasible for hyperspectral images.

The sKPCA-AD has proven successful using unordered spectral pixel data, however, anomalies are often larger objects composed of many neighboring pixels. In this paper we show that fusing spatial information derived from a panchromatic image with spectral information from a hyper/multispectral image can increase the accuracy of the sKPCA-AD. We accomplish this by creating several joint spectral-spatial kernels that are then used by the sKPCA-AD to learn the underlying background model. We take into account the variability introduced by the random subsampling by showing averaged results and variance over several skeletons. We test our methods on two representative datasets and our results show improved performance with one of the proposed joint kernel methods.

Advanced camouflage patterns for military applications consist of highly detailed camouflage patterning and multiple tonal (blended) colors. The complexity of these camouflage patterns establishes a need for additional test methodologies for color and pattern evaluation. One metric for evaluation is apparent color, which is the combination of all visible wavelengths (380-700 nm) of light reflected from large (≥1m²) fabric sample sizes for a given standoff distance (25-100 ft). This follows in that camouflage patterns lose resolution with increasing standoff distance, and eventually all colors within the pattern appear monotone (the “apparent color” of the camouflage pattern). The concept of apparent color, however, is based on far-field and statistical characteristics of camouflage patterns. In contrast, the concept of apparent camouflage pattern is to be associated with intermediate distances between observer and target. Accordingly, quantitative metrics for camouflage-pattern viability based on apparent patterns should be different than those for apparent color, thus providing additional criteria for evaluation. This paper presents discussion and prototype simulations based on the concept of apparent camouflage pattern for model development relevant to evaluating camouflage fabrics.

The recent computing performance revolution has driven improvements in sensor, communication, and storage technology. Multi-decadal remote sensing datasets at the petabyte scale are now available in commercial clouds, with new satellite constellations generating petabytes/year of daily high-resolution global coverage imagery. Cloud computing and storage, combined with recent advances in machine learning, are enabling understanding of the world at a scale and at a level of detail never before feasible. We present results from an ongoing effort to develop satellite imagery analysis tools that aggregate temporal, spatial, and spectral information and that can scale with the high-rate and dimensionality of imagery being collected. We focus on the problem of monitoring food crop productivity across the Middle East and North Africa, and show how an analysis-ready, multi-sensor data platform enables quick prototyping of satellite imagery analysis algorithms, from land use/land cover classification and natural resource mapping, to yearly and monthly vegetative health change trends at the structural field level.