PurposeRecently, diffusion posterior sampling (DPS), where score-based diffusion priors are combined with likelihood models, has been used to produce high-quality computed tomography (CT) images given low-quality measurements. This technique permits one-time, unsupervised training of a CT prior, which can then be incorporated with an arbitrary data model. However, current methods rely on a linear model of X-ray CT physics to reconstruct. Although it is common to linearize the transmission tomography reconstruction problem, this is an approximation to the true and inherently nonlinear forward model. We propose a DPS method that integrates a general nonlinear measurement model.ApproachWe implement a traditional unconditional diffusion model by training a prior score function estimator and apply Bayes’ rule to combine this prior with a measurement likelihood score function derived from the nonlinear physical model to arrive at a posterior score function that can be used to sample the reverse-time diffusion process. We develop computational enhancements for the approach and evaluate the reconstruction approach in several simulation studies.ResultsThe proposed nonlinear DPS provides improved performance over traditional reconstruction methods and DPS with a linear model. Moreover, as compared with a conditionally trained deep learning approach, the nonlinear DPS approach shows a better ability to provide high-quality images for different acquisition protocols.ConclusionThis plug-and-play method allows the incorporation of a diffusion-based prior with a general nonlinear CT measurement model. This permits the application of the approach to different systems, protocols, etc., without the need for any additional training.
Diffusion models have been demonstrated as powerful deep learning tools for image generation in CT reconstruction and restoration. Recently, diffusion posterior sampling, where a score-based diffusion prior is combined with a likelihood model, has been used to produce high quality CT images given low-quality measurements. This technique is attractive since it permits a one-time, unsupervised training of a CT prior; which can then be incorporated with an arbitrary data model. However, current methods only rely on a linear model of x-ray CT physics to reconstruct or restore images. While it is common to linearize the transmission tomography reconstruction problem, this is an approximation to the true and inherently nonlinear forward model. We propose a new method that solves the inverse problem of nonlinear CT image reconstruction via diffusion posterior sampling. We implement a traditional unconditional diffusion model by training a prior score function estimator, and apply Bayes rule to combine this prior with a measurement likelihood score function derived from the nonlinear physical model to arrive at a posterior score function that can be used to sample the reverse-time diffusion process. This plug-and-play method allows incorporation of a diffusion-based prior with generalized nonlinear CT image reconstruction into multiple CT system designs with different forward models, without the need for any additional training. We demonstrate the technique in both fully sampled low dose data and sparse-view geometries using a single unsupervised training of the prior.
Spectral radiography and fluoroscopy with multi-layer flat-panel detectors (FPD) is being actively investigated in a range of clinical applications. For applications involving contrast administration, maximal contrast resolution is achieved when overlaying background anatomy is completely removed. This calls for three-material decomposition (soft tissue, bone, and contrast) enabled by measurements in three energy channels. We have previously demonstrated the feasibility of such decomposition using a triple-layer detector. While algorithmic solutions can be adopted to mitigate noise in the material basis images, in this work, we seek to fundamentally improve the conditioning of the problem through optimized system design. Design parameters include source voltage, the thickness of the top two CsI scintillators, and the thickness of two copper interstitial filters. The design objective is to minimize noise in the basis image containing contrast, chosen as gadolinium in this work to improve separation from bone. The optimized design was compared with other designs with unoptimized scintillator thickness and/or without interstitial filtration. Results show that CsI thickness optimization and interstitial filtration can significantly reduce noise in the gadolinium image by 35.7% and 42.7% respectively within a lung ROI, which in turn boosts detectability of small vessels. Gadolinium and bone signals are separated in all cases. Visualization of coronary vessels is enabled by the combining optimized system design and regularization. Results from this work demonstrate that three-material decomposition can be significantly improved with system design optimization. Optimized designs obtained from this work can inform imaging techniques selection and triple-layer detector fabrication for spectral radiography.
Optimization of CT image quality typically involves balancing variance and bias. In traditional filtered back-projection, this trade-off is controlled by the filter cutoff frequency. In model-based iterative reconstruction, the regularization strength parameter often serves the same function. Deep neural networks (DNNs) typically do not provide this tunable control over output image properties. Models are often trained to minimize the expected mean squared error, which penalizes both variance and bias in image outputs but does not offer any control over the trade-off between the two. We propose a method for controlling the output image properties of neural networks with a new loss function called weighted covariance and bias (WCB). Our proposed method uses multiple noise realizations of the input images during training to allow for separate weighting matrices for the variance and bias penalty terms. Moreover, we show that tuning these weights enables targeted penalization of specific image features with spatial frequency domain penalties. To evaluate our method, we present a simulation study using digital anthropomorphic phantoms, physical simulation of CT measurements, and image formation with various algorithms. We show that the WCB loss function offers a greater degree of control over trade-offs between variance and bias, whereas mean-squared error provides only one specific image quality configuration. We also show that WCB can be used to control specific image properties including variance, bias, spatial resolution, and the noise correlation of neural network outputs. Finally, we present a method to optimize the proposed weights for a spiculated lung nodule shape discrimination task. Our results demonstrate this new image quality can control the image properties of DNN outputs and optimize image quality for task-specific performance.
Restoration of images contaminated by blur is an important processing tool across modalities including computed tomography where the blur induced by various system factors can be complex with dependencies on acquisition and reconstruction protocol, and even be patient-dependent. In many cases, such a blur can be modeled and predicted with high accuracy providing an important input to a classical deconvolution approach. While traditional deblurring methods tend to be highly noise magnifying, deep learning approaches have the potential to improve upon classic performance limits. However, most network architectures base their restoration on data inputs alone without knowledge of the system blur. In this work, we explore a deep learning approach that takes both image inputs as well as information that characterizes the system blur to combine modeling and deep learning approaches. We apply the approach to CT image restoration and compare with an image-only deep learning approach. We find that inclusion of the system blur model improves deblurring performance - suggesting the potential power of the combined modeling and deep learning technique.
Low-contrast lesions are difficult to detect in noisy low-dose CT images. Improving CT image quality for this detection task has the potential to improve diagnostic accuracy and patient outcomes. In this work, we use tunable neural networks for CT image restoration with a hyperparameter to control the variance/bias tradeoff. We use clinical images from a super-high-resolution normal-dose CT scan to synthesize low-contrast low-dose CT images for supervised training of deep learning CT reconstruction models. Those models are trained using with multiple noise realizations so that variance and bias can be penalized separately. We use a training loss function with one hyperparameter called the denoising level, which controls the variance/bias tradeoff. Finally, we evaluate the CT image quality to find the optimal denoising level for low-contrast lesion detectability. We evaluate performance using a shallow neural network model classification model to represent a suboptimal image observer. Our results indicate that the optimal networks for low-contrast lesion detectability are those that prioritize bias reduction rather than mean-squared error, which demonstrates the potential clinical benefit of our proposed tunable neural networks.
KEYWORDS: Neural networks, Denoising, Computed tomography, Complex systems, Algorithm development, Medical imaging, Visualization, Spherical lenses, Space operations, Signal to noise ratio
The rapid development of deep-learning methods in medical imaging has called for an analysis method suitable for non-linear and data-dependent algorithms. In this work, we investigate a local linearity analysis where a complex neural network can be represented as piecewise linear systems. We recognize that a large number of neural networks consists of alternating linear layers and rectified linear unit (ReLU) activations, and are therefore strictly piecewise linear. We investigated the extent of these locally linear regions by gradually adding perturbations to an operating point. For this work, we explored perturbations based on image features of interest, including lesion contrast, background, and additive noise. We then developed strategies to extend these strictly locally linear regions to include neighboring linear regions with similar gradients. Using these approximately linear regions, we applied singular value decomposition (SVD) analysis to each local linear system to investigate and explain the overall nonlinear and data-dependent behaviors of neural networks. The analysis was applied to an example CT denoising algorithm trained on thorax CT scans. We observed that the strictly local linear regions are highly sensitive to small signal perturbations. Over a range of lesion contrast from 0.007 to 0.04 mm−1, there is a total of 33992 linear regions. The Jacobians are also shift-variant. However, the Jacobians of neighboring linear regions are very similar. By combining linear regions with similar Jacobians, we narrowed down the number of approximately linear regions to four over lesion contrast from 0.001 to 0.08 mm−1. The SVD analysis to different linear regions revealed denoising behavior that is highly dependent on the background intensity. Analysis further identified greater amount of noise reduction in uniform regions compared to lesion edges. In summary, the local linearity analysis framework we proposed has the potential for us to better characterize and interpret the non-linear and data-dependent behaviors of neural networks.
Optimization of CT image quality typically involves balancing noise and bias. In filtered back-projection, this trade-off is controlled by the particular filter and cutoff frequency. In penalized-likelihood iterative reconstruction, the penalty weight serves the same function. Deep neural networks typically do not provide this tuneable control over output image properties. Models are often trained to minimize mean squared error which penalizes both variance and bias in image outputs but does not offer any control over the trade-off between the two. In this work, we propose a method for controlling the output image properties of neural networks with a new loss function called weighted covariance and bias (WCB). Our proposed method includes separate weighting parameters to control the relative importance of noise or bias reduction. Moreover, we show that tuning these weights enables targeted penalization of specific image features (e.g. spatial frequencies). To evaluate our method, we present a simulation study using digital anthropormorphic phantoms, physical simulation of non-ideal CT data, and image formation with various algorithms. We show that WCB offers a greater degree of control over trade-offs between variance and bias whereas MSE has only one configuration. We also show that WCB can be used to control specific image properties including variance, bias, spatial resolution, and the noise correlation of neural network outputs. Finally, we present a method to optimize the proposed weights for stimulus detectability. Our results demonstrate the potential for this new capability to control the image properties of DNN outputs and optimize image quality for the task-specific applications.
The proliferation of deep learning image processing calls for a quantitative image quality assessment framework that is suitable for nonlinear, data-dependent algorithms. In this work, we propose a method to systematically evaluate the system and noise responses such that the nonlinear transfer properties can be mapped out. The method involves sampling of lesion perturbations as a function of size, contrast, as well as clinically relevant features such as shape and texture that may be important for diagnosis. We embed the perturbations in backgrounds of varying attenuation levels, noise magnitude and correlation that are associated with different patient anatomies and imaging protocols. The range of system and noise response are further used to evaluate performance for clinical tasks such as signal detection and classification. We performed the assessment for an example CNN-denoising algorithm for low does lung CT screening. The system response of the CNN-denoising algorithm exhibits highly nonlinear behavior where both contrast and higher order lesion features such as spiculated boundaries are not reliably represented for lesions perturbations with small size and low contrast. The noise properties are potentially highly nonstationary, and should be assumed to be the same between the signal-present and signal-absent images. Furthermore, we observer a high degree dependency of both system and noise response on the background attenuation levels. Inputs around zeros are effectively imposed a non-negativity constraint; transfer properties for higher background levels are highly variable. For a detection task, CNN-denoised images improved detectability index by 16-18% compared to low dose CT inputs. For classification task between spiculated and smooth lesions, CNN-denoised images result in a much larger improvement up to 50%. The performance assessment framework propose in this work can systematically map out the nonlinear transfer functions for deep learning algorithms and can potentially enable robust deployment of such algorithms in medical imaging applications.
Recent years have seen the increasing application of deep learning methods in medical imaging formation, processing, and analysis. These methods take advantage of the flexibility of nonlinear neural network models to encode information and features in ways that can outperform conventional approaches. However, because of the nonlinear nature of this processing, images formed by neural networks have properties that are highly datadependent and difficult to analyze. In particular, the generalizability and robustness of these approaches can be difficult to ascertain. In this work, we analyze a class of neural networks that use only piece-wise linear activation functions. This class of networks can be represented by locally linear systems where the linear properties are highly data-dependent - allowing, for example, estimation of noise in image output via standard propagation methods. We propose a nonlinearity index metric that quantifies the fidelity of a local linear approximation of trained models based on specific input data. We applied this analysis to three example CT denoising CNNs to analytically predict the noise properties in the output images. We found that the proposed nonlinearity metric highly correlates with the accuracy of noise predictions. The analysis proposed in this work provides theoretical understanding of the nonlinear behavior of neural networks and enables performance prediction and quantitation under certain conditions.
Manifold learning using deep neural networks been shown to be an effective tool for noise reduction in low-dose CT. We propose a new iterative CT reconstruction algorithm, called Manifold Reconstruction of Differences (MRoD), which combines physical models with a data-driven prior. The MRoD algorithm involves estimating a manifold component, approximating common features among all patients, and the difference component which has the freedom to fit the measured data. While the manifold captures typical patient features (e.g. healthy anatomy), the difference image highlights patient-specific elements (e.g. pathology). We present a framework which combines trained manifold-based modules with physical modules. We present a simulation study using anthropomorphic lung data showing that the MRoD algorithm can both isolate differences between a particular patient and the typical distribution, but also provide significant noise reduction with relatively low bias.
Quantitative estimation of contrast agent concentration is made possible by spectral CT and material decomposition. There are several approaches to modulate the sensitivity of the imaging system to obtain the different spectral channels required for decomposition. Spectral CT technologies that enable this varied sensitivity include source kV-switching, dual-layer detectors, and source-side filtering (e.g., tiled spatial-spectral filters). In this work, we use an advanced physical model to simulate these three spectral CT strategies as well as hybrid acquisitions using all combinations of two or three strategies. We apply model-based material decomposition to a water-iodine phantom with iodine concentrations from 0.1 to 5.0 mg/mL. We present bias-noise plots for the different combinations of spectral techniques and show that combined approaches permit diversity in spectral sensitivity and improve low concentration imaging performance relative to the those strategies applied individually. Better ability to estimate low concentrations of contrast agent has the potential to reduce risks associated with contrast administration (by lowering dosage) or to extend imaging applications into targets with much lower uptake.
Model-based material decomposition (MBMD) directly estimates the material densities from the spectral CT data and has found opportunities for dose reduction via physical and statistical modeling and advanced regularization. However, image properties of material basis volumes can be complex. For example, spatial resolution, noise, and cross-talk can depend on acquisition parameters, regularization, patient size, and anatomical target. In this work, we propose a set of prospective prediction tools for the generalized local impulse response (LIR) that characterizes both in-basis spatial resolution and cross-basis response, as well as noise correlation. The accuracy of noise predictor was validated in a simulation study, comparing predicted and measured in- and cross-basis noise correlations. Employing these predictors, we composed a specialized regularization for cross-talk reduction and showed that such prediction tools are promising for task-based optimization in spectral CT applications.
In this work, we present a novel model-based material decomposition (MBMD) approach for x-ray CT that includes system blur in the measurement model. Such processing has the potential to extend spatial resolution in material density estimates - particularly in systems where different spectral channels exhibit different spatial resolutions. We illustrate this new approach for a dual-layer detector x-ray CT and compare MBMD algorithms with and without blur in the reconstruction forward model. Both qualitative and quantitative comparisons of performance with and without blur modeling are reported. We find that blur modeling yields images with better recovery of high-resolution structures in an investigation of reconstructed line pairs as well as lower cross-talk bias between material bases that is ordinarily found due to mismatches in spatial resolution between spectral channels. The extended spatial resolution of the material decompositions has potential application in a range of high-resolution clinical tasks and spectral CT systems where spectral channels exhibit different spatial resolutions.
Spectral CT is an emerging modality that permits material decomposition and density estimation through the use of energy-dependent information in measurements. Direct model-based material decomposition algorithms have been developed that incorporate statistical models and advanced regularization schemes to improve density estimates and lower exposure requirements. However, understanding and control of the relationship between regularization and image properties is complex with interactions between spectral channels and material bases. In particular, regularization in one material basis can affect the image properties of other material bases, and vice versa. In this work, we derived a closed-form set of local impulse responses for the solutions to a general, regularized, model-based material decomposition (MBMD) objective. These predictors quantify both the spatial resolution in each material image as well as the influence of regularization of one material basis on other material images. This information can be used prospectively to tune regularization parameters for specific imaging goals.
Spectral CT is an emerging modality that uses a data acquisition scheme with varied spectral responses to provide enhanced material discrimination in addition to the structural information of conventional CT. Existing clinical and preclinical designs with this capability include kV-switching, split-filtration, and dual-layer detector systems to provide two spectral channels of projection data. In this work, we examine an alternate design based on a spatialspectral filter. This source-side filter is made up a linear array of materials that divide the incident x-ray beam into spectrally varied beamlets. This design allows for any number of spectral channels; however, each individual channel is sparse in the projection domain. Model-based iterative reconstruction methods can accommodate such sparse spatialspectral sampling patterns and allow for the incorporation of advanced regularization. With the goal of an optimized physical design, we characterize the effects of design parameters including filter tile order and filter tile width and their impact on material decomposition performance. We present results of numerical simulations that characterize the impact of each design parameter using a realistic CT geometry and noise model to demonstrate feasibility. Results for filter tile order show little change indicating that filter order is a low-priority design consideration. We observe improved performance for narrower filter widths; however, the performance drop-off is relatively flat indicating that wider filter widths are also feasible designs.
Material decomposition for imaging multiple contrast agents in a single acquisition has been made possible by spectral CT: a modality which incorporates multiple photon energy spectral sensitivities into a single data collection. This work presents an investigation of a new approach to spectral CT which does not rely on energy-discriminating detectors or multiple x-ray sources. Instead, a tiled pattern of K-edge filters are placed in front of the x-ray to create spatially encoded spectra data. For improved sampling, the spatial-spectral filter is moved continuously with respect to the source. A model-based material decomposition algorithm is adopted to directly reconstruct multiple material densities from projection data that is sparse in each spectral channel. Physical effects associated with the x-ray focal spot size and motion blur for the moving filter are expected to impact overall performance. In this work, those physical effects are modeled and a performance analysis is conducted. Specifically, experiments are presented with simulated focal spot widths between 0.2 mm and 4.0 mm. Additionally, filter motion blur is simulated for a linear translation speeds between 50 mm/s and 450 mm/s. The performance differential between a 0.2 mm and a 1.0 mm focal spot is less than 15% suggesting feasibility of the approach with realistic x-ray tubes. Moreover, for reasonable filter actuation speeds, higher speeds are shown to decrease error (due to improved sampling) despite motion-based spectral blur.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.