KEYWORDS: Image segmentation, Positron emission tomography, Tumors, Medical imaging, 3D modeling, 3D image processing, Machine learning, Neck, Head, Data modeling
The effort involved in creating accurate ground truth segmentation maps hinders advances in machine learning approaches to tumor delineation in clinical positron emission tomography (PET) scans. To address this challenge, we propose a fully convolutional network (FCN) model to delineate tumor volumes from PET scans automatically while relying on weak annotations in the form of bounding boxes (without delineations) around tumor lesions. To achieve this, we propose a novel loss function that dynamically combines a supervised component, designed to leverage the training bounding boxes, with an unsupervised component, inspired by the Mumford-Shah piecewise constant level-set image segmentation model. The model is trained end-to-end with the proposed differentiable loss function and is validated on a public clinical PET dataset of head and neck tumors. Using only bounding box annotations as supervision, the model achieves competitive results with state-of-the-art supervised and semiautomatic segmentation approaches. Our proposed approach improves the Dice similarity by approximately 30% and reduces the unsigned distance error by approximately 7 mm compared to a model trained with only bounding boxes (weak supervision). Also, after the post-processing step (morphological operations), our weak supervision approach differs only 7% in terms of the Dice similarity from the quality of the fully supervised model, for segmentation task.
This paper explores a novel approach to interactive user-guided image segmentation, using eyegaze information
as an input. The method includes three steps: 1) eyegaze tracking for providing user input, such as setting
object and background seed pixel selection; 2) an optimization method for image labeling that is constrained
or affected by user input; and 3) linking the two previous steps via a graphical user interface for displaying the
images and other controls to the user and for providing real-time visual feedback of eyegaze and seed locations,
thus enabling the interactive segmentation procedure. We developed a new graphical user interface supported
by an eyegaze tracking monitor to capture the user's eyegaze movement and fixations (as opposed to traditional
mouse moving and clicking). The user simply looks at different parts of the screen to select which image to
segment, to perform foreground and background seed placement and to set optional segmentation parameters.
There is an eyegaze-controlled "zoom" feature for difficult images containing objects with narrow parts, holes
or weak boundaries. The image is then segmented using the random walker image segmentation method. We
performed a pilot study with 7 subjects who segmented synthetic, natural and real medical images. Our results
show that getting used the new interface takes about only 5 minutes. Compared with traditional mouse-based
control, the new eyegaze approach provided a 18.6% speed improvement for more than 90% of images with high
object-background contrast. However, for low contrast and more difficult images it took longer to place seeds
using the eyegaze-based "zoom" to relax the required eyegaze accuracy of seed placement.
In this paper, we propose an adaptive seeding strategy for visualization of diffusion tensor magnetic resonance
imaging (DT-MRI) data using streamtubes. DT-MRI is a medical imaging modality that captures unique water
diffusion properties and fiber orientation information of the imaged tissues. Visualizing DT-MRI data using
streamtubes has the advantage that not only the anisotropic nature of the diffusion is visualized but also the
underlying anatomy of biological structures is revealed. This makes streamtubes significant for the analysis of
fibrous tissues in medical images. In order to avoid rendering multiple similar streamtubes, an adaptive seeding
strategy is employed which takes into account similarity of tensors in a given region. The goal is to automate
the process of generating seed points such that regions with dissimilar tensors are assigned more seed points
compared to regions with similar tensors. The algorithm is based on tensor dissimilarity metrics that take into
account both diffusion magnitudes and directions to optimize the seeding positions and density of streamtubes
in order to reduce the visual clutter. Two recent advances in tensor calculus and tensor dissimilarity metrics
are utilized: the Log-Euclidean and the J-divergence. Results show that adaptive seeding not only helps to cull
unnecessary streamtubes that would obscure visualization but also do so without having to compute the culled
streamtubes, which makes the visualization process faster.
KEYWORDS: Image segmentation, 3D image processing, Medical imaging, Image processing algorithms and systems, 3D image enhancement, Image quality, Binary data, Signal to noise ratio, Magnetic resonance imaging, Visualization
Segmentation of 3D data is one of the most challenging tasks in medical image analysis. While reliable automatic
methods are typically preferred, their success is often hindered by poor image quality and significant
variations in anatomy. Recent years have thus seen an increasing interest in the development of semi-automated
segmentation methods that combine computational tools with intuitive, minimal user interaction. In an earlier
work, we introduced a highly-automated technique for medical image segmentation, where a 3D extension of the
traditional 2D Livewire was proposed. In this paper, we present an enhanced and more powerful 3D Livewire-based
segmentation approach with new features designed to primarily enable the handling of complex object
topologies that are common in biological structures. The point ordering algorithm we proposed earlier, which
automatically pairs up seedpoints in 3D, is improved in this work such that multiple sets of points are allowed
to simultaneously exist. Point sets can now be automatically merged and split to accommodate for the presence
of concavities, protrusions, and non-spherical topologies. The robustness of the method is further improved by
extending the 'turtle algorithm', presented earlier, by using a turtle-path pruning step. Tests on both synthetic
and real medical images demonstrate the efficiency, reproducibility, accuracy, and robustness of the proposed
approach. Among the examples illustrated is the segmentation of the left and right ventricles from a T1-weighted
MRI scan, where an average task time reduction of 84.7% was achieved when compared to a user performing 2D
Livewire segmentation on every slice.
An important problem in medical image analysis is the segmentation of anatomical regions of interest. Once
regions of interest are segmented, one can extract shape, appearance, and structural features that can be analyzed
for disease diagnosis or treatment evaluation. Diffusion tensor magnetic resonance imaging (DT-MRI) is
a relatively new medical imaging modality that captures unique water diffusion properties and fiber orientation
information of the imaged tissues. In this paper, we extend the interactive multidimensional graph cuts segmentation
technique to operate on DT-MRI data by utilizing latest advances in tensor calculus and diffusion tensor
dissimilarity metrics. The user interactively selects certain tensors as object ("obj") or background ("bkg") to
provide hard constraints for the segmentation. Additional soft constraints incorporate information about both
regional tissue diffusion as well as boundaries between tissues of different diffusion properties. Graph cuts are
used to find globally optimal segmentation of the underlying 3D DT-MR image among all segmentations satisfying
the constraints. We develop a graph structure from the underlying DT-MR image with the tensor voxels
corresponding to the graph vertices and with graph edge weights computed using either Log-Euclidean or the
J-divergence tensor dissimilarity metric. The topology of our segmentation is unrestricted and both obj and bkg
segments may consist of several isolated parts. We test our method on synthetic DT data and apply it to real
2D and 3D MRI, providing segmentations of the corpus callosum in the brain and the ventricles of the heart.
The choice of 3D shape representation for anatomical structures determines the effectiveness with which segmentation,
visualization, deformation, and shape statistics are performed. Medial axis-based shape representations
have attracted considerable attention due to their inherent ability to encode information about the natural
geometry of parts of the anatomy. In this paper, we propose a novel approach, based on nonlinear manifold
learning, to the parameterization of medial sheets and object surfaces based on the results of skeletonization.
For each single-sheet figure in an anatomical structure, we skeletonize the figure, and classify its surface points
according to whether they lie on the upper or lower surface, based on their relationship to the skeleton points.
We then perform nonlinear dimensionality reduction on the skeleton, upper, and lower surface points, to find
the intrinsic 2D coordinate system of each. We then center a planar mesh over each of the low-dimensional
representations of the points, and map the meshes back to 3D using the mappings obtained by manifold learning.
Correspondence between mesh vertices, established in their intrinsic 2D coordinate spaces, is used in order
to compute the thickness vectors emanating from the medial sheet. We show results of our algorithm on real
brain and musculoskeletal structures extracted from MRI, as well as an artificial multi-sheet example. The main
advantages to this method are its relative simplicity and noniterative nature, and its ability to correctly compute
nonintersecting thickness vectors for a medial sheet regardless of both the amount of coincident bending and
thickness in the object, and of the incidence of local concavities and convexities in the object's surface.
To facilitate high level analysis of medical image data in research and clinical environments, a wrapper for the ITK toolkit is developed to allow ITK algorithms to be called in MATLAB. ITK is a powerful open-source toolkit implementing state-of-the-art algorithms in medical image processing and analysis. However, although ITK is rapidly gaining popularity, its user base is mostly restricted to technically savvy developers with expert knowledge of C++ and advanced programming concepts. MATLAB, on the other hand, is well-known for its easy-to-use, powerful prototyping capabilities that significantly improve productivity. Unfortunately, the 3D image processing capabilities of MATLAB are very limited and slow to execute. With the help of the wrapper we introduce in this paper, biomedical computing researchers familiar with MATLAB can harness the power of ITK while avoiding learning C++ and dealing with low-level programming issues. We strongly believe this functionality will be of considerable interest to the medical image computing community. In this paper we provide details about the design and usage of this interface in medical image filtering, segmentation, and registration.
Bicipital root and proximal tendon disorders are an important symptom generator in the shoulder. The accuracy of the diagnosis of many shoulder disorders visually without quantitative shape analysis is limited, motivating a clinical need for some ancillary method to access the proximal biceps. Because of the known inter-relationship of the bicipital groove (BG) with several types of disorders, we propose an approach to the 3D shape description of the BG that captures information relevant to disorders of the shoulder (e.g. width, depth, angles of walls, presence of spurs). Our approach is medial-axis based and captures intuitive aspects of shape such as thickness, bending, and elongation. Our proposed method overcomes the well-known problem of boundary sensitivity in the medial representation as it is applied to representation and analysis of BG shape. We give preliminary quantitative results indicating that this representation does capture shape variation within our experimental data, providing motivation to explore more sophisticated statistical analysis based on this representation in future work. We also provide a method for semi-automatic segmentation of the BG from computed tomography (CT) scans of the shoulder; an important precursor step to BG shape analysis.
Recent advances in medical research hypothesize that certain body
fat, in addition to having a classical role of energy storage, may
also have mechanical function. In particular, we analyzed the
infrapatellar fat pad of Hoffa using 3D CT images of the knee at
multiple angles to determine how the fat pad changes shape as the
knee bends and whether the fat pad provides cushioning in the knee
joint. The images were initially processed using a median filter
then segmented using a region growing technique to isolate the fat
pad from the rest of the knee. Next, rigid registration was
performed to align the series of images to match the reference
image. Finally, multi-resolution FEM registration was completed
between the aligned images. The resulting displacements fields
were used to determine the local volume change of the fat pad as
the knee bends from extension to flexion through different angles.
This multi-angle analysis provides a finer description of the
intermediate deformations compared to earlier work, where only a
pair of images (full extension and flexion) was analyzed.
The boundaries of oral lesions in color images were detected using a live-wire method and compared to expert delineations. Multiple cost terms were analyzed for their inclusion in the final total cost function including color gradient magnitude, color gradient direction, Canny edge detection, and Laplacian zero crossing. The gradient magnitude and direction cost terms were implemented so that they acted directly on the three components of the color image, instead of using a single derived color band. The live-wire program was shown to be considerably more accurate and faster compared to manual segmentations by untrained users.
Previously, "Deformable organisms" were introduced as a novel paradigm for medical image analysis that uses artificial life modelling concepts. Deformable organisms were designed to complement the classical bottom-up deformable models methodologies (geometrical and physical layers), with top-down intelligent deformation control mechanisms (behavioral and cognitive layers). However, a true physical layer was absent and in order to complete medical image segmentation tasks, deformable organisms relied on pure geometry-based shape deformations guided by sensory data, prior structural knowledge, and expert-generated schedules of behaviors. In this paper we introduce the use of physics-based shape deformations within the deformable organisms framework yielding additional robustness by allowing intuitive real-time user guidance and interaction when necessary. We present the results of applying our physics-based deformable organisms, with an underlying dynamic spring-mass mesh model, to segmenting and labelling the corpus callosum in 2D midsagittal magnetic resonance images.
KEYWORDS: Image segmentation, 3D image processing, Medical imaging, Brain, 3D displays, Image processing algorithms and systems, Kidney, Visualization, Edge detection, Detection and tracking algorithms
Segmenting anatomical structures from medical images is usually one of the most important initial steps in many applications, including visualization, computer-aided diagnosis, and morphometric analysis. Manual 2D segmentation suffers from operator variability and is tedious and time-consuming. These disadvantages are accentuated in 3D applications and, the additional requirement of producing intuitive displays to integrate 3D information for the user, makes manual segmentation even less approachable in 3D. Robust, automatic medical image segmentation in 2D to 3D remains an open problem caused particularly by sensitivity to low-level parameters of segmentation algorithms. Semi-automatic techniques present possible balanced solution where automation focuses on low-level computing-intensive tasks that can be hidden from the user, while manual inter-
vention captures high-level expert knowledge nontrivial to capture algorithmically. In this paper we present a 3D extension to the 2D semi-automatic live-wire technique. Live-wire based contours generated semi-automatically on a selected set of slices are used as seed points on new unseen slices in different orientations. The seed points are calculated from intersections of user-based live-wire techniques with new slices. Our algorithm includes a step for ordering the live-wire seed points in the new slices, which is essential for subsequent multi-stage optimal path calculation. We present results of automatically detecting contours in new slices in 3D volumes from a variety of medical images.
KEYWORDS: Image registration, Image segmentation, Computed tomography, Finite element methods, 3D modeling, Magnetic resonance imaging, 3D image processing, Medical imaging, Medicine, Shape analysis
Recent advances in medicine conjecture that certain body fat may have mechanical function in addition to its classical role of energy storage. In particular we aim to analyze if the intra-articular fat pad of Hoffa is merely a space holder or if it changes shape to provide cushioning for the knee bones. Towards this goal, 3D CT images of real knees, as well as a skeletal knee model with fat simulating Hoffa's pad, were acquired in both extension and flexion. Image segmentation was performed to automatically extract the real and simulated fat regions from the extension and flexion images. Utilizing the segmentation results as binary masks, we performed automatic multi-resolution image registration of the fat pad between flexed and extended knee positions. The resulting displacement fields from flexion-extension registration are examined and used to calculate local fat volume changes thus providing insight into shape changes that may have a mechanical component.
KEYWORDS: Lab on a chip, Principal component analysis, Shape analysis, Medical imaging, Image segmentation, Statistical modeling, Statistical analysis, Data modeling, Brain, Magnetic resonance imaging
Powerful, flexible shape models of anatomical structures are required for robust, automatic analysis of medical images. In this paper we investigate a physics-based shape representation and deformation method in an effort to meet these requirements. Using a medial-based spring-mass mesh model, shape deformations are produced via the application of external forces or internal spring actuation. The range of deformations includes bulging, stretching, bending, and tapering at different locations, scales, and with varying amplitudes. Springs are actuated either by applying deformation operators or by activating statistical modes of variation obtained via a hierarchical regional principal component analysis. We demonstrate results on both synthetic data and on a spring-mass model of the corpus callosum, obtained from 2D mid-sagittal brain Magnetic Resonance (MR) Images.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.