Problem definition: Our previous segmentation framework is based on learning methods. Several landmarks are detected, and a reference shape is transformed (either similarity or affine) to fit landmarks. This shape is used as an initialization. Then segmentation is performed using boundary detections and constraints of smoothness. It is similar to a combination of Snakes[1] and boosted cascade method[2], and achieves very good performance in many segmentation applications[3]. However, there is no shape constraints from training data (like Active Shape Model does[4]), which makes the system less stable and may result in undesirable results in some cases. Thus we added such constraints to improve the performance. There are some other limitations of the current framework. First the shape quality (especially in 3D) may be degenerated, which adversely affect the training performance of landmark and boundary detections. Second, the global transformation may not exactly fit the landmark positions. We also tried to solve these problems.
Geometry improvements: Implemented a Python script with C++ wrappers which supports mesh decimation, smoothing, isotropic remesher and local deformation (TPS or LSE). The script also relies on VTK, OpenMesh and OpenFlipper. After optimizing the mesh quality using this script, the training and segmentation performance is improved. The global transformation can be replaced by the local deformation method, when landmark detections are generally correct. Using local deformation can fit landmark positions better.
ASM shape constraint: Implemented a Python script of 3D Active Shape Model (the pose and shape estimation part, not the boundary detection part). This constrain certainly brings in stableness. However, the principal modes do not have large variations in our application (8 modes are selected to cover 90% variation). Thus the modes are not representative enough for shape changing in our 3D cases.
Sparse representation based shape constraints: proposed a framework using sparse representations to discover desired shapes and remove outliers (anomaly detection), and implemented this idea with Python (use CVXMOD as the solver). This method is based on two assumptions: 1) the input shape can be approximately represented by a sparse linear combination of shapes from training data; 2) the input shape may contain errors or outliers, but such outliers are sparse. Our framework can jointly recover the "clean" version of the input shape and detect the outliers.
Other work: I also implemented some other tools, such as (1) a semi-automatic annotation tool (using Matlab) for X-Ray image; (2) a visualization tool (using C++) which shows volume data with multiple surface meshes; (3) a visualization tool which loads a sequence of meshes and displays them like animation. Furthermore, I also participated in the project of auto align system using adaptive multi-structural atlas construction (with Ting Chen).
Patents
• Shaoting Zhang, Yiqiang Zhan, Maneesh Dewan, Ting Chen and Xiang Zhou: System and Methods for Robustly Learning Shape Statistics Using Sparse Representation, invention disclosure, Docket Number: 2010E20120 US.• Ting Chen, Maneesh Dewan, Yiqiang Zhan, Shaoting Zhang and Xiang Zhou: Adaptive Multi-Structural Atlas Construction for Auto Align System, invention disclosure, Docket Number: 2010E17098 US.
2D Lung Segmentation from X-Ray
 |
Figure 1.
Demonstration of anomaly detection. There are two detection errors, which adversely
affect the performance of baseline system (Procrustes analysis) and Active Shape Model. |
 |
Figure 2.
Discover detail information. |
3D Liver Segmentation from Low-dose CT
 |
Figure 3.
Mesh partitioning using affinity propagation. This local shape setting
can learn detail information better, and is more efficient for the optimization solver. |
 |
Figure 4.
3D initialization based on landmark detection.
Comparison between the baseline system (Procrustes analysis) and sparse representation. |
 |
Figure 5.
3D deformation based on boundary detection.
Comparison between the baseline system (Procrustes analysis), Active Shape Model and sparse representation. |
References
[1]. M. Kass, A. Witkin and D. Terzopoulos, Snakes: Active contour models, IJCV 1988.
[2]. P. Viola and M. Jones, Rapid Object Detection Using a Boosted Cascade of Simple Features, CVPR 2001.
[3]. Y. Zhan, M. Dewan, X. Zhou: Cross Modality Deformable Segmentation Using Hierarchical Clustering and Learning. MICCAI 2009
[4]. T. Cootes, C. Taylor, D. Cooper, and J. Graham. Active shape model - their training and application. CVIU, 1995.
Other Tools -- Visualization Interface
 |
Figure 6.
Visualization tools. It is written using C++, VTK, ITK and QT.
It displays volume data with surface meshes. |
Other Tools -- 2D Annotation
 |
Figure 7.
2D annotation tools. It is written using Matlab. |