mirror of https://github.com/CGAL/cgal
143 lines
10 KiB
Plaintext
143 lines
10 KiB
Plaintext
namespace CGAL {
|
|
/*!
|
|
\mainpage Surface Mesh Segmentation
|
|
|
|
\author Ilker O. Yaz
|
|
|
|
\anchor elephant_sdf_partition
|
|
\image latex elephant_sdf_partition.png "Elephant model: SDF values & Segmentation" width=12cm
|
|
\image html elephant_sdf_partition.png "Elephant model: SDF values & Segmentation (Going to be changed-improved)"
|
|
|
|
# Introduction #
|
|
Mesh segmentation is the process of partitioning a mesh into smaller and meaningful sub-meshes. The application domain is wide and includes, but is not limited to modeling, rigging and texturing, shape-retrieval, and deformation.
|
|
|
|
This package provides an implementation of the algorithm presented in \cite shapira2008consistent. It relies on the Shape Diameter Function (SDF) which provides an estimate of the local volume diameter for each facet of the mesh.
|
|
|
|
The segmentation algorithm first computes SDF values for each facets and does a soft clustering of facets with similar SDF values.
|
|
These clusters are then refined using a graph-cut algorithm which also considers surface-based features such as dihedral-angle and concavity.
|
|
|
|
The API gives access to both SDF values and the segmentation of the mesh. That way an alternative implementation of the SDF can be directly plug into the segmentation algorithm.
|
|
|
|
# Overview of the Segmentation Process #
|
|
The segmentation algorithm consists of three major parts: Shape Diameter Function (SDF), soft clustering, and graph-cut for hard clustering.
|
|
|
|
## Shape Diameter Function ##
|
|
The Shape Diameter Function (SDF) provides a connection between the surface and its volume. More precisely, the SDF is a scalar-valued function defined on the surface which measures the corresponding local volume diameter.
|
|
The main handiness of the SDF is being able to distinguish thick and thin parts of the mesh by bringing in a volume-based feature to the surface. Another key feature of the SDF is its pose-invariant nature, which means that SDF values remains largely unaffected after changes of pose.
|
|
|
|
The SDF over a surface is computed by processing each facets one by one. For a given facet, the SDF value computation begins with casting several rays sampled from a cone which
|
|
is constructed using the centroid of the facet as apex and inward-normal of the facet as axis. Using these casted rays (which intuitively correspond to sampling for local volume),
|
|
the SDF value is calculated by first applying outlier removal and then taking weighted average of ray lengths.
|
|
|
|
After calculating SDF values for each facet, bilateral smoothing (an edge-preserving filtering technique) is applied. The purpose of edge-preserving smoothing is keeping SDF edges (fast changes on SDF values)
|
|
in-place without blurring, since they are good candidates for segment boundaries. (TODO: I don't get the last sentense) (change this sentence)
|
|
|
|
\image latex pose_changes_low.png "Effect of pose changes on segmentation" width=12cm
|
|
\image html pose_changes_low.png "Effect of pose changes on segmentation"
|
|
|
|
## Soft Clustering ##
|
|
The soft clustering on computed SDF values consists in first grouping facets using k-means clustering algorithm which is initialized with k-means++
|
|
(an algorithm for choosing random seeds for clusters) and run multiple times. Among these runs, we choose clustering result that has minimum with-in
|
|
cluster error and use it to initialize expectation maximization algorithm for fitting Gaussian mixture models.
|
|
|
|
Note that there is no direct relation between number of clusters (parameter for soft clustering) and number of segments (disconnected components). Intuitively, number of clusters represents level of segmentation by clustering facets, which have close SDF values, together without considering their connectivity. For example, for a human model, 2 clusters might result in clustering 4 legs and head of the model into one cluster, and remaining parts into another cluster (TODO: use the elephant or add an example showing that). However, large number of clusters likely to result in detailed segmentation of the mesh with large number of segments.
|
|
|
|
The output of this step is a matrix that contains probability values for each facet to belong to each cluster. These probability values are used as input in the graph-cut step than follows.
|
|
|
|
## Graph-Cut ##
|
|
The final hard clustering, which gives the final partitioning of the mesh, is obtained by minimizing an energy function. This energy function combines the aforementionned probability matrix and geometric surface features.
|
|
|
|
The expression of energy function that is minimized using alpha-expansion graph cut algorithm is the following:
|
|
|
|
<table border="0">
|
|
<tr>
|
|
<td>
|
|
\f$ E(\bar{x}) = \sum\limits_{f \in F} e_1(f, x_f) + \lambda \sum\limits_{ \{f,g\} \in N} e_2(x_f, x_g) \f$
|
|
|
|
\f$ e_1(f, x_f) = -log(max(P(f|x_f), \epsilon)) \f$
|
|
|
|
\f$ e_2(x_f, x_g) =
|
|
\left \{
|
|
\begin{array}{rl}
|
|
-log(\theta(f,g)/\pi) &\mbox{ $x_f \ne x_g$} \\
|
|
0 &\mbox{ $x_f = x_g$}
|
|
\end{array}
|
|
\right \} \f$
|
|
</td>
|
|
<td>
|
|
where:
|
|
- \f$F\f$ is the set of facets,
|
|
- \f$N\f$ is the set of pairs of neighbor facets,
|
|
- \f$x_f\f$ is the cluster assigned to facet \f$f\f$,
|
|
- \f$P(f|x_p)\f$ is the probability of assigning facet \f$f\f$ to cluster \f$x_p\f$,
|
|
- \f$\theta(f,g)\f$ is the dihedral angle between neighbor facets \f$f\f$, and \f$g\f$.
|
|
- \f$\epsilon\f$ is the minimal probability threshold
|
|
- \f$\lambda \in [0,1]\f$ is a smoothness parameter
|
|
</td>
|
|
</tr>
|
|
</table>
|
|
|
|
The first term of the energy function provides the contribution of the soft clustering probabilities.
|
|
The second term of the energy function is larger when two neighbor facets sharing a sharp edge are in the same cluster. The smoothness parameter can be used
|
|
to make this geometric criteria more or less prevalent.
|
|
|
|
# API #
|
|
This package provides three functions:
|
|
- CGAL::sdf_values_computation : computes SDF values as described in the original paper.
|
|
- CGAL::surface_mesh_segmentation_from_sdf_values : computes the mesh segmentation from SDF values.
|
|
- CGAL::surface_mesh_segmentation : computes SDF values and mesh segmentation in one go (i.e. combines two functions listed above).
|
|
|
|
##The SDF Computation##
|
|
The function CGAL::sdf_values_computation provides an implementation of the SDF computation for a given `CGAL Polyhedron`. After computation, the following post-processing steps are applied:
|
|
- Facets with no SDF values (i.e. zero) are assigned to average SDF value of its neighbors. If still there is any facet which has no SDF value, minimum SDF value greater than zero is assigned to it. (TODO: Does this appends in case there are holes in the mesh?)
|
|
- Smoothed with bilateral filtering.
|
|
- Linearly normalized between [0,1].
|
|
|
|
The outputs are a property map which associates every facet with its SDF value, and pair of minimum and maximum SDF values before linear normalization.
|
|
|
|
###Example####
|
|
\include Surface_mesh_segmentation/sdf_values_computation_example.cpp
|
|
|
|
##Surface Mesh Segmentation##
|
|
The function CGAL::surface_mesh_segmentation_from_sdf_values computes a segmentation of the mesh using provided SDF values.
|
|
The output is a property map which associates a segment-id (an integer between [0, number of segments -1]) to each facet. Formally, a segment is a set of connected facets which are placed under same cluster after graph-cut.
|
|
|
|
###Example###
|
|
\include Surface_mesh_segmentation/surface_mesh_segmentation_from_sdf_values_example.cpp
|
|
|
|
The function CGAL::surface_mesh_segmentation combines the two aforementionned functions. Note that for segmenting the mesh several times with different parameters (i.e. number of levels, and smoothing lambda),
|
|
it is wise to first compute SDF values using CGAL::sdf_values_computation, and then call CGAL::surface_mesh_segmentation_from_sdf_values with same SDF values.
|
|
|
|
###Example###
|
|
\include Surface_mesh_segmentation/surface_mesh_segmentation_example.cpp
|
|
|
|
##Using a polyhedron with an id per facet##
|
|
The previous examples use a `std::map` as propery maps for storing the sdf values and the segmentation results. This example uses
|
|
a polyhedron type with a facet type having an extra id field together with a vector as underlying data structure in the property maps.
|
|
The main advantage is to decrease the complexity of accessing associated data with facets from logaritmic to constant.
|
|
|
|
###Example###
|
|
\include Surface_mesh_segmentation/surface_mesh_segmentation_with_facet_ids_example.cpp
|
|
|
|
|
|
|
|
# Implementation history#
|
|
The initial implementation of this package is the result of the work of the author during the 2012 season
|
|
of the Google Summer of Code.
|
|
|
|
# Extra: (ideas to append) #
|
|
- Talk about effect of "number of rays" on SDF values, by providing RSME and average error ? (for now drop)
|
|
- Talk about effect of "number of levels" on segmentation results ? giving a few screen shot. (put a picture in TODO)
|
|
- Talk about pose-invariant feature of segmentation by giving a few screen shots from "benchmark" models (same models with different poses) ? (segmentation not sdf screen-shot)
|
|
- SL: I would take one example and use a Facet type with an id to avoid using a map (see BGL/examples/BGL_polyhedron_3/kruskal_with_stored_id.cpp for an example of Polyhedron instantiation) (use vector pmap)
|
|
- SL: should we offer a way to get sdf without normalization? or that least return the max of the sdf so that the sdf is the value in [0,1] x that number?
|
|
(return pairs)
|
|
- TODO: you do not talk about the limitation of the input mesh (triangulated, manifold, self-intersections, close, holes ...)
|
|
(is_pure_triangle precond in ref manual & in user manual talk about triangulated possible with small holes etc)
|
|
- TODO: there is nothing about the traits class and operations used. Is AABB_tree the only thing you use?
|
|
|
|
--------------------------------------------------------------------------------------------------------------
|
|
- IOY: It would be nice if we can shorten "examples" code sections (actually I search around whether there is any functionality in doxygen for generating collapsable/expandable sections (so that we can collapse unrelated part by default like includes or polyhedron creation/read), but couldn't find one.)
|
|
- SL: I don't like the idea of removing include files as this hide what need to be included. To me the length of example is OK.
|
|
*/
|
|
} /* namespace CGAL */ |