diff --git a/Surface_mesh_segmentation/doc/Doxyfile b/Surface_mesh_segmentation/doc/Doxyfile index efce313fb8c..bda98a8eb3a 100644 --- a/Surface_mesh_segmentation/doc/Doxyfile +++ b/Surface_mesh_segmentation/doc/Doxyfile @@ -190,7 +190,7 @@ SERVER_BASED_SEARCH = NO #--------------------------------------------------------------------------- # configuration options related to the LaTeX output #--------------------------------------------------------------------------- -GENERATE_LATEX = NO +GENERATE_LATEX = YES LATEX_OUTPUT = latex LATEX_CMD_NAME = latex MAKEINDEX_CMD_NAME = makeindex diff --git a/Surface_mesh_segmentation/doc/Surface_Mesh_Segmentation.txt b/Surface_mesh_segmentation/doc/Surface_Mesh_Segmentation.txt index 1d0cf0a8d46..056dcceee5c 100644 --- a/Surface_mesh_segmentation/doc/Surface_Mesh_Segmentation.txt +++ b/Surface_mesh_segmentation/doc/Surface_Mesh_Segmentation.txt @@ -5,64 +5,83 @@ namespace CGAL { \author Ilker O. Yaz \anchor Figure-1 -\image latex elephant_sdf_partition.png "Figure 1: Elephant model with SDF values & Segmentation." width=12cm +\image latex elephant_sdf_partition.png "Elephant model with SDF values & Segmentation." width=12cm \image html elephant_sdf_partition.png "Figure 1: Elephant model with SDF values & Segmentation." # Introduction # Mesh segmentation is the process of partitioning a mesh into smaller and meaningful sub-meshes. The application domain is wide and includes, -but is not limited to modeling, rigging and texturing, shape-retrieval, and deformation. +but is not limited to modeling, rigging, texturing, shape-retrieval, and deformation. A detailed survey on mesh segmentation techniques can be found in \cite Shamir2008SegmentationSurvey. -This package provides an implementation of the algorithm presented in \cite shapira2008consistent. It relies on the Shape Diameter Function (SDF) which +This package provides an implementation of the algorithm presented in \cite Shapira2008Consistent. It relies on the Shape Diameter Function (SDF) which provides an estimate of the local volume diameter for each facet of the mesh. Given SDF values, the segmentation algorithm first applies soft clustering on facets. These clusters are then refined using a graph-cut algorithm which also considers surface-based features such as dihedral-angle and concavity. The API gives access to both the SDF computation and segmentation for a given triangulated mesh. -That way an alternative implementation of the SDF can be directly plug into the segmentation algorithm. +That way an alternative implementation of the SDF can be directly plugged into the segmentation algorithm. +Also same SDF values can be used multiple times as a parameter for the segmentation algorithm. + +Since the mesh segmentation problem is ill-posed, we also evaluate results of our implementation by using data set and evaluation software provided by \cite Chen2009SegmentationBenchmark, +and provide detailed results at the end of the manual. # Overview of the Segmentation Process # The segmentation algorithm consists of three major parts: Shape Diameter Function (SDF), soft clustering, and graph-cut for hard clustering. ## Shape Diameter Function ## The Shape Diameter Function (SDF) provides a connection between the surface and its volume. -More precisely, the SDF is a scalar-valued function defined on the surface which measures the corresponding local volume diameter. +More precisely, the SDF is a scalar-valued function defined on facets of the surface which measures the corresponding local volume diameter. The main handiness of the SDF is being able to distinguish thick and thin parts of the mesh by bringing in a volume-based feature to the surface. -Another key feature of the SDF is its pose-invariant nature, which means that SDF values remains largely unaffected after changes of pose \ref Figure-2. +Another key feature of the SDF is its pose-invariant nature, which means that SDF values remain largely unaffected after changes of pose \ref Figure-3. The SDF over a surface is computed by processing each facets one by one. For a given facet, the SDF value computation begins with casting several rays sampled from a cone which is constructed using the centroid of the facet as apex and inward-normal of the facet as axis. Using these casted rays (which intuitively correspond to a local volume sampling), -the SDF value is calculated by first applying outlier removal and then taking weighted average of ray lengths. - -After calculating SDF values for each facet, bilateral smoothing (an edge-preserving filtering technique) is applied. -The purpose of edge-preserving smoothing is removing noise while keeping fast changes on SDF values in-place without smoothing, -since they are natural candidates for segment boundaries. +the SDF value is calculated by first applying outlier removal and then taking average of ray lengths. \anchor Figure-2 -\image latex pose_changes_low.png "Figure 2: Effect of pose changes on segmentation." width=12cm -\image html pose_changes_low.png "Figure 2: Effect of pose changes on segmentation." +\image latex vogel_uniform_biased.png "Comparison of biased-to-center and uniform disk sampling for 64 rays." width=12cm +\image html vogel_uniform_biased.png "Figure 2: Comparison of biased-to-center and uniform disk sampling for 64 rays." + +We use sampling method from \cite Vogel1979Sampling to sample points from base of the cone, then combine them with apex of the cone as origin while constructing rays. +Our sampling method is biased to center in order to make sampling uniform to angle. +As a result, we do not use mentioned weighting schema in the paper in order to reduce the contributions of rays with larger angles. +A comparison with biased and uniform sampling of points can be seen in \ref Figure-2. The final SDF value of a facet is then calculated by averaging the ray lengths which fall into one Median Absolute Deviation (MAD) from the median of all lengths. + +After calculating SDF values for each facet, bilateral smoothing (an edge-preserving filtering technique) \cite Tomasi1998Bilateral is applied. +The purpose of edge-preserving smoothing is removing noise while keeping fast changes on SDF values in-place without smoothing, +since they are natural candidates for segment boundaries. +Window size \f$ w \f$, range parameter \f$ \sigma_r \f$, and spatial parameter \f$ \sigma_s \f$ are parameters for bilateral smoothing and determined as follows: +windows size is set to \f$ w = \sqrt{number\:of\:facet / 2000} + 1 \f$, +spatial parameter is half of the window size \f$ \sigma_s = w /2 \f$, +range parameter is determined for each facet by computing standard deviation of SDF values around the SDF value of current facet \f$ \sigma_{r_i} = \sqrt{1/|w_i|\sum_{f_j \in w_i}(SDF(f_j) - SDF(f_i))^2} \f$. + +\anchor Figure-3 +\image latex pose_changes_sdf_low_3.png "Effect of pose changes on segmentation and SDF values." width=12cm +\image html pose_changes_sdf_low_3.png "Figure 3: Effect of pose changes on segmentation and SDF values." ## Soft Clustering ## The soft clustering on computed SDF values first groups facets using k-means clustering algorithm which is initialized with k-means++ (an algorithm for choosing random seeds for clusters) and run multiple times. Among these runs, we choose clustering result that has minimum with-in cluster error and use it to initialize expectation maximization algorithm for fitting Gaussian mixture models. -Note that there is no direct relationship between the number of clusters (parameter for soft clustering) and the number of segments (disconnected components). +Note that there is no direct relationship between the number of clusters (parameter for soft clustering) and the number of segments (i.e. disconnected components / surface patches). Intuitively, the number of clusters represents the number of levels of a segmentation by clustering facets which have close SDF values without considering their connectivity. However, a large number of clusters is likely to result in detailed segmentation of the mesh -with a large number of segments \ref Figure-3. +with a large number of segments \ref Figure-4. The output of this step is a matrix that contains probability values for each facet to belong to each cluster. These probability values are used as input in the graph-cut step that follows. -\anchor Figure-3 -\image latex effect_of_levels.png "Figure 3: Effect of the number of clusters on the segmentation. Number of clusters were set to 4, 3, and 2 respectively." width=12cm -\image html effect_of_levels.png "Figure 3: Effect of the number of clusters on the segmentation. Number of clusters were set to 4, 3, and 2 respectively." +\anchor Figure-4 +\image latex effect_of_levels.png "Effect of different number of clusters on the segmentation. Number of clusters were set to 4, 3, and 2 respectively." width=12cm +\image html effect_of_levels.png "Figure 4: Effect of different number of clusters on the segmentation. Number of clusters were set to 4, 3, and 2 respectively." ## Graph-Cut ## The final hard clustering, which gives the final partitioning of the mesh, is obtained by minimizing an energy function. -This energy function combines the aforementioned probability matrix and geometric surface features. +This energy function combines the aforementioned probability matrix and geometric surface features. +The result of the algorithm is assigned clusters for each facet, +however we postprocess the result and produce a unique ID for each set of facets which are connected and placed under same cluster (i.e segments). -The expression of energy function that is minimized using alpha-expansion graph cut algorithm is the following: +The expression of energy function that is minimized using alpha-expansion graph cut algorithm \cite Boykov2001FastApproximate is the following: @@ -85,7 +104,8 @@ where: - \f$N\f$ is the set of pairs of neighbor facets, - \f$x_f\f$ is the cluster assigned to facet \f$f\f$, - \f$P(f|x_p)\f$ is the probability of assigning facet \f$f\f$ to cluster \f$x_p\f$, - - \f$\theta(f,g)\f$ is the dihedral angle between neighbor facets \f$f\f$, and \f$g\f$, + - \f$\theta(f,g)\f$ is the dihedral angle between neighbor facets \f$f\f$, and \f$g\f$, + concave angles and convex angles are weighted by 1 and 0.1 respectively, - \f$\epsilon\f$ is the minimal probability threshold, - \f$\lambda \in [0,1]\f$ is a smoothness parameter. @@ -93,57 +113,70 @@ where:
The first term of the energy function provides the contribution of the soft clustering probabilities. -The second term of the energy function is larger when two neighbor facets sharing a sharp edge are in the same cluster. The smoothness parameter can be used -to make this geometric criteria more or less prevalent. +The second term of the energy function is larger when two neighbor facets sharing a sharp and concave edge are in the same cluster. +The smoothness parameter can be used to make this geometric criteria more or less prevalent. + +Basically, assigning a high value to smoothness parameter results in small number of segments since constructing a segment boundary would be expensive. +Another words, merging facets which are placed under different clusters is less expensive than separating them and creating boundaries. +Similarly, assigning smaller values to smoothness parameter results in high number of segments by getting closer to the result of soft clustering +(notice that setting smoothness parameter to zero directly returns the result of soft clustering). Effect of different smoothness parameters is demonstrated in \ref Figure-5. + +\anchor Figure-5 +\image latex dino_different_lambda_small.png "Effect of different smoothness parameters on the segmentation (10 clusters). Smoothness parameters were set to 0.0, 0.1, 0.25, 0.5 and 1.0 respectively." width=16cm +\image html dino_different_lambda_small.png "Figure 5: Effect of different smoothness parameters on the segmentation (10 clusters). The smoothness parameter was set to 0.0, 0.1, 0.25, 0.5 and 1.0 respectively." + # API # This package provides three functions: - - CGAL::sdf_values_computation : computes SDF values as described in the original paper. - - CGAL::surface_mesh_segmentation_from_sdf_values : computes the mesh segmentation from SDF values. - - CGAL::surface_mesh_segmentation : computes SDF values and mesh segmentation in one go (i.e. combines two functions listed above). + - CGAL::compute_sdf_values : computes SDF values as described in the original paper. + - CGAL::segment_from_sdf_values : computes the mesh segmentation from SDF values. + - CGAL::compute_sdf_values_and_segment : computes SDF values and mesh segmentation in one go (i.e. combines two functions listed above). -These functions expects a manifold triangulated polyhedron without boundary as input. Note that the current implementation is working with +These functions expects a manifold, normal oriented, and triangulated polyhedron without boundary as input. Note that the current implementation is working with polyhedron with boundaries, but considering how the SDF value are computed, using a polyhedron with large holes is likely to result in meaningless SDF values, and therefore unreliable segmentation. The current implementation of the computation of the SDF values relies on the AABB_tree package. -This operation is robust when the AABBTraits model provides has exact predicates. +This operation is reliable when the AABBTraits model provided has exact predicates. ##The SDF Computation## -The function CGAL::sdf_values_computation provides an implementation of the SDF computation for a given `CGAL Polyhedron`. +The function CGAL::compute_sdf_values provides an implementation of the SDF computation for a given `CGAL Polyhedron`. After computation, the following post-processing steps are applied: - - Facets with no SDF values (i.e. zero) are assigned to average SDF value of its neighbors. If still there is any facet which has no SDF value, minimum SDF value greater than zero is assigned to it. + - Facets with no SDF values (i.e. zero) are assigned to average SDF value of its neighbors. If still there is any facet which has no SDF value, minimum SDF value greater than zero is assigned to it. Note that this step is not inherited from the paper. - Smoothed with bilateral filtering. - Linearly normalized between [0,1]. The outputs are a property map which associates every facet with its SDF value, and pair of minimum and maximum SDF values before linear normalization. ###Example#### -\include Surface_mesh_segmentation/sdf_values_computation_example.cpp +\include Surface_mesh_segmentation/compute_sdf_values_example.cpp ##Surface Mesh Segmentation## -The function CGAL::surface_mesh_segmentation_from_sdf_values computes a segmentation of the mesh using provided SDF values. +The function CGAL::segment_from_sdf_values computes a segmentation of the mesh using provided SDF values. +Note that these SDF values can be any set of scalar value associated with each facet as long as they are linearly normalized between [0, 1]. +This function also gives opportunity to use same SDF values multiple times with different parameters in segmentation stage. + The outputs are number of segments and a property map which associates a segment-id (an integer between [0, number of segments -1]) to each facet. -Formally, a segment is a set of connected facets which are placed under same cluster after graph-cut. +Formally, a segment is a set of connected facets which are placed under same cluster after graph-cut. Note that the number of clusters, which is the input parameter of the algorithm, and the number of partitions in the final segmentation (the actual output of the algorithm) are not equal. ###Example### -\include Surface_mesh_segmentation/surface_mesh_segmentation_from_sdf_values_example.cpp +\include Surface_mesh_segmentation/segment_from_sdf_values_example.cpp -The function CGAL::surface_mesh_segmentation combines the two aforementioned functions. +The function CGAL::compute_sdf_values_and_segment combines the two aforementioned functions. Note that for segmenting the mesh several times with different parameters (i.e. number of levels, and smoothing lambda), -it is wise to first compute the SDF values using CGAL::sdf_values_computation, and then call CGAL::surface_mesh_segmentation_from_sdf_values with the same SDF values. +it is wise to first compute the SDF values using CGAL::compute_sdf_values. Then, computed SDF values can be used several times when calling CGAL::segment_from_sdf_values with different parameters (i.e. number of levels, and smoothing lambda). ###Example### -\include Surface_mesh_segmentation/surface_mesh_segmentation_example.cpp +\include Surface_mesh_segmentation/compute_sdf_values_and_segment_example.cpp -##Using a polyhedron with an id per facet## -The previous examples use a `std::map` as property maps for storing the sdf values and the segmentation results. This example uses -a polyhedron type with a facet type having an extra id field together with a vector as underlying data structure in the property maps. +##Using a polyhedron with an ID per facet## +The previous examples use a `std::map` as property maps for storing the SDF values and the segmentation results. This example uses +a polyhedron type with a facet type having an extra ID field together with a vector as underlying data structure in the property maps. The main advantage is to decrease the complexity of accessing associated data with facets from logarithmic to constant. ###Example### -\include Surface_mesh_segmentation/surface_mesh_segmentation_with_facet_ids_example.cpp +\include Surface_mesh_segmentation/compute_sdf_values_and_segment_with_facet_ids_example.cpp # Implementation history# The initial implementation of this package is the result of the work of the author during the 2012 season