Elsevier

Computers & Graphics

Volume 102, February 2022, Pages 144-153
Computers & Graphics

Special Section on 3DOR 2021
SHREC’21: Quantifying shape complexity

https://doi.org/10.1016/j.cag.2021.09.005Get rights and content

Highlights

  • Two novel datasets with associated ground truth are introduced.

  • Segmentation is used as a proxy task for measuring complexity.

  • The performance of 2D/3D complexity measures are systematically compared/evaluated.

Abstract

This paper presents the results of SHREC’21 track: Quantifying Shape Complexity. Our goal is to investigate how good the submitted shape complexity measures are (i.e. with respect to ground truth) and investigate the relationships between these complexity measures (i.e. with respect to correlations). The dataset consists of three collections: 1800 perturbed cube and sphere models classified into 4 categories, 50 shapes inspired from the fields of architecture and design classified into 2 categories, and the data from the Princeton Segmentation Benchmark, which consists of 19 natural object categories. We evaluate the performances of the methods by computing Kendall rank correlation coefficients both between the orders produced by each complexity measure and the ground truth and between the pair of orders produced by each pair of complexity measures. Our work, being a quantitative and reproducible analysis with justified ground truths, presents an improved means and methodology for the evaluation of shape complexity.

Introduction

Shape complexity is studied across several fields such as psychology [1], design [2], [3], computer vision [4]. In the context of 3D shapes, it has the potential to be useful in shape retrieval [5], [6], measuring neurological development and disorders [7], [8], in determining the processes and costs involved for manufacturing products [2], [9], etc. Early work on shape complexity appears in the literature of experimental psychology as well as in literature related to design and aesthetics. The classical aesthetic notions of “unity” and “variety” [10], or comparably, “order” and “complexity” [11] are directly connected to the complexity of spatial objects. One of the first measures of complexity for polygonal shapes can be found in [11]. Attneave [1] conducted human experiments to seek correlations of shape complexity with scale, curvedness, symmetry and number of turns. On the basis of the variety in the responses from human subjects, Attneave states that shape complexity is ill-defined. With the premise of circles being the simplest shapes, a natural candidate for the quantification of shape complexity is P2/A. In several works [1], [3] it is used as a measure of the complexity along with other indicators. In most other works [4], [12], [13], [14], [15], [16], tools from information theory, on top of various geometric features are used to quantify complexity. Work that relates complexity to algorithmic information theory and is applied to objects of art and design can also be found in Stiny and Gips [17]. Rossignac [18] provides a classification of shape complexity that focuses on measuring different aspects of computer representations for 3D shapes. The variety of approaches taken in the quantification of shape complexity further supports the claim that complexity can obtain a variety of meanings based on the approach that one chooses to take in a particular research area and for the particular task at hand.

There is a lack of benchmark datasets for shape complexity. Even the methodologies in the literature need improvements. For example, in many cases just visual results are reported without quantitative analysis [13], [19]. The methods are neither compared to other methods nor evaluated in terms of statistical consistency. In this track paper, we aim to account for and investigate different aspects of complexity that can help other researchers to develop and test their methods. In particular, we investigate how good the submitted shape complexity measures are (i.e. with respect to ground truth) and investigate the relationships between these complexity measures (i.e. with respect to pairwise correlations). Due to the ill-defined nature of complexity, a linear order may not make sense. Hence, we propose to explore complexity using multiple tasks and multiple shape collections.

The first collection is composed of subgroups obtained by introducing additive or subtractive noise to two basic shapes: sphere and cube. The purpose is to investigate the relation of complexity to noise level. The second collection is composed of artificial 3D shapes constructed by transforming and combining multiple elements, and evaluated by experts to provide ground truth. The purpose is to investigate the complexity methods in relation to perceptual categories. The final collection is an already existing 3D shape dataset which was originally developed as a segmentation benchmark. We repurpose this data and use the segmentation ground truth as a means to investigate 3D shape complexity via a proxy (secondary) task. The main contributions of this work are as follows:

  • Generation of two novel shape collections with associated ground truth, and repurposing of a previous segmentation benchmark for assessing complexity measures.

  • Systematic evaluation of the performance of a selection of both 2D and 3D classical and recent shape complexity measures.

  • Assessment of similarities and differences between different measures by using pairwise correlations and clustering based on their performance with respect to multiple ground truths.

Note that due to the ill-defined nature of shape complexity our dataset contains three shape collections with different characteristics. Each collection in the dataset contains a different type of 3D object with ground truth defined and obtained in a different manner.

The paper is organized as follows: In Section 2 the dataset is introduced. In Section 3 the ground truths and the evaluation strategy are explained. In Section 4 the short descriptions of the participating methods are included. In Section 5 the evaluation results for each collection in the dataset are presented. Finally, Section 6 is Discussion and Section 7 is Conclusion.

Section snippets

Dataset

The used dataset consists of three collections each aiming to account for a different aspect of shape complexity. The first two collections are created synthetically, and the third is an existing collection consisting of natural shapes. The ground truth for the first collection is based on the parameters used in creating the collection. For the second collection, the ground truth is provided by two design experts on the final design object. The purpose of the third collection is to investigate

Collection 1

For the first collection, the two parameters |w| and |c| used in creating the shapes constitute the ground truth. We expect the complexity scores to increase as either of the parameters increase. The performance of the methods are measured in a controlled experiment manner: we keep one of the parameters fixed and let the other vary. The performance of a method is then measured by averaging the Kendall rank correlation coefficient over the groups. This results in six measures of performance (one

Methods

We present the examined methods in this section. A total of 19 methods are presented in 6 groups:

  • 1.

    A multi-scale measure of complexity for arbitrary dimensional discrete shapes [16] by M. F. Arslan, Section 4.1,

  • 2.

    Alpha-shape complexity [21] by J. Gardiner and C. Brassey, Section 4.2

  • 3.

    Discrepancy [15] by A. Genctav, Section 4.3,

  • 4.

    PARCELLIN distance [14] by M. Genctav, Section 4.4,

  • 5.

    2D multi-view based shape convexity measures C1, C2, [22], [23] by P. L. Rosin Section 4.5

  • 6.

    2D multi-view based shape

Results

Since ground truths provide only the order information, we are interested in the order relations rather than linear relationship between actual values that could be measured by Pearson correlation coefficient or any other parametric relation. Even for pairwise comparison of measures in Section 6, order correlation seems as a more meaningful measure rather than some preassumed parametric relation which may or may not exist. Hence, we use only Kendall rank correlation as a robust rank correlation

Discussion

Despite the lack of full shape information, 2D methods are observed to perform unexpectedly well when compared with 3D methods, especially for Collection 3. However, it should be noted that for the results presented in this paper, the participating 3D methods also do not make use of the full shape information because some of them [14], [21] down-sample the shapes in Collection 1 and they all need to voxelize the shapes in Collection 2 and 3. Also, [16] solves for only a limited number of steps

Conclusion

We have introduced a novel 3D dataset to evaluate shape complexity measures. Using this dataset we not only evaluated the methods with respect to ground truth but also with respect to each other under a rich variety of ordering tasks in order to see how they are related in the context of shape complexity. To evaluate methods with respect to each other, we clustered measures in the tau-based feature space, and displayed pairwise rank correlations between orders induced by all pair of methods.

We

CRediT authorship contribution statement

Mazlum Ferhat Arslan: Conceptualization, Methodology, Software, Formal analysis, Data curation, Writing – original draft, Writing – review & editing, Visualization. Alexandros Haridis: Conceptualization, Methodology, Data curation, Writing – original draft, Writing – review & editing. Paul L. Rosin: Conceptualization, Methodology, Formal analysis, Writing – original draft, Writing – review & editing, Supervision. Sibel Tari: Conceptualization, Methodology, Formal analysis, Writing – original

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (35)

  • BackesA.R. et al.

    Characterizing 3D shapes using fractal dimension

  • AraiK.

    Visualization of 3D object shape complexity with wavelet descriptor and its application to image retrievals

    J Vis

    (2012)
  • Nitzken M, Casanova M, Gimel’farb G, Elnakib A, Khalifa F, Switala A, El-Baz A. 3D shape analysis of the brain cortex...
  • VolarevicN. et al.

    Shape complexity measure study

    Ann DAAAM Proc

    (2005)
  • HutchesonF.
  • BirkhoffG.D.

    Aesthetic measure

    (1933)
  • MaragosP.

    Pattern spectrum and multiscale shape representation

    IEEE Trans Pattern Anal Mach Intell

    (1989)
  • Cited by (6)

    This article has been certified as Replicable by the Graphics Replicability Stamp Initiative: http://www.replicabilitystamp.org.

    1

    Track organizers.

    View full text