Elsevier

Pattern Recognition Letters

Volume 136, August 2020, Pages 40-47
Pattern Recognition Letters

Fuzzy logic and histogram of normal orientation-based 3D keypoint detection for point clouds

https://doi.org/10.1016/j.patrec.2020.05.010Get rights and content

Highlights

  • We propose a novel fuzzy logic and histogram of the normal orientation-based 3D keypoint detection scheme.

  • We estimate the relevant parameters instead of finding the best empirical parameters for various datasets.

  • We propose a soft decision boundary by a fuzzy rule-based system.

  • The proposed approach improves the performance of the 3D detector, especially in noisy point cloud data.

Abstract

Point cloud processing has gained consideration for 3D object recognition and classification tasks. In this context, an important task is to detect the distinct and repeatable 3D keypoints. Many 3D keypoint detectors with low repeatability and distinctiveness have been proposed. The detection of highly repeatable and distinct keypoints is still an open problem. To address this issue, we propose a fuzzy logic and Histogram of Normal Orientation (HoNO)-based 3D keypoint detection scheme for Point Cloud (PC) data. To measure saliency, we exploit the structure of the PC and compute the eigenvalues of the covariance matrix and the HoNO to measure saliency. The histogram (HoNO) salient value is computed by the kurtosis values, which estimate the spread of the histogram. From the kurtosis and smallest eigenvalues, we compute the difference of the kurtosis values and the difference of the smallest eigenvalues of the query point against all the neighbouring points. The difference of kurtosis values and difference of smallest eigenvalues are applied to a fuzzy rule-based scheme for the keypoints detection. We compare the proposed algorithm with the state-of-the-art 3D keypoint detectors on five benchmark datasets. Experimental results demonstrate the superior performance of the proposed detector on most of the benchmark datasets both in terms of absolute and relative repeatability.

Introduction

In the last decade, Point Cloud (PC) processing has received increasing attention in the literature because of its high relevance for various applications like object classification and recognition. In this prospect, the detection of repeatable and distinct 3D keypoints on the PC is an important low-level task. Simple keypoint detectors come with a low computational cost, but suffer from low repeatability and distinctiveness like sparse sampling and mesh decimation [1]. Due to limited performance, these techniques are not efficient for real applications. In the last few years, 3D keypoint detection schemes have received great interest [1], [2], [3].

Many 3D keypoint detectors directly process the PC data to detect the keypoints, while other detectors convert them (the PC data) into meshes before further processing. The conversion of PCs into meshes increases the computational cost especially for large PC datasets, which typically contain millions of points. For computational efficiency, direct operations on the PC are preferred. The existing 3D keypoint detectors either use the geometry described by the PC or the photometric appearance (RGB information) to detect the salient points. Based on the selection of scales for salient points estimation, the 3D keypoint detectors can be mainly classified into two broad categories, i.e, fixed and adaptive scale detectors [1]. A fixed-scale detector finds the keypoints at a fixed scale, while adaptive-scale detectors establish a scale-space and detect the keypoints at different scales. The adaptive scale keypoint detection schemes typically exhibit superior performance (high absolute and relative repeatability) compared to fixed scale methods [1].

Few examples of fixed scale detectors are Local Surface Patches (LSP) [4], Heat Kernel Signature (HKS) [5], Intrinsic Shape Signatures (ISS) [6], Key-Point Quality (KPQ) [7], Harris 3D [8] and Histogram of Normal Orientation Detector (HoNO-D) [9]. The LSP [4] employs the Shape Index (SI) metric [10] to estimate the salient points. The keypoints are either local minima or maxima in a specified region. The LSP detects the keypoints quite uniformly [1], but has low relative repeatability. The HKS [5] uses the heat kernel on meshes to estimate the salient points, the local maxima of salient points are considered as the keypoints. The HKS has high memory complexity and low absolute repeatability. The ISS [6] considers the ratio of two successive eigenvalues of the covariance matrix. The smallest eigenvalues are used for the Non-Maximum Suppression (NMS) of the salient points. The ISS detects distinct keypoints uniformly but still has low repeatability particularly on some challenging datasets. The KPQ detector [7] manipulates the scatter matrix similar to ISS, but considers only two principle directions for pruning. The curvature value is used to perform the NMS of the salient points. The KPQ detects multiple keypoints on high curvature areas and avoids planar surfaces. It has low repeatability compared to adaptive scale version of the detector KPQ-AS. Harris 3D [8] algorithm uses surface normals to compute gradients in the covariance matrix. Harris 3D is an implementation of the 2D scheme in the 3D domain. The 2D-based scheme application in the 3D domain limits its performance in terms of repeatability. The HoNO-D [9] computes the Histogram of Normal Orientation (HoNO) to detect the salient region, and pruning is done by HoNO and eigenvalues of the covariance matrix. Despite the superior performance, the HoNO-D has many drawbacks. One of the main disadvantages of the HoNO-D is the empirical selection of the best parameter values for each dataset, which is impractical and cumbersome. The other issue is that the performance drastically decreases in the presence of noise, as they ignore the correlations of salient values (kurtosis and eigenvalue) to detect the keypoints. Prakhya et al. deal with two salient values as two independent detectors and consider the salient points of both detectors as the keypoints.

Among adaptive scale detectors, Laplace-Beltrami Scale Space (LBSS) [11], Salient Points (SP) [12], MeshDoG [13] and Key-Point Quality Adaptive Scale (KPQ-AS) [7] are prominent keypoint detectors. In LBSS [11] the scale space is built by the Laplace-Beltrami operator on increasing supports around each vertex of a 3D mesh. Both scale selection and NMS are achieved by the Laplace-Beltrami operators. The LBSS detects the points in high curvature surfaces as keypoints [1] but the repeatability is comparatively lower than the other adaptive scale keypoint detectors. In SP [12] the saliency is measured by applying the DoG operator on 3D coordinates of the vertices. As they utilize the 3D coordinates, the surface geometry of the points changes during the scale space. To perform saliency-based NMS, all saliency values at different scales are summed and all local maxima are discarded which have a lower value than a pre-defined percentage of the global maxima. The SP detects a limited number of keypoints and has low performance in terms of repeatability compared with other adaptive keypoint detectors specifically in the presence of noise. MeshDoG [13] applies the Difference-of-Gaussian (DoG) at multiple scales. The NMS is conducted at a predefined support radius on the current and adjacent scales. The MeshDoG detects multiple keypoint around high curvature areas therefore the distribution of keypoints is not uniform. While the MeshDoG exhibits notable performance on many datasets, but the performance degrades drastically in the presence of noise. The KPQ-AS [7] builds the scale-space by increasing the size of the support while the pruning and NMS steps are performed as in KPQ. The KPQ-AS detected keypoints are unevenly distributed [1] and detect multiple point on high curvature areas. Despite the acceptable performance on some databases, the performance degrades on challenging datasets (see Section 3 for more detail) which limits its application on a real scenario. For a more comprehensive review of unsupervised keypoint detectors, we refer the reader to [1], [3].

Recently, learning-based techniques have been proposed for the detection of the 3D keypoints [14], [15], [16]. Among them, a fixed scale learning-based keypoint detector [14] and an adaptive scale learning-based detector [15] have been proposed for a specific descriptor. The learning-based keypoint detector [16] has been proposed for depth images and provides high distinctiveness in the presence of noise, however with low repeatability, which is also a relevant factor for the object recognition pipeline. Although the learning-based methods have prominent performance, they require a large amount of training data. Hence, the performance of learning-based methods is limited to the number of models used for training, the training dataset and the specific descriptor, which limits their practical application.

From the literature overview [1], we noticed that the existing algorithms have low repeatability and distinctiveness or they focus on some specific region, high curvature. The performance of most state-of-the-art detectors degrades in the presence of noise, which restricts the use of the existing 3D keypoint detectors. To address the issues of existing detectors, we propose a fuzzy logic and HoNO-based keypoint detection scheme named as Fuzzy-HoNO in the rest of the manuscript. The Fuzzy-HoNO is based on the HoNO-based fixed scale detector [9]. The proposed technique modifies the HoNO-D presented in [9] by applying a fuzzy rule-based scheme to deal with the drawbacks of HoNO-D. A fuzzy rule-based system is implemented to determine the soft decision boundaries and make the proposed detector more robust especially in the presence of the noise. Our study shows that the soft decision boundaries improve the performance of the proposed detector compared with the state-of-the-art detectors where hard decision boundaries are used. The selection of hard decision boundary and to ignorethe correlation between the input values limit the efficiency of the existing algorithms. We argue that the soft decision boundary employs more information, and better reveals the confidence about the decision, hence increases the effectiveness of the decision. So, increases the efficiency of the proposed algorithm compared to the existing algorithms, especially in noisy PC data, which is confirmed by experimental evaluation (Section 3). In the proposed 3D keypoint detector two inputs are given to the fuzzy rule-based system. The first input is the difference of the kurtosis values of the HoNO of a query point against all the neighbouring points within a predefined support radius. The second input is the difference of the smallest eigenvalue of the covariance matrix of the query point and the neighbourhood points within a predefined distance. To avoid the laborious process of finding the best threshold values for each dataset, we use the average kurtosis and the average eigenvalue as the main parameters of fuzzy rule-based system. We tested the proposed algorithm on five benchmark datasets, and evaluate the performance of the proposed 3D keypoint detector against the state-of-the-art detectors in terms of absolute and relative repeatability. From experimental evaluation, one can observe a noticeable improvement in absolute and relative repeatability of the proposed 3D keypoint detector compared with the state-of-the-art 3D keypoint detectors.

Section snippets

Proposed technique

The proposed technique estimates the HoNO and the Eigenvalue Decomposition (EVD) of the covariance matrix at a pre-defined support radius to measure the salient points. Then, the proposed technique calculate the kurtosis value of the histogram to estimate the spread. Finally, to detect the keypoints, the proposed technique use a fuzzy rule-based system on the difference of kurtosis values and the difference of the smallest eigenvalue of the covariance matrix.

Experimental evaluation

The proposed algorithm is implemented in C++ using PCL [22] and FuzzyLite [23]. We compare the proposed technique with existing methods on publicly available benchmark datasets, on the SHOT website [21].

Conclusion

The proposed Fuzzy-HoNO 3D keypoint detector significantly improves the performance of the HoNO detector by using a fuzzy rule-based decision process. The fuzzy rule-based scheme improves the performance of the keypoint detector by using soft decision boundaries. Apart from that, it also reduces the burden of finding the specific parameters for every database by adaptively selecting the parameter values. The adaptive parameters and soft boundary condition make the proposed algorithm more

Declaration of Competing Interest

No conflicts of Interest.

Acknowledgments

The work of M. Z. Iqbal is supported by a Ph.D. scholarship provided by the Higher Education Commission (HEC) of Pakistan in collaboration with DAAD Germany. We acknowledge Stanford University Computer Graphics Laboratory and Prof. Ajmal Saeed Mian of UWA for providing the dataset. We also acknowledge the great effort of Federico Tombari and Samuele Salti for providing the 3D keypoint detection benchmark datasets (the SHOT-SpaceTime, the SHOT-SpaceTime, the Random Views and the Retrieval), the

References (28)

  • H. Chen et al.

    3D free-form object recognition in range images using local surface patches

    Pattern Recognit. Lett.

    (2007)
  • S.M. Prakhya et al.

    Detecting keypoint sets on 3D point clouds via histogram of normal orientations

    Pattern Recognit. Lett.

    (2016)
  • F. Tombari et al.

    Performance evaluation of 3D keypoint detectors

    Int. J. Comput. Vis.

    (2013)
  • S. Salti et al.

    A performance evaluation of 3d keypoint detectors

    IEEE International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission

    (2011)
  • V.K. Ghorpade et al.

    Performance evaluation of 3d keypoint detectors for time-of-flight depth data

    14th International Conference on Control, Automation, Robotics and Vision

    (2016)
  • J. Sun et al.

    A concise and provably informative multi-scale signature based on heat diffusion

    Comput. Graph. Forum

    (2009)
  • Y. Zhong

    Intrinsic shape signatures: a shape descriptor for 3D object recognition

    IEEE 12th International Conference on Computer Vision Workshops

    (2009)
  • A. Mian et al.

    On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes

    Int. J. Comput. Vis.

    (2010)
  • I. Sipiran et al.

    Harris 3D: a robust extension of the harris operator for interest point detection on 3d meshes

    Vis. Comput.

    (2011)
  • C. Dorai et al.

    COSMOS-A representation scheme for 3D free-form objects

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1997)
  • R. Unnikrishnan et al.

    Multi-scale interest regions from unorganized point clouds

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

    (2008)
  • U. Castellani et al.

    Sparse points matching by combining 3D mesh saliency with statistical descriptors

    Comput. Graph. Forum

    (2008)
  • A. Zaharescu et al.

    Surface feature detection and description with applications to mesh matching

    IEEE Conference on Computer Vision and Pattern Recognition

    (2009)
  • S. Salti et al.

    Learning a descriptor-specific 3D keypoint detector

    IEEE International Conference on Computer Vision

    (2015)
  • Cited by (0)

    View full text