The equivalence of two definitions of compatible homography matrices

https://doi.org/10.1016/j.patrec.2020.03.033Get rights and content

Highlights

  • We present two definitions of compatible homography matrices related to two fixed views.

  • We show that the equivalence of the two definitions.

  • We point out consequences for the development of homography consistency constraints.

Abstract

In many computer vision applications, one acquires images of planar surfaces from two different vantage points. One can use a projective transformation to map pixel coordinates associated with a particular planar surface from one image to another. The transformation, called a homography, can be represented by a unique, to within a scale factor, 3 × 3 matrix. One requires a different homography matrix, scale differences apart, for each planar surface whose two images one wants to relate. However, a collection of homography matrices forms a valid set only if the matrices satisfy consistency constraints implied by the rigidity of the motion and the scene. We explore what it means for a set of homography matrices to be compatible and show that two seemingly disparate definitions are in fact equivalent. Our insight lays the theoretical foundations upon which the derivation of various sets of homography consistency constraints can proceed.

Introduction

Estimating the homography induced by a plane between two views from image measurements is a key step towards performing such tasks as metric rectification [10], [21], panorama generation [3], motion estimation [13], [17], [25], or camera calibration [35]. A closely related estimation issue is that of estimating multiple homographies between two views (see Fig. 1). Successful tackling of this issue underpins many practical applications including non-rigid motion detection [19], [34], enhanced image warping [12], multiview 3D reconstruction [20], augmented reality [27], indoor navigation [28], multi-camera calibration [32], camera-projector calibration [26], or ground-plane recognition for object detection and tracking [1]. The estimation of a set of matrices representing homographies between two views cannot be simply reduced to the estimation of individual matrices. The reason for this is that homographies between two views are intrinsically interdependent and the corresponding matrices are subject to compatibility constraints. The identification of a full set of compatibility constraints is a challenging problem and its solution is predicated on finding a characterisation of interdependent homography matrices. As it turns out, multiple homography matrices can be characterised by adopting two different approaches. One of these exploits latent variables that capture the two underlying views and the planes giving rise to the homographies. The other is based on the correlation that exists between the fundamental matrix and the homographies between two views. The aim of this paper is to show that these two approaches are equivalent.

Interestingly, when properly exploited, the two different characterisations of multiple homography matrices lead to two different sets of compatibility constraints. The situation here resembles that arising in regard to the trifocal tensor from multiple view geometry: the entries of the tensor are amenable to several sets of constraints [18], [24] and each such set is derived from a specific characterisation of what constitutes a valid trifocal tensor among tensors of an algebraically relevant type.

Section snippets

Two approaches

A set of 3 × 3 invertible matrices constitutes a set of compatible homography matrices if the matrices of the set represent homographies induced by multiple planes between two views. Mathematically, the compatibility requirement can be captured by two seemingly different definitions. We describe these next.

Auxiliary results

Before showing that the two definitions of compatible homography matrices are equivalent, we present some auxiliary results that we state below in the form of two theorems.

For a length-3 vector a=[a1,a2,a3], let [a] ×  denote the 3 × 3 anti-symmetric matrix given by[a]×=[0a3a2a30a1a2a10].We recall the following fundamental property of the matrix [a] × : if we let  ×  denote the cross products of length-3 vectors, then a×y=[a]×y for each yR3.

Theorem 1

If F is a 3 × 3 matrix of rank two, then there

Equivalence of the definitions

The argument for establishing the equivalence of Definitions 1 and 2 splits into two parts.

Definition 1 implies Definition 2. Let H1,,HIR3×3 be invertible matrices satisfying Eq. (3) for AR3×3, b,v1,,vIR3, and w1,,wIR. We start with the claim that A may safely be assumed invertible—in fact A may be assumed equal to any of the invertible matrices Hi.

First note that wi ≠ 0 for each i=1,,I. Indeed, if wi=0 held for some i, then Hi would be equal to bvi and hence would be of rank one,

Conclusion

We have presented two definitions of what constitutes a set of compatible homography matrices for the description of a set of homographies between two fixed views, and we have shown the equivalence of these definitions. As mentioned earlier, one of the two definitions was used, in [6], to develop a full set of compatibility constraints for multiple homography matrices. The result of the present paper opens an avenue for identification of another full set of compatibility constraints for

Declaration of Competing Interest

Wojciech Chojnacki, Zygmunt L. Szpak and Mårten Wadenbäck certify that they have no affiliations with or involvement in any organisation or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs)

Acknowledgements

This research was supported by the Australian Research Council and the Foundation for Scientific Research and Education in Mathematics (SVeFUM).

References (35)

  • X. Guo et al.

    Homography-based block motion estimation for video coding of PTZ cameras

    J. Vis. Commun. Image R.

    (2016)
  • Z.L. Szpak et al.

    Robust multiple homography estimation: an ill-solved problem

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    (2015)
  • J. Arrospide et al.

    Homography-based ground plane detection using a single on-board camera

    IET Intell. Transp. Syst.

    (2010)
  • D.S. Bernstein

    Matrix Mathematics: Theory, Facts, and Formulas

    (2009)
  • M. Brown et al.

    Automatic panoramic image stitching using invariant features

    Int. J. Computer Vision

    (2007)
  • P. Chen et al.

    Rank constraints for homographies over two views: revisiting the rank four constraint

    Int. J. Computer Vision

    (2009)
  • W. Chojnacki et al.

    Multiple homography estimation with full consistency constraints

    In Proceedings of the International Conference on Digital Image Computing: Techniques and Applications

    (2010)
  • W. Chojnacki et al.

    Full explicit consistency constraints in uncalibrated multiple homography estimation

    Proceedings of the 14th Asian Conference on Computer Vision, vol. 11361 of Lecture Notes in Computer Science

    (2019)
  • W. Chojnacki et al.

    Enforcing consistency constraints in uncalibrated multiple homography estimation using latent variables

    Mach. Vision Appl.

    (2015)
  • W. Chojnacki et al.

    A dimensionality result for multiple homography matrices

    Proceedings of the 13th International Conference on Computer Vision

    (2011)
  • W. Chojnacki et al.

    On the dimension of the set of two-view multi-homography matrices

    Complex Anal. Oper. Theory

    (2013)
  • A. Criminisi et al.

    Single view metrology

    Int. J. Computer Vision

    (2000)
  • O.D. Faugeras et al.

    Motion and structure from motion in a piecewise planar environment

    Int. J. Pattern Recogn. Artif. Intell.

    (1988)
  • J. Gao et al.

    Constructing image panoramas using dual-homography warping

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    (2011)
  • R.I. Hartley et al.

    Multiple View Geometry in Computer Vision

    (2004)
  • E. Hewitt et al.

    Real and Abstract Analysis

    (1975)
  • R.A. Horn et al.

    Matrix Analysis

    (2013)
  • Cited by (2)

    View full text