Elsevier

Pattern Recognition

Volume 116, August 2021, 107930
Pattern Recognition

Compensating over- and underexposure in optical target pose determination

https://doi.org/10.1016/j.patcog.2021.107930Get rights and content

Highlights

  • A wrong (over or under) exposure leads to a bias in optical target pose determination.

  • Concentric circles targets allow detecting and compensating for any bias caused by exposure.

  • A rigorous sub-pixel edge detector is formulated based on a pixel-level image formation model.

  • A new algorithm uses geometric constraints of concentric circles to determine the exposure bias.

  • Experimental evaluation in a controlled environment confirms unbiased pose determination.

Abstract

Optical coded targets allow to determine the relative pose of a camera, on a metric scale, from one image only. Furthermore, they are easily and efficiently detected, opening to a wide range of applications in robotics and computer vision. In this work we describe the effect of pixel saturation and non-ideal lens Point Spread Function, causing the apparent position of the corners and the edges of the target to change as a function of the camera exposure time. This effect, which we call exposure bias, is frequent in over- or underexposed images and introduces a systematic error in the estimated camera pose. We propose an algorithm that is able to estimate and correct for the exposure bias exploiting specific geometric features of a common target design based on concentric circles. Through rigorous laboratory experiments carried out in a highly controlled environment, we demonstrate that the proposed algorithm is seven times more precise and three times more accurate in the target distance estimation than the algorithms available in the literature.

Introduction

Optical coded targets generally consist of a set of high-contrast geometric features (such as lines, squares or circles), in which a code is embedded to exclude false matches and distinguish multiple targets in a scene. Knowing the physical dimensions of the target and the camera calibration allows us to determine from a single image the metric pose of a target relative to the camera. Several target designs and detection algorithms have been proposed in the literature, such as ARToolKit [1], AprilTags [2], and ArUco [3], which have been successfully employed in multiple applications, such as camera calibration, photogrammetry, augmented reality, machine vision and robotics in general.

Target detection algorithms establish correspondences between the geometric features in object space and their projections on the image plane. From these correspondences, the pose of the camera relative to the target can be inferred. For instance, the main target feature in ArUco is a black square on a white background; its four corners are localized by a sub-pixel corner detector and the camera pose is determined by solving the Perspective-n-Points (PnP) problem [4].

The accuracy of the camera pose depends on how precisely the critical target features can be measured on the image. A major source of bias is the exposure time. Indeed, the apparent position of edges and corners may be different in long-exposed images because of the combined effect of a non-ideal lens Point Spread Function (PSF) and pixel saturation. In Fig. 1, we compare two images of an ArUco target taken with two different exposure times: the apparent size of the target is different in the two images and the target on the right seems to be farther away from the camera. We refer to this effect as exposure bias, i.e., the systematic error in the estimated camera pose that is a function of the exposure time.

Since saturation in over- or underexposed images leads to a loss of information, it seems difficult to recover the true position of an image feature, such as a corner or an edge, just by using the intensity of pixels in its vicinity. Therefore, image exposure has to be optimized to avoid saturation. However, this is difficult in many cases because of unpredictable light conditions, variable scene illumination as functions of time and space, and reflections on the target surface. Moreover, auto-exposure algorithms are typically implemented at sensor level to optimize either the overall scene exposure or the exposure of a predefined area, which does not guarantee an optimal exposure of the target. As an example, in [5], an UAV follows a terrestrial mapping vehicle that has an optical target mounted on the roof. The pose computed thanks to the optical target have been proved essential to increase the accuracy of reconstructed 3D models by means of photogrammetry. In such outdoor conditions, it is difficult to optimise the illumination or exposure of the camera to obtain at the same time an optimal, unsaturated picture of the target, and an optimal illumination in the rest of the scene. This also holds in real-world indoor navigation scenarios, e.g., warehouses, where the light conditions are not uniform and difficult to control. An algorithm to optimize the camera exposure time in real-time was presented by the authors in [6], however this would allow to optimize the target exposure only, possibly compromising the usability of the images for other applications, e.g., in mapping.

In this work, we introduce a new optical target design and its corresponding pose determination algorithm that are not affected by either the exposure time or the illumination. The proposed target design is shown in Fig. 2 and it consists of a black ring over a white background; inside the ring, there are small white dots in which a code is embedded. The same pattern is repeated in a fractal fashion to allow for multi-scale detection (e.g., from very far or very near). In this design, the over- or underexposure affects the apparent thickness of the black ring. However, this effect is orthogonal to the one that we obtain by moving the camera farther or closer to the target, leading to a uniform scaling of the whole target and preserving the proportions of every feature (see the purple and red arrows in Fig. 2). Hence, we can design an algorithm that exploits this effect to estimate and correct the exposure bias in contour positions.

Many studies have investigated how to determine the camera pose from concentric circular features. For instance, in [7], a method for camera calibration based on concentric circles was presented, and another formulation was proposed in [8]. The geometric and algebraic constraints related to the projection of concentric circles were discussed in [9]. In [10], the authors addressed the issue of how to accurately locate the center of two concentric circles based on their projections (non-concentric ellipses). However, all these works assume that the apparent location of the circles on the image plane is not biased by the exposure. Here, we propose a new algorithm for camera pose determination that exploits the specific geometry of the target to estimate and compensate for any exposure bias in the localization of edges.

This work is organized as follows. In Section 2, we introduce the image processing pipeline for our target design. Each step of the pipeline will be detailed in later sections: in Section 3 and Section 4 we develop a method to determine with sub-pixel precision the apparent image location of the two concentric circles of the target, taking into account a non-ideal lens PSF. In Section 5, we determine the camera pose by using the position of a set of edge points and assuming that those are the projections of two concentric circles of known diameter. In Section 6, we modify such method to compensate for the exposure bias, which we link to the deviations in the apparent thickness of the black ring. In Section 7, we validate our exposure compensation algorithm through acquired ground truth data, proving its illumination invariance and the orthogonality between the distance effect and the exposure bias effect.

Section snippets

Overview of the image processing pipeline

In this section, we summarize the steps required to determine the position and the orientation of the proposed target relative to the camera.

  • 1.

    The target is located in the image and its main features, the inner and the outer circle, are coarsely identified. First, a pixel-level edge detection algorithm (e.g., Canny [11]) is applied, then the elliptical contours are clustered in concentric couples. For each couple, the distinctive target code is searched (see again Fig. 2), and the two contours

Edge formation model

In this section, we derive a closed-form model for the light intensity in the vicinity of black/white transitions (i.e., edges) of the target, as measured from the imaging sensor. This model will be used in the next section to construct a rigorous sub-pixel edge detector.

Consider an arbitrary 3D point P on a black/white edge of the target and refer to Fig. 3. Each point on the target maps to a point on the image plane via a function Π(·):R3R2, so that P=Π(P). Π(·) accounts for the camera pose

Subpixel edge position

In the following section, we consider the problem of determining the sub-pixel location of P given that an initial guess, P0, and the vector orthogonal to the edge and passing trought P0, n, are known. These are obtained in previous image processing steps. For example, in [10], the authors determine the pixel-level edge points of circular target features by using the Canny edge detector. Next, an ellipse is fitted on those as in [12]; from the ellipse, n can be obtained in closed form for

Camera pose determination

We present a new rigorous approach to determine the camera pose starting from a set of edge points being the projection of the two concentric circles composing the target (see Fig. 2). In Section 6, we will extend this approach to estimate and compensate for the exposure bias.

After applying the sub-pixel detection algorithm introduced in Section 4, a set of edge points P^i is available. These are the projections of object points lying on two concentric circles of known radius. The camera pose

Correcting for the exposure bias

In the previous section, we have seen how to determine the position and the orientation of the camera relative to the proposed target from a set of sub-pixel edge locations obtained as described in Section 4. However, we also saw that these measurements can be biased if the exposure time is not ideal (see Fig. 4). Indeed, in case of over- or underexposure, the apparent position of P^i can be displaced along the line normal to the edge ni: When the image is overexposed, such displacement is

Experimental evaluation

The goal of our method is to determine the position of a camera relative to a target compensating for any possible bias caused by over- or underexposure. We validated it in a highly controlled environment: refer to Fig. 10. Both the camera A and the target D were fixed on a rigid tripod, and a set of photos with different exposure times was taken to simulate normal or difficult light conditions. The same set-up was repeated for 20 different positions of the target, and, for each of them, the

Conclusions

We have introduced a new method to rigorously fit the image of a concentric circle target to determine the pose of the camera that sees it. Moreover, we have addressed the problem of the exposure that displaces black and white edges, moving them towards the white direction and leading to systematic effects. Then, we have proposed a way to estimate this displacement and reduce the error.

We performed experimental evaluations of the algorithm in controlled laboratory conditions: We measured the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research has been partially funded by the European Community Horizon 2020 Programme under Grant No. 641518: “EGNOS-GPS/GALILEO-based high-resolution terrestrial-aerial sensing system” managed by the European GNSS Agency (GSA).

Emmanuel Cledat received his master degree in geomatic engineering from Institut national des sciences appliquées (INSA), Strasbourg, France in 2015. His master thesis focused on 3D modeling of complex industrial environments using terrestrial laser scanning and was run in collaboration with Électricité de France (EDF). He received his Ph.D. from École polytechnique fédérale de Lausanne (EPFL), Switzerland, for his thesis under the supervision of Prof. Jan Skaloud. He is currently associate

References (15)

  • S. Garrido-Jurado et al.

    Automatic generation and detection of highly reliable fiducial markers under occlusion

    Pattern Recognit

    (2014)
  • D. Wagner et al.

    Pose tracking from natural features on mobile phones

    Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality

    (2008)
  • E. Olson

    AprilTag: A robust and flexible visual fiducial system

    Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)

    (2011)
  • Y. Wu et al.

    Pnp problem revisited

    J Math Imaging Vis

    (2006)
  • P. Molina et al.

    First results of a tandem terrestrial-unmanned aerial mapkite system with kinematic ground control points for corridor mapping

    Remote Sens (Basel)

    (2017)
  • L. Jospin et al.

    Photometric long-range positioning of led targets for cooperative navigation in uavs

    Drones

    (2019)
  • G. Jiang et al.

    Detection of concentric circles for camera calibration

    Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on

    (2005)
There are more references available in the full text version of this article.

Cited by (0)

Emmanuel Cledat received his master degree in geomatic engineering from Institut national des sciences appliquées (INSA), Strasbourg, France in 2015. His master thesis focused on 3D modeling of complex industrial environments using terrestrial laser scanning and was run in collaboration with Électricité de France (EDF). He received his Ph.D. from École polytechnique fédérale de Lausanne (EPFL), Switzerland, for his thesis under the supervision of Prof. Jan Skaloud. He is currently associate professor at Institut géographique national (IGN) and Ecole Nationale des Sciences Géographiques (ENSG). His current research interests are image processing, computer vision, photogrammetry, laser scanning and geographic information systems.

Marc Rufener received his bachelor and master degree in environmental engineering from École polytechnique fédérale de Lausanne (EPFL), Switzerland, in 2015 and 2017. His master thesis involved the optimization of a source-receptor air quality model to simulate the effects of emission reductions. He is currently database manager at the canton administration of Neuchâtel, Switzerland. His research interests span spatial data analysis, 3D modeling and geomatics.

Davide A. Cucci received his bachelor and master degrees in computer engineering from Politecnico di Milano, Italy, in 2008 and 2011. From the same university, he obtained the Ph.D. degree on 2014. His thesis covered the development of multi-sensor fusion methods for the real-time navigation and mapping in mobile robotics. He is post-doc researcher at the Geodetic Engineering Laboratory, École polytechnique fédérale de Lausanne (EPFL), Switzerland. His research interests lie in methods for high-accuracy navigation and mapping using small unmanned aerial vehicles, but also span autonomous navigation and control, photogrammetry, computer vision and signal processing.

View full text