Abstract
Accurate camera calibration is a precondition for many computer vision applications. Calibration errors, such as wrong model assumptions or imprecise parameter estimation, can deteriorate a system’s overall performance, making the reliable detection and quantification of these errors critical. In this work, we introduce an evaluation scheme to capture the fundamental error sources in camera calibration: systematic errors (biases) and uncertainty (variance). The proposed bias detection method uncovers smallest systematic errors and thereby reveals imperfections of the calibration setup and provides the basis for camera model selection. A novel resampling-based uncertainty estimator enables uncertainty estimation under non-ideal conditions and thereby extends the classical covariance estimator. Furthermore, we derive a simple uncertainty metric that is independent of the camera model. In combination, the proposed methods can be used to assess the accuracy of individual calibrations, but also to benchmark new calibration algorithms, camera models, or calibration setups. We evaluate the proposed methods with simulations and real cameras.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Note, that the normalization of \(\epsilon _{\mathrm {bias}}\) is different to Hagemann et al. (2020), to clarify its relation to the estimated accuracy \({\hat{s}}_d^2\). The definition of the bias ratio, however, remains unaffected.
Here, we assume the underlying distribution is Gaussian but might be subject to sporadic outliers. The MAD multiplied by a factor of 1.4826 gives a robust estimate for the standard deviation (Rousseeuw and Croux 1993).
The decomposition of the target must lead to an overdetermined estimation problem.
As reference, we used the average of ten calibrations with 50 random images each.
Mapping error compared to the simulated ground-truth camera model.
Mapping error compared to the reference calibration.
The calibrations shown in Fig. 4 already take into account the board non-planarity.
References
Abraham, S., & Förstner, W. (1998). Calibration errors in structure from motion. In P. Levi, M. Schanz, R. J. Ahlers, & F. May (Eds.), Mustererkennung 1998 (pp. 117–124). Berlin Heidelberg: Springer.
Beck, J., & Stiller, C. (2018). Generalized B-spline Camera Model. In: 2018 IEEE Intelligent Vehicles Symposium (IV). pp. 2137–2142. IEEE, https://ieeexplore.ieee.org/abstract/document/8500466/
Bodenham, D. A., & Adams, N. M. (2016). A comparison of efficient approximations for a weighted sum of chi-squared random variables. Statistics and Computing, 26(4), 917–928.
Cheong, L. F., & Peh, C. H. (2004). Depth distortion under calibration uncertainty. Computer Vision and Image Understanding, 93(3), 221–244. https://doi.org/10.1016/j.cviu.2003.09.003
Cheong, L. F., & Xiang, X. (2011). Behaviour of SFM algorithms with erroneous calibration. Computer Vision and Image Understanding, 115(1), 16–30. https://doi.org/10.1016/j.cviu.2010.08.004
Cramariuc, A., Petrov, A., Suri, R., Mittal, M., Siegwart, R., & Cadena, C. (2020). Learning camera miscalibration detection. arXiv preprint arXiv:2005.11711
Davison, A.C., & Hinkley, D.V. (1997). Bootstrap methods and their application. No. 1, Cambridge university press
Doran, H. E., Doran, H., et al. (1989). Applied regression analysis in econometrics (Vol. 102). CRC Press.
Edwards, M.J., Hayes, M.P., & Green, R.D. (2020)Experimental validation of bias in checkerboard corner detection. In: 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ). pp. 1–6. IEEE
Förstner, W., & Wrobel, B. P. (2016). Photogrammetric computer vision. Berlin: Springer.
Furgale, P., Rehder, J., Siegwart, R. (2013). Unified temporal and spatial calibration for multi-sensor systems. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 1280–1286. IEEE
Hagemann, A., Knorr, M., Janssen, H., Stiller, C. (2020). Bias detection and prediction of mapping errors in camera calibration. In: German Conference on Pattern Recognition. Springer , in press.
Hartley, R., & Zisserman, A. (2004). Multiple view geometry in computer vision, https://doi.org/10.1017/CBO9780511811685, oCLC: 171123855
Imhof, J. P. (1961). Computing the distribution of quadratic forms in normal variables. Biometrika, 48(3/4), 419–426.
Lavest, J.M., Viala, M., Dhome, M. (1998). Do we really need an accurate calibration pattern to achieve a reliable camera calibration? In: European Conference on Computer Vision. pp. 158–174. Springer
Luhmann, T., Robson, S., Kyle, S., & Boehm, J. (2013). Close-range photogrammetry and 3D imaging. Beijing: Walter de Gruyter.
MathWorks: Matlab Computer Vison Toolbox, https://de.mathworks.com/help/vision/camera-calibration-and-3-d-vision.html, retrieved on January 8th, 2021
Mei, C., Rives, P.: Single view point omnidirectional camera calibration from planar grids. In: Proceedings 2007 IEEE International Conference on Robotics and Automation. pp. 3945–3950. IEEE (2007)
OpenCV: OpenCV Fisheye Camera Model, https://docs.opencv.org/master/db/d58/group_calib3d_fisheye.html, retrieved on January 8th, 2021
OpenCV: OpenCV Tutorial Camera Calibrations, https://docs.opencv.org/master/d7/d21/tutorial_interactive_calibration.html, retrieved on January 8th, 2021
Oth, L., Furgale, P., Kneip, L., Siegwart, R. (2013). Rolling shutter camera calibration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1360–1367
Ozog, P., & Eustice, R.M. (2013). On the importance of modeling camera calibration uncertainty in visual SLAM. In: 2013 IEEE International Conference on Robotics and Automation. pp. 3777–3784. IEEE, https://ieeexplore.ieee.org/abstract/document/6631108/
Peng, S., & Sturm, P .(2019). Calibration Wizard: A Guidance System for Camera Calibration Based on Modelling Geometric and Corner Uncertainty. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1497–1505
Rautenberg, U., & Wiggenhagen, M. (2002). Abnahme und ueberwachung photogrammetrischer messsysteme nach vdi 2634, blatt 1. PFG 2/2002, S.117-124 , https://www.ipi.uni-hannover.de/fileadmin/ipi/publications/VDI2634_1e.pdf
Richardson, A., Strom, J., & Olson, E. (2013). AprilCal: Assisted and repeatable camera calibration. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 1814–1821. IEEE, https://ieeexplore.ieee.org/abstract/document/6696595/
Rojtberg, P., Kuijper, A. (2018). Efficient pose selection for interactive camera calibration. In: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). pp. 31–36. IEEE , https://ieeexplore.ieee.org/abstract/document/8613748/
ROS: ROS Tutorial MonocularCalibration, http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration, retrieved on January 8th, 2021
Rousseeuw, P. J., & Croux, C. (1993). Alternatives to the median absolute deviation. Journal of the American Statistical association, 88(424), 1273–1283.
Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006). A toolbox for easily calibrating omnidirectional cameras. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 5695–5701. IEEE
Schoeps, T., Larsson, V., Pollefeys, M., & Sattler, T. (2019). Why Having 10,000 Parameters in Your Camera Model is Better Than Twelve. arXiv preprint arXiv:1912.02908
Semeniuta, O. (2016). Analysis of camera calibration with respect to measurement accuracy. Procedia Cirp, 41, 765–770.
Strauss, T. (2015). Kalibrierung von Multi-Kamera-Systemen. KIT Scientific Publishing, https://d-nb.info/1082294497/34
Strauß, T., Ziegler, J., & Beck, J. (2014). Calibrating multiple cameras with non-overlapping views using coded checkerboard targets. In: 17th international IEEE conference on intelligent transportation systems (ITSC). pp. 2623–2628. IEEE
Sturm, P.F., & Maybank, S.J. (1999). On plane-based camera calibration: A general algorithm, singularities, applications. In: Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149). vol. 1, pp. 432–437. IEEE
Sun, W., & Cooperstock, J. R. (2006). An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques. Machine Vision and Applications, 17(1), 51–67.
Svoboda, T., & Sturm, P. (1996) What can be done with a badly calibrated Camera in Ego-Motion Estimation?, https://hal.inria.fr/inria-00525701/
Triggs, B., McLauchlan, P.F., Hartley, R.I., & Fitzgibbon, A.W. (2000). Bundle Adjustment - A Modern Synthesis. In: Goos, G., Hartmanis, J., van Leeuwen, J., Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice, vol. 1883, pp. 298–372. Springer Berlin Heidelberg, Berlin, Heidelberg, https://doi.org/10.1007/3-540-44480-7_21
Vision, A. (2021). Manta g-235, https://www.alliedvision.com/en/products/cameras/detail/Manta/G-235.html, retrieved on January 11th,
Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1330–1334.
Zucchelli, M., & Kosecka, J. (2001). Motion bias and structure distortion induced by calibration errors. pp. 68.1–68.10. British Machine Vision Association https://doi.org/10.5244/C.15.68, http://www.bmva.org/bmvc/2001/papers/51/index.html
Acknowledgements
We thank Paul-Sebastian Lauer (Robert Bosch GmbH) for supporting the experimental setup and the data acquisition. We also thank the reviewers for the valuable comments.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by Zeynep Akata.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Hagemann, A., Knorr, M., Janssen, H. et al. Inferring Bias and Uncertainty in Camera Calibration. Int J Comput Vis 130, 17–32 (2022). https://doi.org/10.1007/s11263-021-01528-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-021-01528-x