Abstract
The accurate estimation of camera pose using numerous line correspondences in real time is a challenging task. This paper presents a non-iterative approach to solve the Perspective-n-Line (PnL) problem. The method can provide high speed and global optimality, as well as linear complexity. A nonlinear least squares (non-LLS) objective function is first formulated by parameterizing the rotation matrix with Cayley representation. A system of three third-order equations is then derived from its optimality conditions, and then, it is solved directly based on the Gröbner basis technique. Finally, the camera pose can be easily obtained by back-substitution. A major advantage of the proposed method lies in scalability, as the size of the elimination template used in the Gröbner basis technique is independent to the number of line correspondences. Extensive and detailed experiments on synthetic data and real images are conducted, demonstrating that the proposed method achieves an accuracy that is equivalent or superior to the leading methods, but with reduced computational requirements. The source code is available at https://github.com/dannyshin1/danny/tree/master/OPnL1.
Similar content being viewed by others
References
Abdelaziz, Y.I.: Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. In: ASP Symposium on Close-Range Photogrammetry (1971)
Ansar, A., Daniilidis, K.: Linear pose estimation from points or lines. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 578–589 (2003)
Bonnal, C., Ruault, J.M., Desjean, M.C.: Active debris removal: recent progress and current trends. Acta Astronaut. 85, 51–60 (2013)
Brezov, D.S., Mladenova, C.D., Mladenov, I.M.: New perspective on the gimbal lock problem. In: AIP Conference Proceedings 1570(1), 367–374 (2013)
Bronson, R., Costa, G.B.: An Introduction to Optimization. Wiley, Hoboken (2009)
Chen, H.H.: Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. IEEE Trans. Pattern Anal. Mach. Intell. 6, 530–541 (1991)
Cropp, A., Palmer, P., Underwood, C.I.: Pose estimation of target satellite for proximity operations. In: 14th Annual AIAA/USU Conference on Small Satellites (2000)
David, P., DeMenthon, D., Duraiswami, R., Samet, H.: Simultaneous pose and correspondence determination using line features. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., vol. 2, pp. II–II. IEEE (2003)
Dhome, M., Richetin, M., Lapreste, J.T., Rives, G.: Determination of the attitude of 3d objects from a single perspective view. IEEE Trans. Pattern Anal. Mach. Intell. 11(12), 1265–1278 (1989)
Dong, G., Zhu, Z.: Incremental inverse kinematics based vision servo for autonomous robotic capture of non-cooperative space debris. Adv. Space Res. 57(7), 1508–1514 (2016)
Ferraz, L., Binefa, X., Moreno-Noguer, F.: Very fast solution to the PnP problem with algebraic outlier rejection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 501–508. IEEE (2014)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Gander, W.: Least Squares Fit of Point Clouds. Springer, Berlin (1997)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)
Hesch, J.A., Roumeliotis, S.I.: A direct least-squares (DLS) method for PnP. In: International Conference on Computer Vision, pp. 383–390. IEEE (2011)
Kelsey, J.M., Byrne, J., Cosgrove, M., Seereeram, S., Mehra, R.K.: Vision-based relative pose estimation for autonomous rendezvous and docking. In: 2006 IEEE Aerospace Conference, pp. 20–pp. IEEE (2006)
Kneip, L., Li, H., Seo, Y.: UPnP: an optimal O(n) solution to the absolute pose problem with universal applicability. In: European Conference on Computer Vision, pp. 127–142. Springer (2014)
Kubota, T., Sawai, S., Hashimoto, T., Kawaguchi, J.: Robotics and autonomous technology for asteroid sample return mission. In: ICAR’05. Proceedings., 12th International Conference on Advanced Robotics, 2005., pp. 31–38. IEEE (2005)
Kukelova, Z., Bujnak, M., Pajdla, T.: Automatic generator of minimal problem solvers. In: European Conference on Computer Vision, pp. 302–315. Springer (2008)
Kumar, R., Hanson, A.: Robust methods for estimating pose and a sensitivity analysis. Comput. Vis. Image Underst. 60(3), 313–342 (1994)
Larsson, V., Kukelova, Z., Zheng, Y.: Making minimal solvers for absolute pose estimation compact and robust. In: International Conference on Computer Vision, pp. 2335–2343. IEEE (2017)
Lepetit, V., Moreno-Noguer, F., Pascal, F.: EPnP: An accurate O(n) solution to the PnP problem. Int. J. Comput. Vision 81(2), 155–166 (2009)
Li, S., Xu, C., Xie, M.: A robust O(n) solution to the perspective-n-point problem. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1444–1450 (2012)
Liu, Y., Huang, T.S., Faugeras, O.D.: Determination of camera location from 2D to 3D line and point correspondences. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 82–88. IEEE (1988)
Mirzaei, F.M., Roumeliotis, S.I.: Globally optimal pose estimation from line correspondences. In: International Conference on Robotics and Automation, pp. 5581–5588. IEEE (2011)
Mourikis, A.I., Trawny, N., Roumeliotis, S.I., Johnson, A.E., Ansar, A., Matthies, L.: Vision-aided inertial navigation for spacecraft entry, descent, and landing. IEEE Trans. Rob. 25(2), 264–280 (2009)
Nakano, G.: Globally optimal DLS method for PnP problem with Cayley parameterization. In: British Machine Vision Conference, pp. 78.1–78.11 (2015)
Opromolla, R., Fasano, G., Rufino, G., Grassi, M.: A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations. Prog. Aerosp. Sci. 93, 53–72 (2017)
Přibyl, B., Zemčĺk, P., Čadĺk, M.: Camera pose estimation from lines using plücker coordinates. In: British Machine Vision Conference, pp. 1–12 (2015)
Silva, M.S., Ferreira, R., Gaspar, J.: Camera calibration using a color-depth camera: points and lines based DLT including radial distortion. In: International Conference on Intelligent Robots and Systems. IEEE/RSJ (2012)
Urban, S., Leitloff, J., Hinz, S.: Mlpnp -a real-time maximum likelihood solution to the perspective-n-point problem. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 3(3), 131–138 (2016)
Vakhitov, A., Funke, J., Moreno-Noguer, F.: Accurate and linear time pose estimation from points and lines. In: European Conference on Computer Vision, pp. 583–599. Springer (2016)
Wang, P., Xu, G., Cheng, Y., Yu, Q.: A simple, robust and fast method for the perspective-n-point problem. Pattern Recogn. Lett. 108, 31–37 (2018)
Wang, P., Xu, G., Cheng, Y., Yu, Q.: Camera pose estimation from lines: a fast, robust and general method. Mach. Vis. Appl. 25(5), 603–614 (2019)
Xu, C., Zhang, L., Cheng, L., Koch, R.: Pose estimation from line correspondences: a complete analysis and a series of solutions. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1209–1222 (2017)
Yan, K., Zhao, R., Tian, H., Liu, E., Zhang, Z.: A high accuracy method for pose estimation based on rotation parameters. Measurement 122, 392–401 (2018)
Zhang, L., Xu, C., Lee, K.M., Koch, R.: Robust and efficient pose estimation from line correspondences. In: Asian Conference on Computer Vision, pp. 217–230 (2012)
Zhang, X., Zhang, Z., Li, Y., Zhu, X., Yu, Q., Ou, J.: Robust methods for estimating pose and a sensitivity analysis. Appl. Opt. 51, 936–948 (2012)
Zheng, Y., Kuang, Y., Sugimoto, S., Åström, K., Okutomi, M.: Revisiting the PnP problem: A fast, general and optimal solution. In: International Conference on Computer Vision, pp. 2344–2351. IEEE (2013)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 61473148) and the China Scholarship Council (No. 201906830092).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yu, Q., Xu, G. & Cheng, Y. An efficient and globally optimal method for camera pose estimation using line features. Machine Vision and Applications 31, 48 (2020). https://doi.org/10.1007/s00138-020-01100-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00138-020-01100-6