一种基于结构特征的单目视觉惯性里程计

doi:10.12305/j.issn.1001-506X.2023.10.25

Abstract

Abstract:

In response to the problem of poor robustness of traditional visual odometers that only rely on point features, a monocular visual inertial odometry that utilize point features, line features and structural features are deduced. The structural feature is an orthonormal basis that satisfy the Manhattan world hypothesis. This feature contains the parallel and orthogonal information of the structural object. The introduction of structural feature provide an overall constrain for visual inertial odometry and can reduce the accumulative error of rotation effectively. Meanwhile, to ensure the real-time performance of the system after the introduction of structural features, a robust and efficient algorithm is proposed to extract structural feature, which is based on the matching relationship of line features and the direction of gravity vector. The experimental results demonstrate that, compared with the existing open source systems, the proposed system can effectively reduce the accumulated rotation error while maintaining the real-time performance.

Key words: simultaneous localization and mapping, point feature, line feature, structural feature, inertial measurement unit

CLC Number:

TP391

Liping YAN, Chengwei LI, Bokai XU, Yuanqing XIA, Bo XIAO. A monocular visual inertial odometry based on structural features[J]. Systems Engineering and Electronics, 2023, 45(10): 3207-3217.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Table 1

Table 2

Table 3

References 41

1	陈凤东, 洪炳镕. 基于特征地图的移动机器人全局定位与自主泊位方法[J]. 电子学报, 2010, 38 (6): 1256- 1261.
	CHEN F D , HONG B R . A global localization and self-docking method for mobile robot based on feature map[J]. Acta Electronica Sinica, 2010, 38 (6): 1256- 1261.
2	王晓龙, 刘海颖, 王景琪. 基于分层SLAM的空地多智能体协同导航[J]. 系统工程与电子技术, 2020, 42 (1): 166- 171.
	WANG X L , LIU H Y , WANG J Q . Collaborative navigation of air-ground multi-agent based on hierarchical SLAM[J]. Systems Engineering and Electronics, 2020, 42 (1): 166- 171.
3	陈宝华, 邓磊, 陈志祥, 等. 基于即时稠密三维重构的无人机视觉定位[J]. 电子学报, 2017, 45 (6): 1294- 1300. doi: 10.3969/j.issn.0372-2112.2017.06.003
	CHEN B H , DENG L , CHEN Z X , et al. Instant dense 3D reconstruction-based UAV vision localization[J]. Acta Electronica Sinica, 2017, 45 (6): 1294- 1300. doi: 10.3969/j.issn.0372-2112.2017.06.003
4	王通典, 刘洁瑜, 吴宗收, 等. 基于多尺度光流融合特征点视觉-惯性SLAM方法[J]. 系统工程与电子技术, 2022, 44 (3): 977- 985.
	WANG T D , LIU J Y , WU Z S , et al. Visual-inertial SLAM method based on multi-scale optical flow fusion feature point[J]. Systems Engineering and Electronics, 2022, 44 (3): 977- 985.
5	GUI J J , GU D B , WANG S , et al. A review of visual inertial odometry from filtering and optimisation perspectives[J]. Advanced Robotics, 2015, 29 (20): 1289- 1301. doi: 10.1080/01691864.2015.1057616
6	ZUO X X, XIE X J, LIU Y, et al. Robust visual SLAM with point and line features[C]//Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017: 1775-1782.
7	HE Y J , ZHAO J , GUO Y , et al. PL-VIO: tightly coupled monocular visual-inertial odometry using point and line features[J]. Sensors, 2018, 18 (4): 1-25, 1159. doi: 10.1109/JSEN.2017.2782543
8	赵良玉, 金瑞, 朱叶青, 等. 基于点线特征融合的双目惯性SLAM算法[J]. 航空学报, 2021, 42 (12): 325117.
	ZHAO L Y , JIN R , ZHU Y Q , et al. Stereo visual-inertial SLAM with point and line features[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42 (12): 325117.
9	LEE G H, FRAUNDORFER F, POLLEFEYS M. MAV visual SLAM with plane constraint[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2011: 3139-3144.
10	李海丰, 胡遵河, 陈新伟. PLP-SLAM: 基于点、线、面特征融合的视觉SLAM方法[J]. 机器人, 2017, 39 (2): 214- 220.
	LI H F , HU Z H , CHEN X W . PLP-SLAM: a visual SLAM method based on point-line-plane feature fusion[J]. Robot, 2017, 39 (2): 214- 220.
11	孙沁璇, 苑晶, 张雪波, 等. PLVO: 基于平面和直线融合的RGB-D视觉里程计[J]. 自动化学报, 2023, 49 (10): 1001- 1013.
	SUN Q X , YUAN J , ZHANG X B , et al. PLVO: plane-line-based RGB-D visual odometry[J]. Acta Automatica Sinica, 2023, 49 (10): 1001- 1013.
12	CAMPOSECO F, POLLEFEYS M. Using vanishing points to improve visual-inertial odometry[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2015: 5219-5225.
13	KIM P, COLTIN B. Low-drift visual odometry in structured environments by decoupling rotational and translational motion[C]// Proc. of the IEEE International Conference on Robotics and Automation, 2018: 7247-7253.
14	GUO R , PENG K , ZHOU D , et al. Robust visual compass using hybrid features for indoor environments[J]. Electronics, 2019, 8 (2): 220. doi: 10.3390/electronics8020220
15	KNEIP L, WEISS S, SIEGWART R. Deterministic initialization of metric state estimation filters for loosely-coupled mono-cular vision-inertial systems[C]//Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011: 2235-2241.
16	WEISS S, SIEGWART R. Real-time metric state estimation for modular vision-inertial systems[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2011: 4531-4537.
17	BLOESCH M , BURRI M , OMARI S , et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. The International Journal of Robotics Research, 2017, 36 (10): 1053- 1072. doi: 10.1177/0278364917728574
18	LEUTENEGGER S , LYNEN S , BOSSE M , et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. The International Journal of Robotics Research, 2015, 34 (3): 314- 334.
19	LI M , MOURIKIS A I . High-precision, consistent EKF-based visual-inertial odometry[J]. The International Journal of Robotics Research, 2013, 32 (6): 690- 711.
20	BLOESCH M , BURRI M , OMARI S , et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. The International Journal of Robotics Research, 2017, 36 (10): 1053- 1072.
21	LEUTENEGGER S , LYNEN S , BOSSE M , et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. The International Journal of Robotics Research, 2015, 34 (3): 314- 334.
22	CAMPOS C , ELVIRA R , RODRÍGUEZ J J G , et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Trans.on Robotics, 2021, 37 (6): 1874- 1890.
23	FORSTER C , CARLONE L , DELLAERT F , et al. On-Manifold preintegration for real-time visual-inertial odometry[J]. IEEE Trans.on Robotics, 2017, 33 (1): 1- 21.
24	LU X H, J Y Y, LI H, et al. 2-Line exhaustive searching for real-time vanishing point estimation in Manhattan world[C]// Proc. of the IEEE Winter Conference on Applications of Computer Vision, 2017: 345-353.
25	BAZIN J C , DEMONCEAUX C , VASSEUR P , et al. Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment[J]. The International Journal of Robotics Research, 2012, 31 (1): 63- 81.
26	DENIS P, ELDER J H, ESTRADA F J. Efficient edge based methods for estimating Manhattan frames in urban imagery[C]//Proc. of the European Conference on Computer Vision, 2008: 97-210.
27	BAZIN J C, POLLEFEYS M. 3-line RANSAC for orthogonal vanishing point detection[C]//Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012: 4282-4287.
28	MIRZAEI F M, ROUMELIOTIS S I. Optimal estimation of vanishing points in a manhattan world[C]// Proc. of the IEEE International Conference on Computer Vision, 2011: 2454-2461.
29	WILDENAUER H, HANBURY A. Robust camera self-calibration from monocular images of Manhattan worlds[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2012: 2831-2838.
30	TRETYAK E, BARINOVA O, KOHLI P, et al. Geometric image parsing in man-made environments[C]//Proc. of the International Journal of Computer Vision, 2012, 97(3): 305-321.
31	ZHOU H Z , ZOU D P , PEI L , et al. StructSLAM: visual SLAM with building structure lines[J]. IEEE Trans.on Vehi-cular Technology, 2015, 64 (4): 1364- 1375.
32	ZOU D P , WU Y X , PEI L , et al. StructVIO: visual-inertial odometry with structural regularity of man-made environments[J]. IEEE Trans.on Robotics, 2019, 35 (4): 999- 1013.
33	GOMEZ-OJEDA R , MORENO F A , ZUNIGA-NOEL D , et al. PL-SLAM: a stereo SLAM system through the combination of points and line segments[J]. IEEE Trans.on Robotics, 2019, 35 (3): 734- 746.
34	LI H A, YAO J, BAZIN J, et al. A monocular SLAM system leveraging structural regularity in Manhattan world[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2018: 2518-2525.
35	KIM P, COLTIN B. Low-drift visual odometry in structured environments by decoupling rotational and translational motion[C]// Proc. of the IEEE International Conference on Robotics and Automation, 2018: 7247-7253.
36	LUCAS B, KANADE T. An iterative image registration technique with an application to stereo vision[C]//Proc. of the International Joint Conference on Artificial Intelligence, 1981: 674-679.
37	AKINLAR C , TOPAL C . EDLINES: a real-time line segment detector with a false detection control[J]. Pattern Recognition Letters, 2011, 32 (13): 1633- 1642.
38	QIN T , LI P , SHEN S J . VINS-Mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Trans.on Robotics, 2018, 34 (4): 1004- 1020.
39	BARTOLI A , STURM P . Structure-from-motion using lines: representation, triangulation, and bundle adjustment[J]. Computer Vision and Image Understanding, 2005, 100 (3): 416- 441.
40	BURRI M , NIKOLIC J , GOHL P , et al. The EuRoC micro aerial vehicle datasets[J]. International Journal of Robotics Research, 2016, 35 (10): 1157- 1163.
41	GENEVA P, ECKENHOFF K, LEE W, et al. Openvins: a research platform for visual-inertial estimation[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2020: 4666-4672.

序列	OpenVINS		PL-VIO		MF-VIO		FMF-VIO
序列	平移/m	旋转/rad	平移/m	旋转/rad	平移/m	旋转/rad	平移/m	旋转/rad
MH_01_easy	/	/	0.139 788¹	0.026 952	0.172 364²	0.024 544²	0.175 494	0.023 975¹
MH_02_easy	/	/	0.141 971	0.043 020	0.136 833²	0.038 554¹	0.136 259¹	0.038 887²
MH_03_medium	/	/	0.264 665	0.042 052	0.260 564¹	0.037 934¹	0.263 443²	0.038 871²
MH_04_difficult	/	/	0.362 234	0.040 540	0.329 141²	0.039 593¹	0.325 993¹	0.039 655²
MH_05_ difficult	/	/	0.277 990	0.026 338	0.269 991¹	0.025 451²	0.270 585²	0.022 942¹
V1_01_easy	0.064¹	0.020 926¹	0.079 288	0.143 704	0.068 954¹	0.141 590²	0.070 522	0.141 952
V1_03_difficult	0.202	0.103 568²	0.202 742	0.085 312	0.114 108¹	0.112 057	0.181 205²	0.082 737¹
V2_01_easy	0.073	0.027 663¹	0.087 473	0.053 186	0.059 390²	0.029 635	0.058 855¹	0.029 366²
V2_02_medium	0.079¹	0.041 364¹	0.147 834	0.073 479²	0.141 045²	0.077 073	0.141 081	0.077 313
V2_03_ difficult	/	/	0.308 307	0.077 721	0.214 972¹	0.037 335¹	0.257 571²	0.044 254²

序列	MF	GMF	G2MF
MH_01_easy	63.320 76	12.049 29	9.625 10
MH_02_easy	59.049 39	13.145 60	8.789 46
MH_03_ medium	57.212 60	11.961 64	9.586 75
MH_04_difficult	58.012 88	11.695 74	8.921 13
MH_05_ difficult	57.783 94	12.289 53	8.775 61
V1_01_easy	58.892 84	10.977 85	8.411 83
V1_03_difficult	56.624 59	10.368 42	9.129 10
V2_01_easy	56.284 12	9.939 593	9.404 82
V2_02_medium	58.044 27	11.710 93	9.303 44
V2_03_ difficult	58.629 07	10.307 59	9.595 35

模型	操作	PL-VIO	FMF-VIO
前端	点特征提取与匹配	18.729 40	17.150 06
	线段特征提取与匹配	122.447 10	36.727 67
	结构特征提取与匹配	-	8.775 61
后端	非线性优化	30.827 74	33.498 91
后端	边缘化	5.100 24	8.137 24

[1]	Keke LU, Chao WANG, Junfeng WU, Feng QIAN. Visual-inertial odometry based on exposure controlled by gradient information [J]. Systems Engineering and Electronics, 2023, 45(5): 1496-1502.
[2]	Guofeng ZHOU, Zhe CHEN, Rui LYU, Haifeng TU, Shan XU. Virtual IMU and turntable attitude compensation for HITL simulation [J]. Systems Engineering and Electronics, 2022, 44(4): 1329-1335.
[3]	Tongdian WANG, Jieyu LIU, Zongshou WU, Qiang SHEN, Erliang YAO. Visual-inertial SLAM method based on multi-scale optical flow fusion feature point [J]. Systems Engineering and Electronics, 2022, 44(3): 977-985.
[4]	Lei YANG, Huijuan LI, Bo HUANG, Wei LIU, Pucheng LI. High resolution SAR imagery with structural feature enhancement under two-layer sparse group Lasso [J]. Systems Engineering and Electronics, 2021, 43(2): 351-362.
[5]	Leiyu CHEN, Zhijie ZHOU, Shuaiwen TANG, You CAO. Weapon equipment performance evaluation method with multi-information fusion [J]. Systems Engineering and Electronics, 2020, 42(7): 1527-1533.
[6]	Yueyang BEN, Xiaofeng WEI, Qianqian GAO, Xianpeng ZHAO, Ping'an DAI, Yan SUN. Error analysis and compensation based on ship heave measurement of SINS [J]. Systems Engineering and Electronics, 2020, 42(7): 1590-1596.
[7]	Xinhao DONG, Zhijie ZHOU, Youmin ZHANG, Zhichao FENG, You CAO. Forecasting method of the error coefficient for SIMU based on belief rule base [J]. Systems Engineering and Electronics, 2020, 42(12): 2867-2874.
[8]	Xiaolong WANG, Haiying LIU, Jingqi WANG. Collaborative navigation of air-ground multi-agent based on hierarchical SLAM [J]. Systems Engineering and Electronics, 2020, 42(1): 166-171.
[9]	WEI Heng, L Qiang, LIN Huican, ZHANG Yang, LIANG Jian. Survey on multi-robot SLAM back-end optimization algorithm [J]. Systems Engineering and Electronics, 2017, 39(11): 2553-2565.
[10]	DONG Chun-mei, REN Shun-qing, CHEN Xi-jun. IMU calibration method based on analysis for the errors of three-axis turntable [J]. Systems Engineering and Electronics, 2016, 38(4): 895-901.
[11]	LIU Dan, DUAN Jianmin, YU Hongxiao. FastSLAM algorithm based on adaptive fading extended Kalman filter [J]. Systems Engineering and Electronics, 2016, 38(3): 644-651.
[12]	PEI Fu-jun, CHENG Yu-hang, LI Hao-yang, JU He-hua. Distributed simultaneous localization and mapping algorithm based on partition of space-region [J]. Systems Engineering and Electronics, 2015, 37(3): 639-645.
[13]	LIU Ming-yong, DONG Ting-ting, ZHANG Li-chuan. Underwater SLAM navigation algorithm based on random beacons [J]. Systems Engineering and Electronics, 2015, 37(12): 2830-2834.
[14]	XIA Wei-xing， YANG Xiao-dong. High accuracy estimation algorithm for IMU error of under water INS [J]. Systems Engineering and Electronics, 2013, 35(10): 2176-2185.
[15]	DU Hang-yuan, ZHAO Yu-xin, YANG Yong-peng, HAN Qing-nan. SLAM algorithm based on random finite set [J]. Journal of Systems Engineering and Electronics, 2012, 34(7): 1452-1457.

A monocular visual inertial odometry based on structural features

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 41

Related Articles 15

Recommended Articles

Metrics

Comments