A fast, accurate and dense feature matching algorithm for aerial images

doi:10.23919/JSEE.2020.000085

Journal of Systems Engineering and Electronics ›› 2020, Vol. 31 ›› Issue (6): 1128-1139.doi: 10.23919/JSEE.2020.000085

• ELECTRONICS TECHNOLOGY • Previous Articles Next Articles

A fast, accurate and dense feature matching algorithm for aerial images

Ying LI(), Guanghong GONG*(), Lin SUN()

¹ School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China

Received:2019-07-18 Online:2020-12-18 Published:2020-12-29
Contact: Guanghong GONG E-mail:1193126485@qq.com;ggh@ggh.buaa.edu.cn;2019@163.com
About author:|LI Ying was born in 1995. She received her B.S. degree from Beijing Forestry University in 2017. Then she entered Beihang University for her M.S. degree and became a doctoral candidate in 2019. She is pursuing her Ph.D. degree in navigation, guidance and control from Beihang University, China. Her research interests include image processing, environment modeling and simulation, virtual reality and augmented reality. E-mail: 1193126485@qq.com||GONG Guanghong was born in 1968. She received her B.S., M.S. and Ph.D. degrees from Beihang University, in 1990, 1993 and 1997, respectively. She is currently a professor in the Key Laboratory of Advanced Simulation Aeronautical Technology, Beihang University. She is also an executive director of the China Simulation Society and a director of the Environmental Modeling and Simulation Professional Committee. She has won the Science and Technology Progress Award of the Ministry of Education and the National Defense Science and Technology Progress Award for many times. Her research interests are virtual reality and distributed interactive simulation. E-mail: ggh@buaa.edu.cn||SUN Lin was born in 1995. He received his B.S. and M.S. degrees from Beihang University. During his master’s degree program, his main research direction is terrain 3D reconstruction. His research interests include virtual reality and environment simulation. E-mail: sunlin 2019@163.com
Supported by:
This work was supported by the Equipment Pre-Research Foundation of China (6140001020310)

Abstract

Abstract:

Three-dimensional (3D) reconstruction based on aerial images has broad prospects, and feature matching is an important step of it. However, for high-resolution aerial images, there are usually problems such as long time, mismatching and sparse feature pairs using traditional algorithms. Therefore, an algorithm is proposed to realize fast, accurate and dense feature matching. The algorithm consists of four steps. Firstly, we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution. Secondly, to realize further screening of the mismatches, a feature screening algorithm based on similarity judgment or local optimization is proposed. Thirdly, to make the algorithm more widely applicable, we combine the results of different algorithms to get dense results. Finally, all matching feature pairs in the low-resolution images are restored to the original images. Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time, screen out the mismatches, and improve the number of matches.

Key words: feature matching, feature screening, feature fusion, aerial image, three-dimensional (3D) reconstruction

Ying LI, Guanghong GONG, Lin SUN. A fast, accurate and dense feature matching algorithm for aerial images[J]. Journal of Systems Engineering and Electronics, 2020, 31(6): 1128-1139.

Figures/Tables 12

Fig 1

Table 1

Table 2

Fig 2

Fig 3

Fig 4

Fig 5

Table 3

Table 4

Table 5

Fig 6

Table 6

References 37

1	LIU T Comparison of software Inpho, PhotoScan and Pix4D for UAV orthophoto processing. Technology and Industry Across the Straits, 2017, 221 (11): 82- 84.
2	BENTLEY. Context capture: create 3D models from simple photographs. https://www.bentley.com/en/products/brands/contextcapture.
3	LOWE D G Distinctive image feature from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
4	BAY H, TUYTELAARS T, GOOL L J V. SURF: speeded up robust features. Proc. of the European Conference on Computer Vision, 2006: 404–417.
5	ROSTEN E, DRUMMOND T. Machine learning for high-speed corner detection. Proc. of the European Conference on Computer Vision, 2006: 430–443.
6	RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF. Proc. of the IEEE International Conference on Computer Vision, 2011: 2564–2571.
7	LEUTENEGGER S, CHLI M, SIEGWART R Y. Brisk: binary robust invariant scalable keypoints. Proc. of the IEEE International Conference on Computer Vision, 2011, 58(11): 2548–2555.
8	ALAHI A, ORTIZ R, VANDERGHEYNST P. FREAK: fast retina keypoint. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2012: 510–517.
9	PABLO F A, ADRIEN B, ANDREW J D. KAZE features. Proc. of the European Conference on Computer Vision, 2012: 214–227.
10	PABLO F A, JESUS N, ADRIEN B. Fast explicit diffusion for accelerated features in nonlinear scale spaces. Proc. of the British Machine Vision Conference, 2013: 1–12.
11	TAREEN S A K, SALEEM Z. A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. Proc. of the IEEE International Conference on Computing, Mathematics and Engineering Technologies, 2018: 1–10.
12	LAM S K, LIM T C, WU M, et al Data-path unrolling with logic folding for area-time-efficient FPGA-based FAST corner detector. Journal of Real Time Image Processing, 2019, 16 (6): 2147- 2158. doi: 10.1007/s11554-017-0725-0
13	CHEN S J, ZHENG S Z, XU Z G, et al An improved image matching method based on SURF algorithm. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, 42 (3): 179- 184.
14	ZHU J T, GONG C F, ZHAO M X, et al Image mosaic algorithm based on PCA-ORB feature matching. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, 42 (3): 83- 89.
15	GUO S L, HAN L N, HAO X T GW28-e0419 image detection scale-invariant feature transform algorithm based on feature matching improves image matching accuracy. Journal of the American College of Cardiology, 2017, 70 (16): C10.
16	LI J Y, HU Q W, AI M Y RIFT: multi-modal image matching based on radiation-variation insensitive feature transform. IEEE Trans. on Image Processing, 2020, 29, 3296- 3310. doi: 10.1109/TIP.2019.2959244
17	YU X Y, LEI H, GAO W, et al. Image matching algorithm with color information based on SIFT. Proc. of the 10th International Conference on Digital Image Processing, 2018. DOI:10.1117/12.2502820.
18	LI Q L, WANG G Y, LIU J G, et al Robust scale-invariant feature matching for remote sensing image registration. IEEE Geoscience and Remote Sensing Letters, 2009, 6 (2): 287- 291. doi: 10.1109/LGRS.2008.2011751
19	MA C, HU X, XIAO J, et al Improved ORB algorithm using three-patch method and local gray difference. Sensors, 2020, 20 (4): 975. doi: 10.3390/s200409751
20	WU Y Research on feature point extraction and matching machine learning method based on light field imaging. Neural Computing and Applications, 2019, 31 (12): 8157- 8169. doi: 10.1007/s00521-018-3962-7
21	ZHANG Y, LI C, CAO C Q, et al. An improved ORB feature point matching algorithm. Proc. of the 2nd International Conference on Computer Science and Artificial Intelligence, 2018: 207–211.
22	LI J Y, HU Q W, AI M Y 4FP-structure: a robust local region feature descriptor. Photogrammetric Engineering and Remote Sensing, 2017, 83 (12): 813- 826. doi: 10.14358/PERS.83.12.813
23	ZHU S Z, LIU L Z, CHEN S Image feature detection algorithm based on the spread of Hessian source. Multimedia Systems, 2017, 23 (1): 105- 117. doi: 10.1007/s00530-015-0453-x
24	HARTMANN W, HAVLENA M, SCHINDLER K Recent developments in large-scale tie-point matching. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 115, 47- 62. doi: 10.1016/j.isprsjprs.2015.09.005
25	XI W F, SHI Z T, LI D S Comparisons of feature extraction algorithm based on unmanned aerial vehicle image. Open Physics, 2017, 15 (1): 472- 478. doi: 10.1515/phys-2017-0053
26	AMIN S, NAIILA M Uniform competency-based local feature extraction for remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 135, 142- 157. doi: 10.1016/j.isprsjprs.2017.11.019
27	LI M M, XIE W F Remote sensing image matching algorithm for coastal zone based on local features. Journal of Coastal Research, 2019, 93 (sp1): 723- 728. doi: 10.2112/SI93-100.1
28	YUAN X X, CHEN S Y, YUAN W, et al Poor textural image tie point matching via graph theory. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 129, 21- 31. doi: 10.1016/j.isprsjprs.2017.04.015
29	WU Y D, MING Y, TAN H An efficient hyper-graph based tie point matching method for aerial imagery. Remote Sensing Letters, 2020, 11 (4): 313- 322. doi: 10.1080/2150704X.2020.1714773
30	CHEN S T, FENG R, ZHANG Y Y, et al Aerial image matching method based on HSI hash learning. Pattern Recognition Letters, 2019, 117, 131- 139. doi: 10.1016/j.patrec.2018.06.008
31	SONG W H, JUNG H G, GWAK I Y, et al Oblique aerial image matching based on iterative simulation and homography evaluation. Pattern Recognition, 2019, 87, 317- 331. doi: 10.1016/j.patcog.2018.10.027
32	MARTIN A F, ROBERT C B Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography. Communications of Association for Computing Machinery, 1981, 24 (6): 381- 395. doi: 10.1145/358669.358692
33	GUI Y, SU A, DU J Point-pattern matching method using SURF and shape context. Optik International Journal for Light and Electron Optics, 2013, 124 (14): 1869- 1873. doi: 10.1016/j.ijleo.2012.05.037
34	XI W F, LI D S, LI J S, et al. Research on an improved algorithm for eliminating gross error in UAV image mosaic. Proc. of the 26th International Conference on Geoinformatics, 2018: 1–5.
35	LI J Y, HU Q W, AI M Y LAM: locality affine-invariant feature matching. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 154, 28- 40. doi: 10.1016/j.isprsjprs.2019.05.006
36	GAO H W, JIANG Y Q, LIU J G, et al. Zooming image based false matches elimination algorithms for robot navigation. Advances in Mechanical Engineering, 2017, 9(12): 168781401773815.
37	GESTO-DIAZ M, TOMBARI F, GONZALEZAGUILERA D, et al Feature matching evaluation for multimodal correspondence. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 129, 179- 188. doi: 10.1016/j.isprsjprs.2017.05.007

Image resolution/ (pixels × pixels)	Key points’ number of a single picture (median)	Number of all tie points	Tie points’ number of a single picture (median)
1 840 × 1 228	6 693	35 388	1 229
3 680 × 2 456	24 254	32 609	1 007
7 360 × 4 912	43 455	23 924	828
14 720 × 9 824	43 549	13 859	400

Image resolution/(pixels × pixels)	Image result	Number	Time/s
3 680 × 2 456		(i) 71 751 (ii) 36	(iii) 335.27 (iv) 4.65
1 840 × 1 228		(i) 15 166 (ii) 36	(iii) 17.33 (iv) 1.21
920 × 614		(i) 3 848 (ii) 207	(iii) 2.93 (iv) 0.45
460 × 307		(i) 944 (ii) 164	(iii) 1.68 (iv) 0.28
230 × 153		(i) 232 (ii) 48	(iii) 1.21 (iv) 0.24

Algorithm	Image result	Number	Time/s
Original result		(i) 164 (ii) 164 (iii) 161 (iv) 3	–
PSNR		(i) 164 (ii) 151 (iii) 150 (iv) 1	(v) 0.691
SSIM		(i) 164 (ii) 87 (iii) 85 (iv) 2	(v) 0.676
Mean hash		(i) 164 (ii) 107 (iii) 105 (iv) 2	(v) 0.205
Local optimization		(i) 164 (ii) 118 (iii) 118 (iv) 0	(v) 1.355

Algorithm	Image result	Number of pairs	Time/s
SURF		118	4.353
ORB		107	4.758
SURF + ORB		221	10.446

Image resolution/(pixels × pixels)	Image result
460 × 307
3 680 × 2 456

A fast, accurate and dense feature matching algorithm for aerial images

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 12

References 37

Related Articles 4

Recommended Articles

Metrics

Comments

Method	Image result	SfM result	Time/s	Number
SURF + RANSAC			697	(i) 1 501 (ii) 1 500
The proposed algorithm			344	(i) 1 862 (ii) 1 862
SURF + RANSAC			262	(i) 103 (ii) 103
The proposed algorithm			113	(i) 447 (ii) 446
SURF + RANSAC			167	(i) 113 (ii) 113
The proposed algorithm			129	(i) 486 (ii) 486

[1]	Minggang Gan, Yulong Cheng, Yanan Wang, and Jie Chen. Hierarchical particle filter tracking algorithm based on multi-feature fusion [J]. Systems Engineering and Electronics, 2016, 27(1): 51-.
[2]	An Liu, Donghua Zhou, Lixin Chen, and Maoyin Chen. Saliency detection and edge feature matching approach for crater extraction [J]. Systems Engineering and Electronics, 2015, 26(6): 1291-1300.
[3]	Peng Wang, Qibin Zhang, and Zonghai Chen. Feature extension and matching for mobile robot global localization [J]. Journal of Systems Engineering and Electronics, 2015, 26(4): 840-.
[4]	Ma Yu, Gu Xiaodong & Wang Yuanyuan. Feature fusion method for edge detection of color images [J]. Journal of Systems Engineering and Electronics, 2009, 20(2): 394-399.