Real-time tracking of fast-moving object in occlusion scene

doi:10.23919/JSEE.2024.000058

Journal of Systems Engineering and Electronics ›› 2024, Vol. 35 ›› Issue (3): 741-752.doi: 10.23919/JSEE.2024.000058

• CONTROL THEORY AND APPLICATION • Previous Articles

Real-time tracking of fast-moving object in occlusion scene

Yuran LI¹^,²(), Yichen LI¹^,²(), Monan ZHANG¹^,²(), Wenbin YU¹^,²^,*(), Xinping GUAN¹^,²()

¹ Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China
² Key Laboratory of Systems Control and Information Processing, Ministry of Education of China, Shanghai 200240, China

Received:2023-10-27 Online:2024-06-18 Published:2024-06-19
Contact: Wenbin YU E-mail:liyuran20000220@sjtu.edu.cn;liyichensjtu@sjtu.edu.cn;mnzhang@sjtu.edu.cn;yuwenbin@sjtu.edu.cn;xpguan@sjtu.edu.cn
About author:
LI Yuran was born in 2000. She received her B.E. degree in automation from Xi’an Jiaotong University, Xi’an, China, in 2021. She is currently pursuing her M.E. degree in control science and engineering at Shanghai Jiaotong University, Shanghai, China. Her current research interests include computer vision and deep learning. E-mail: liyuran20000220@sjtu.edu.cn

LI Yichen was born in 1993. He received his B.S. degree in detection, guidance and control technology from Northwestern Polytechnical University, Xi’an, China, in 2016 and Ph.D. degree in control science and engineering from Shanghai Jiao Tong University, Shanghai, China, in 2022. He is now a postdoc in control science and engineering with Shanghai Jiao Tong University, Shanghai, China. His current research interests include underwater multi-robot localization and trajectory planning, wireless networks, and information fusion. E-mail: liyichensjtu@sjtu.edu.cn

ZHANG Monan was born in 1993. He received his B.E. degree in measurement and control technology and instrumentation from Harbin Engineering University, Harbin, China, in 2016, and M.E. degree in aeronautical and astronautical science and technology from Harbin Institute of Technology, Harbin, China, in 2018. He is currently pursuing his Ph.D. degree in control science and engineering at Shanghai Jiao Tong University, Shanghai, China. His current research interests include target detection and tracking in computer vision. E-mail: mnzhang@sjtu.edu.cn

YU Wenbin Was born in 1983. He received his Ph.D. degree in control science and engineering from Shanghai Jiao Tong University, Shanghai, China, in 2016. He is currently an associate professor with the Department of Automation, Shanghai Jiao Tong University. His research interests include data fusion and control strategy for AUV system. E-mail: yuwenbin@sjtu.edu.cn

GUAN Xinping was born in 1963. He received his Ph.D. degree in control and systems from Harbin Institute of Technology, Harbin, China in 1999. In 2007, he joined the Department of Automation, Shanghai Jiao Tong University, Shanghai, China, where he is currently a Distinguished University Professor and the Director of the Key Laboratory of Systems Control and Information Processing, Ministry of Education of China. His current research interests include wireless sensor networks, ground-air communication of aircraft, and cognitive radio networks and their applications in industry. E-mail: xpguan@sjtu.edu.cn
Supported by:
This work was supported by the National Nature Science Foundation of China (62373246;62203299), the Oceanic Interdisciplinary Program of Shanghai Jiao Tong University (SL2022MS008;SL2020ZD206; SL2022MS010).

Abstract

Abstract:

Tracking the fast-moving object in occlusion situations is an important research topic in computer vision. Despite numerous notable contributions have been made in this field, few of them simultaneously incorporate both object’s extrinsic features and intrinsic motion patterns into their methodologies, thereby restricting the potential for tracking accuracy improvement. In this paper, on the basis of efficient convolution operators (ECO) model, a speed-accuracy-balanced model is put forward. This model uses the simple correlation filter to track the object in real-time, and adopts the sophisticated deep-learning neural network to extract high-level features to train a more complex filter correcting the tracking mistakes, when the tracking state is judged to be poor. Furthermore, in the context of scenarios involving regular fast-moving, a motion model based on Kalman filter is designed which greatly promotes the tracking stability, because this motion model could predict the object’s future location from its previous movement pattern. Additionally, instead of periodically updating our tracking model and training samples, a constrained condition for updating is proposed, which effectively mitigates contamination to the tracker from the background and undesirable samples avoiding model degradation when occlusion happens. From comprehensive experiments, our tracking model obtains better performance than ECO on object tracking benchmark 2015 (OTB100), and improves the area under curve (AUC) by about 8% and 32% compared with ECO, in the scenarios of fast-moving and occlusion on our own collected dataset.

Key words: speed-accuracy balanced, motion modeling, constrained updater

Yuran LI, Yichen LI, Monan ZHANG, Wenbin YU, Xinping GUAN. Real-time tracking of fast-moving object in occlusion scene[J]. Journal of Systems Engineering and Electronics, 2024, 35(3): 741-752.

Figures/Tables 12

Fig 1

Fig 2

Fig 3

Fig 4

Table 1

Table 2

Fig 5

Fig 6

Fig 7

Fig 8

Fig 9

Fig 10

References 31

1	KHATTAK A S, RAJA G, ANJUM N, et al. Integration of mean-shift and particle ﬁlter: a survey. Proc. of the 12th International Conference on Frontiers of Information Technology, 2014: 286−291.
2	GHEDIA N, VITHALANI C, KOTHARI A M, et al. Existing research in video surveillance system. https://doi.org/10.1007/978-3-030-90910-9_2.
3	ZHOU N, LAU L, BAI R, et al A genetic optimization resampling based particle ﬁltering algorithm for indoor target tracking. Remote Sensing, 2021, 13 (1): 132- 154. doi: 10.3390/rs13010132
4	WAILA G S, KUMAR A, SAXENA A, et al Robust object tracking with crow search optimized multi-cue particle ﬁlter. Pattern Analysis and Applications, 2020, 23, 1439- 1455. doi: 10.1007/s10044-019-00847-7
5	ZHANG L A survey of target tracking algorithms based on correlation ﬁltering. International Core Journal of Engineering, 2022, 8 (4): 566- 576.
6	DU S D, WANG S P An overview of correlation-ﬁlter-based object tracking. IEEE Trans. on Computational Social Systems, 2021, 9 (1): 18- 31.
7	HENRIQUES J F, CASEIRO R, MARTINS P, et al High-speed tracking with kernelized correlation ﬁlters. IEEE Trans on Pattern Analysis and Machine Intelligence, 2014, 37 (3): 583- 596.
8	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation ﬁlters. Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010: 2544−2550.
9	BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple: complementary learners for real-time tracking. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1401−1409.
10	MA C, YANG X K, ZHANG C Y, et al. Long-term correlation tracking. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5388–5396.
11	DANELLJAN M, HAGER G, SHAHBAZ K, et al. Learning spatially regularized correlation filters for visual tracking. Proc. of the IEEE International Conference on Computer Vision, 2015: 4310−4318.
12	DANELLJAN M, BHAT G, SHAHBAZ K, et al. Eco: efﬁcient convolution operators for tracking. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6638−6646.
13	LIU S, LIU D Y, SRIVASTAVA G, et al Overview and methods of correlation ﬁlter algorithms in object tracking. Complex & Intelligent Systems, 2021, 7, 1895- 1917.
14	ZHAO S K, SUN K W, JI Y F, et al. Correlation filter-based object tracking algorithms. Proc. of the IEEE International Conference on Information Communication and Signal Processing, 2020: 57−62
15	WANG D C, BAI C S, WU K J Survey of video object detection based on deep learning. Journal of Frontiers of Computer Science & Technology, 2021, 15 (9): 1563- 1578.
16	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking. Proc. of the Computer Vision-ECCV 2016 Workshops, 2016: 850−865.
17	VOIGTLAENDER P, LUITEN J, TORR P H S, et al. Siam R-CNN: visual tracking by re-detection. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6578−6588.
18	LI B, WU W, WANG Q, et al. Siamrpn++: evolution of siamese visual tracking with very deep networks. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4282−4291.
19	WANG N, ZHOU W G, WANG J, et al. Transformer meets tracker: exploiting temporal context for robust visual tracking. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 1571−1580.
20	DANELLJAN M, BHAT G, KHAN F S, et al. Atom: accurate tracking by overlap maximization. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4660−4669.
21	BHAT G, DANELLJAN M, GOOL L V, et al. Learning discriminative model prediction for tracking. Proc. of the IEEE/CVF International Conference on Computer Vision, 2019: 6182−6191.
22	WU Y, LIM J, YANG M H. Online object tracking: a benchmark. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2013: 2411−2418.
23	WU Y, LIM J, YANG M H Object tracking benchmark. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1834- 1848. doi: 10.1109/TPAMI.2014.2388226
24	KRISTAN M, MATAS J, LEONARDIS A, et al. The ninth visual object tracking vot2021 challenge results. Proc. of the IEEE/CVF International Conference on Computer Vision, 2021: 2711−2738.
25	MUELLER M, SMITH N, GHANEM B. A benchmark and simulator for uav tracking. Proc. of the Computer Vision-ECCV 2016 Workshops, 2016: 445−461.
26	JIANG M, LI R, LIU Q S, et al High speed long-term visual object tracking algorithm for real robot systems. Neurocomputing, 2021, 434, 268- 284. doi: 10.1016/j.neucom.2020.12.113
27	MA C, HUANG J B, YANG X, et al Adaptive correlation ﬁlters with long-term and short-term memory for object tracking. International Journal of Computer Vision, 2018, 126, 771- 796. doi: 10.1007/s11263-018-1076-4
28	YANG Z F, CHEN X, Optimized searching strategy for kcf object tracking algorithm. Journal of Wuhan Institute of Technology, 2019, 41(1): 98−102. (in Chinese)
29	DANELLJAN M, ROBINSON A, SHAHBAZ K F, et al. Beyond correlation ﬁlters: learning continuous convolution operators for visual tracking. Proc. of the Computer Vision-ECCV 2016 Workshops, 2016: 472−488.
30	LI X, ZHOU J L, HOU J Q, et al Research on improved moving object tracking method based on eco-hc. Journal of Nanjing University (Natural Science), 2020, 56 (2): 216- 226.
31	WANG N Y, SHI J P, YEUNG D Y, et al. Understanding and diagnosing visual tracking systems. Proc. of the IEEE International Conference on Computer Vision, 2015: 3101−3109.

Speed-accuracy-balanced model	Constrained model updater	Motion model
Deep-HC	Renew	Kalman filter

Scene	Module	AUC	Scene	Module	AUC
MB	Deep-HC-renew	0.626	FM	Deep-HC-renew	0.633
	Deep-HC	0.617		Deep-HC	0.625
	Deep-HC-renew-KF	0.614		ECO-HC	0.621
	ECO-HC	0.610		Deep-HC-renew-KF	0.608
IPR	Deep-HC-renew-KF	0.549	OPR	Deep-HC-renew	0.567
	Deep-HC-renew	0.539		Deep-HC	0.556
	Deep-HC	0.532		ECO-HC	0.556
	ECO-HC	0.530		Deep-HC-renew-KF	0.552
BC	Deep-HC-renew-KF	0.631	LR	ECO-HC	0.594
	Deep-HC-renew	0.627		Deep-HC-renew-KF	0.594
	Deep-HC	0.611		Deep-HC-renew	0.592
	ECO-HC	0.583		Deep-HC	0.586
OCC	Deep-HC-renew-KF	0.574	IV	Deep-HC-renew-KF	0.616
	Deep-HC-renew	0.574		Deep-HC-renew	0.609
	Deep-HC	0.562		Deep-HC	0.595
	ECO-HC	0.562		ECO-HC	0.588
OV	Deep-HC-renew-KF	0.548	SV	Deep-HC-renew-KF	0.594
	Deep-HC-renew	0.530		ECO-HC	0.593
	ECO-HC	0.526		Deep-HC-renew	0.592
	Deep-HC	0.525		Deep-HC	0.587
	Deep-HC-renew	0.615		Deep-HC-renew	0.568
all	Deep-HC-renew-KF	0.612	DEF	Deep-HC	0.564
	Deep-HC	0.609		ECO-HC	0.558
	ECO-HC	0.604		Deep-HC-renew-KF	0.554

Real-time tracking of fast-moving object in occlusion scene

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 12

References 31

Related Articles 0

Recommended Articles

Metrics

Comments