Modeling of UAV path planning based on IMM under POMDP framework

doi:10.21629/JSEE.2019.03.12

Journal of Systems Engineering and Electronics ›› 2019, Vol. 30 ›› Issue (3): 545-554.doi: 10.21629/JSEE.2019.03.12

• Systems Engineering • Previous Articles Next Articles

Modeling of UAV path planning based on IMM under POMDP framework

Qiming YANG(), Jiandong ZHANG*(), Guoqing SHI()

School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China

Received:2017-11-13 Online:2019-06-01 Published:2019-07-04
Contact: Jiandong ZHANG E-mail:yangqm@mail.nwpu.edu.cn;jdzhang@nwpu.edu.cn;shiguoqing@nwpu.edu.cn
About author:YANG Qiming was born in 1988. He received his M.S. degree in Northwestern Polytechnical University, Xi'an, China in 2013. Since 2016, he has been a Ph.D. candidate in electronic science and technology in Northwestern Polytechnical University. His main research interests are artificial intelligence and its application on control and decision of UAV. E-mail:yangqm@mail.nwpu.edu.cn|ZHANG Jiandong was born in 1974. He is an associate professor at the Department of System and Control Engineering in Northwestern Polytechnical University, China. He received both his M.S. and Ph.D. degrees in system engineering from the same university. His research fields and interests include modeling simulation and effectiveness evaluation of complex systems, development and design of integrated avionics system, and system measurement & test technologies. E-mail:jdzhang@nwpu.edu.cn|SHI Guoqing was born in 1974. He is an associate professor at the Department of System and Control Engineering in Northwestern Polytechnical University, China. He received his M.S. and Ph.D. degrees in system engineering from the same university. His research fields and interests include integrated avionics system measurement & test technologies, development and design of embedded realtime systems, modeling simulation and effectiveness evaluation of complex systems, etc. E-mail:shiguoqing@nwpu.edu.cn
Supported by:
the Aeronautical Science Foundation of China(20135153031);the Aeronautical Science Foundation of China(20135553035);the Aeronautical Science Foundation of China(2017ZC53033);This work was supported by the Aeronautical Science Foundation of China (20135153031; 20135553035; 2017ZC53033)

Abstract

Abstract:

In order to enhance the capability of tracking targets autonomously of unmanned aerial vehicle (UAV), the partially observable Markov decision process (POMDP) model for UAV path planning is established based on the POMDP framework. The elements of the POMDP model are analyzed and described. The state transfer law in the model can be described by the method of interactive multiple model (IMM) due to the diversity of the target motion law, which is used to switch the motion model to accommodate target maneuvers, and hence improving the tracking accuracy. The simulation results show that the model can achieve efficient planning for the UAV route, and effective tracking for the target. Furthermore, the path planned by this model is more reasonable and efficient than that by using the single state transition law.

Key words: partially observable Markov decision process (POMDP), interactive multiple model (IMM), filtering, path planning, target tracking, state transfer law

Qiming YANG, Jiandong ZHANG, Guoqing SHI. Modeling of UAV path planning based on IMM under POMDP framework[J]. Journal of Systems Engineering and Electronics, 2019, 30(3): 545-554.

Figures/Tables 7

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

References 38

1	CLAUDIA S, ROHAN B, FRANCESCO N, et al. Review of the current state of UAV regulations. Remote Sensing, 2017, 9 (5): 459. doi: 10.3390/rs9050459
2	CAI J W, LONG H Y, ZHANG X. Key technologies for cooperative combat system of manned/unmanned aerial vehicles. Command Information System and Technology, 2013, 4 (2): 10- 14.
3	ZHANG Y Z, ZHANG J D, SHI G Q. Research on modeling and simulation of UAV warning radar. Fire Control & Command Control, 2016, 41 (2): 93- 96.
4	RAGI S, CHONG E K P. UAV path planning in a dynamic environment via partially observable Markov decision process. IEEE Trans. on Aerospace & Electronic Systems, 2013, 49 (4): 2397- 2412.
5	JI X, LI J. Online motion planning for UAV under uncertain environment. Proc. of the IEEE International Symposium on Computational Intelligence and Design, 2016, 514- 517.
6	ALVAREZ F V. Uncertainty based online planning for UAV target finding in cluttered and GPS-denied environments. Proc. of the IEEE Aerospace Conference, 2016, 1- 9.
7	EATON C M, CHONG E K P, MACIEJEWSKI A A. Robust UAV path planning using POMDP with limited FOV sensor. Proc. of the IEEE Conference on Control Technology and Applications, 2017, 1530- 1535.
8	VIEN N A, TOUSSAINT M. POMDP manipulation via trajectory optimization. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015, 242- 249.
9	RAGI S, CHONG E K P. Decentralized control of unmanned aerial vehicles for multitarget tracking. Proc. of the International Conference on Unmanned Aircraft Systems, 2013, 260- 268.
10	CUI N, HONG L, LAYNE J R. A comparison of nonlinear filtering approaches with an application to ground target tracking. Signal Processing, 2005, 85 (8): 1469- 1492. doi: 10.1016/j.sigpro.2005.01.010
11	MILLER S A, HARRIS Z A, CHONG E K P. A POMDP framework for coordinated guidance of autonomous UAVs for multitarget tracking. EURASIP Journal on Advances in Signal Processing, 2009, 2009 (1): 1- 17.
12	CAPITAN J, MERINO L, OLLERO A. Cooperative decisionmaking under uncertainties for multi-target surveillance with multiples UAVs. Journal of Intelligent & Robotic Systems, 2016, 84 (1-4): 1- 16.
13	VANEGAS F, CAMPBELL D, ROY N, et al. UAV tracking and following a ground target under motion and localisation uncertainty. Proc. of the IEEE Aerospace Conference, 2017, 1- 10.
14	KRAKOW L W, CHONG E K P. Autonomous UAV control:balancing target tracking and persistent surveillance. Proc. of the IEEE Conference on Control Technology and Applications, 2017, 1524- 1529.
15	GUPTA A, BESSONOV D, LI P. A decision-theoretic approach to detection-based target search with a UAV. Proc. of the IEEE International Conference on Intelligent Robots and Systems, 2018, 5304- 5309. doi: 10.1109/IROS.2017.8206423
16	ZHAO Y, WANG X, KONG W, et al. Decision-making of UAV for tracking moving target via information geometry. Proc. of the IEEE Control Conference, 2016, 5611- 5617.
17	CHANEL C P C, TEICHTEILKÖNIGSBUCH F, LESIRE C. POMDP-based online target detection and recognition for autonomous UAVs. Proc. of the 7th Conference on Prestigious Applications of Intelligent Systems, 2012, 955- 960.
18	WAN K F, GAO X G, LI B, et al. Mission planning of passive networked sensors for cooperative anti-stealth detection based on POMDP. Acta Armamentarii, 2015, 36 (4): 731- 743.
19	RAGI S, CHONG EDWIN K P. Decentralized guidance control of UAVs with explicit optimization of communication. Journal of Intelligent & Robotic Systems, 2014, 73 (1): 811- 822.
20	WAN K, GAO X, BO L I, et al. Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets. Journal of Systems Engineering and Electronics, 2018, 29 (1): 74- 85.
21	ILHAN U, GARDASHOVA L, KILIC K. UAV using DecPOMDP model for increasing the level of security in the company. Procedia Computer Science, 2016, 102 (2016): 458- 464.
22	MORERE P, MARCHANT R, RAMOS F. Sequential Bayesian optimization as a POMDP for environment monitoring with UAVs. Proc. of the IEEE International Conference on Robotics and Automation, 2017, 6381- 6388.
23	VANEGAS F, GONZALEZ F. Enabling UAV navigation with sensor and environmental uncertainty in cluttered and GPSdenied environments. Sensors, 2016, 16 (5): No. 666. doi: 10.3390/s16050666
24	SONG H, XIAO M, XIAO J, et al. A POMDP approach for scheduling the usage of airborne electronic countermeasures in air operations. Aerospace Science & Technology, 2016, 48 (2016): 86- 93.
25	GOMES-BORGES M, MALTESE D, VANHEEGHE P, et al. A risk-based sensor management using random finite sets and POMDP. Proc. of the International Conference on Information Fusion, 2017, 1588- 1596.
26	CORK L, WALKER R. Sensor fault detection for UAVs using a nonlinear dynamic model and the IMM-UKF algorithm. Proc. of the Information, Decision and Control Conference, 2017, 230- 235.
27	ZANG R C, CUI P Y, CUI H T, et al. Integrated navigation algorithm based on IMM-UKF. Control Theory & Applications, 2007, 24 (4): 634- 638.
28	GONG S L, WANG B F, WU H L, et al. Tracking of moving targets on airport surface based on IMM algorithm. Systems Engineering and Electronics, 2011, 33 (10): 2322- 2326.
29	XU L, LI X R. Hybrid grid multiple-model estimation with application to maneuvering target tracking. IEEE Trans. on Aerospace and Electronic Systems, 2016, 52 (1): 122- 136. doi: 10.1109/TAES.2015.140423
30	WEI Z, WEI W, YUAN G. An improved interacting multiple model filtering algorithm based on the cubature Kalman filter for maneuvering target tracking. Sensors, 2016, 16 (6): No. 805. doi: 10.3390/s16060805
31	GAO B, GAO S, ZHONG Y, et al. Interacting multiple model estimation-based adaptive robust unscented Kalman filter. International Journal of Control Automation & Systems, 2017, 15 (2): 1- 13.
32	KHALID S S, ABRAR S. A low-complexity interacting multiple model filter for maneuvering target tracking. Proc. of the International Journal of Electronics and Communications, 2017, 73, 157- 164. doi: 10.1016/j.aeue.2017.01.011
33	WEN Q, ZHOU Y, HU L, et al. Comparison of filtering techniques for simultaneous localization and tracking. Proc. of the IEEE International Conference on Estimation, Detection and Information Fusion, 2015, 387- 392.
34	XU S, YOU P, YONG S. Maneuver detection and track for HBG aircraft based on IMM algorithm. Proc. of the IEEE International Conference on Instrumentation & Measurement, 2016, 384- 388.
35	YANG Q M, ZHANG J D, SHI G Q. Path planning for unmanned aerial vehicle passive detection under the framework of partially observable Markov decision process. Proc. of the 30th Chinese Control and Decision Conference, 2018, 3896- 3903.
36	BOUTON M, COSGUN A, KOCHENDERFER M J. Belief state planning for autonomously navigating urban intersections. Proc. of the IEEE Intelligent Vehicles Symposium, 2017, 825- 830.
37	TIPIREDDY R, CHATTERJEE S, PAULSON P, et al. Agentcentric approach for cybersecurity decision-support with partial observability. Proc. of the IEEE International Symposium on Technologies for Homeland Security, 2017, 1- 6. doi: 10.1109/THS.2017.7943478
38	SHAN G L, ZHANG Z N. Non-myopic sensor scheduling for low radiation risk tracking using mixed POMDP. Transactions of the Institute of Measurement & Control, 2015, 39 (2): 230- 243.

[1]	Zhuanhua ZHANG, Gongjian ZHOU. Maneuvering target state estimation based on separate modeling of target trajectory shape and dynamic characteristics [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 1195-1209.
[2]	Lanyong ZHANG, Ruixuan ZHANG. Research on UAV cloud control system based on ant colony algorithm [J]. Journal of Systems Engineering and Electronics, 2022, 33(4): 805-811.
[3]	Gang LIU, Zhibiao AN, Songyang LAO, Wu LI. Firepower distribution method of anti-ship missile based on coupled path planning [J]. Journal of Systems Engineering and Electronics, 2022, 33(4): 1010-1024.
[4]	Keyi LI, Zhengkun GUO, Gongjian ZHOU. State estimation in range coordinate using range-only measurements [J]. Journal of Systems Engineering and Electronics, 2022, 33(3): 497-510.
[5]	Yuanshi ZHANG, Minghai PAN, Weijun LONG, Hua LI, Qinghua HAN. Joint waveform selection and power allocation algorithm in manned/unmanned aerial vehicle hybrid swarm based on chance-constraint programming [J]. Journal of Systems Engineering and Electronics, 2022, 33(3): 551-562.
[6]	Liangliang WANG, Gongjian ZHOU. Multiframe track-before-detect method based on velocity filtering in mixed coordinates [J]. Journal of Systems Engineering and Electronics, 2022, 33(2): 247-258.
[7]	Alireza MOHSENI, Vincent DUCHAINE, Tony WONG. Experimental study of path planning problem using EMCOA for a holonomic mobile robot [J]. Journal of Systems Engineering and Electronics, 2021, 32(6): 1450-1462.
[8]	Chen TIAN, Yang PEI, Peng HOU, Qian ZHAO. Multi-target tracking algorithm based on PHD filter against multi-range-false-target jamming [J]. Journal of Systems Engineering and Electronics, 2020, 31(5): 859-870.
[9]	Yang SU, Ting CHENG, Zishu HE, Xi LI, Yanxi LU. Adaptive resource management for multi-target tracking in co-located MIMO radar based on time-space joint allocation [J]. Journal of Systems Engineering and Electronics, 2020, 31(5): 916-927.
[10]	Zhichao BAO, Qiuxi JIANG, Fangzheng LIU. Multiple model efficient particle filter based track-before-detect for maneuvering weak targets [J]. Journal of Systems Engineering and Electronics, 2020, 31(4): 647-656.
[11]	Caihua WU, Jianchao MA, Xiuwei ZHANG, Dang XIE. User space transformation in deep learning based recommendation [J]. Journal of Systems Engineering and Electronics, 2020, 31(4): 674-684.
[12]	Lili SUN, Yunhe CAO, Wenhua WU, Yutao LIU. A multi-target tracking algorithm based on Gaussian mixture model [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 482-487.
[13]	Gang CHEN, Jun WANG. Target detection method in passive bistatic radar [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 510-519.
[14]	Haowei ZHANG, Junwei XIE, Jiaang GE, Zhaojian ZHANG, Wenlong LU. Finite sensor selection algorithm in distributed MIMO radar for joint target tracking and detection [J]. Journal of Systems Engineering and Electronics, 2020, 31(2): 290-302.
[15]	Zhen XU, Enze ZHANG, Qingwei CHEN. Rotary unmanned aerial vehicles path planning in rough terrain based on multi-objective particle swarm optimization [J]. Journal of Systems Engineering and Electronics, 2020, 31(1): 130-141.

Modeling of UAV path planning based on IMM under POMDP framework

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 7

References 38

Related Articles 15

Recommended Articles

Metrics

Comments