Adaptive dwell scheduling based on Q-learning for multifunctional radar system

doi:10.23919/JSEE.2025.000111

Journal of Systems Engineering and Electronics ›› 2025, Vol. 36 ›› Issue (4): 985-993.doi: 10.23919/JSEE.2025.000111

• SYSTEMS ENGINEERING • Previous Articles

Adaptive dwell scheduling based on Q-learning for multifunctional radar system

Siyu HENG(), Ting CHENG(), Zishu HE(), Yuanqing WANG(), Luqing LIU()

School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China

Received:2023-08-04 Online:2025-08-18 Published:2025-09-04
Contact: Ting CHENG E-mail:sy_heng1999@qq.com;citrus@uestc.edu.cn;zshe@uestc.edu.cn;wyq13069883010@163.com;2421305188@qq.com
About author:
HENG Siyu was born in 1999. She received her B.S. degree from Jilin University, Changchun, China, in 2021. She is currently pursuing her M.S. degree with the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China. Her research interests include radar dwell scheduling and radar resource management. E-mail: sy_heng1999@qq.com

CHENG Ting was born in 1982. She received her B.S. and Ph.D. degrees in electronics engineering from University of Electronic Science and Technology of China, Chengdu, China, in 2006 and 2008, respectively. She is currently an associate professor with the School of Information and Communication Engineering, University of Electronic Science and Technology of China. Her research interests include target tracking, radar dwell scheduling, radar resource management, and cognitive radar. E-mail: citrus@uestc.edu.cn

HE Zishu was born in 1962. He received his B.S., M.S. and Ph.D. degrees in signal and information processing from the University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 1984, 1988, and 2000, respectively. He is currently a professor of signal and information processing in the School of Electronic Engineering, UESTC. His research interests include array signal processing, digital beam forming, the theory on multiple input multiple output (MIMO) communication and MIMO radar, adaptive signal processing, and interference cancellation. E-mail: zshe@uestc.edu.cn

WANG Yuanqing was born in 1997. He received his B.S. degree from Harbin Engineering University, Harbin, China, in 2020. He is currently pursuing his M.S. degree with the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China. His research interests include suppression jamming and radar resource management. E-mail: wyq13069883010@163.com

LIU Luqing was born in 2001. She received her B.S. degree from the Wuhan University of Technology, Wuhan, China, in 2022. She is currently pursuing her M.S. degree with the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China. Her research interests include radar dwell scheduling and radar resource management. E-mail: 2421305188@qq.com
Supported by:
This work was supported by the National Natural Science Foundation of China (61771095; 62031007).

Abstract

Abstract:

The dwell scheduling problem for a multifunctional radar system is led to the formation of corresponding optimization problem. In order to solve the resulting optimization problem, the dwell scheduling process in a scheduling interval (SI) is formulated as a Markov decision process (MDP), where the state, action, and reward are specified for this dwell scheduling problem. Specially, the action is defined as scheduling the task on the left side, right side or in the middle of the radar idle timeline, which reduces the action space effectively and accelerates the convergence of the training. Through the above process, a model-free reinforcement learning framework is established. Then, an adaptive dwell scheduling method based on Q-learning is proposed, where the converged Q value table after training is utilized to instruct the scheduling process. Simulation results demonstrate that compared with existing dwell scheduling algorithms, the proposed one can achieve better scheduling performance considering the urgency criterion, the importance criterion and the desired execution time criterion comprehensively. The average running time shows the proposed algorithm has real-time performance.

Key words: multifunctional radar, dwell scheduling, reinforcement learning, Q-learning

Siyu HENG, Ting CHENG, Zishu HE, Yuanqing WANG, Luqing LIU. Adaptive dwell scheduling based on Q-learning for multifunctional radar system[J]. Journal of Systems Engineering and Electronics, 2025, 36(4): 985-993.

Figures/Tables 7

Fig 1

Fig 2

Fig 3

Table 1

Fig 4

Fig 5

Fig 6

References 33

1	TUNCER O, CIRPAN H A A networked radar resource management approach utilizing target priority and maneuver for cooperative air defense fire control radars. IEEE Access, 2023, 11, 136279- 136291. doi: 10.1109/ACCESS.2023.3337799
2	SHI C G, DING L T, QIU W, et al. Joint optimization of target assignment and resource allocation for multi-target tracking in phased array radar network. Proc. of the IEEE Radar Conference, 2020: 1−5.
3	ZHANG D L, SUN J, YI W, et al. Joint jamming beam and power scheduling for suppressing netted radar system. Proc. of the IEEE Radar Conference, 2021: 1−6.
4	YANG T, YUAN Y, YI W. Multi-domain resource scheduling for surveillance radar anti-jamming based on Q-learning. Proc. of the IEEE Radar Conference, 2023: 1−6.
5	PELL C Phased-array radars. IEE Review, 1988, 34 (9): 363- 367. doi: 10.1049/ir:19880149
6	SHI C G, WANG Y J, SALOUS S, et al Joint transmit resource management and waveform selection strategy for target tracking in distributed phased array radar network. IEEE Trans. on Aerospace and Electronic Systems, 2022, 58 (4): 2762- 2778. doi: 10.1109/TAES.2021.3138869
7	ORMAN A J, POTTS N C, SHAHANI A K, et al A hybrid adaptively genetic algorithm for task scheduling problem in the phased array radar. European Journal of Operational Research, 1996, 90 (1): 13- 25. doi: 10.1016/0377-2217(95)00307-X
8	PAN W Application of adaptive genetic algorithm to optimal scheduling of phased array radar. Electronic Information Warfare Technology, 2014, 29 (1): 38- 41.
9	ZHANG H W, XIE J W, GE J A, et al A hybrid adaptively genetic algorithm for task scheduling problem in the phased array radar. European Journal of Operational Research, 2019, 272 (3): 868- 878. doi: 10.1016/j.ejor.2018.07.012
10	MENG F Q, TIAN K S Phased-array radar task scheduling method for hypersonic-glide vehicles. IEEE Access, 2020, 8, 221288- 221298. doi: 10.1109/ACCESS.2020.3043338
11	ZHANG H W, XIE J W, GE J A, et al Hybrid particle swarm optimization algorithm based on entropy theory for solving DAR scheduling problem. Tsinghua Science and Technology, 2019, 24 (3): 282- 290. doi: 10.26599/TST.2018.9010052
12	ZHANG H W, XIE J W, SHENG C Adaptive scheduling algorithm over comprehensive priority for phased array radar. Acta Armamentarii, 2016, 37 (11): 2163- 2169.
13	DING H T, ZHOU L. Research on task scheduling algorithm based on fuzzy logic priority. Proc. of the IEEE International Conference on Information Technology, Big Data and Artificial Intelligence, 2020: 669−675.
14	JIMENEZ M I, VAL L D, VILLACORTA J J, et al. Design of task scheduling process for a multifunction radar. IET Radar, Sonar & Navigation, 2012, 6(5): 341−347.
15	CHENG T, HE Z S, TANG T Dwell scheduling algorithm for multifunction phased array radars based on the scheduling gain. Journal of Systems Engineering and Electronics, 2008, 19 (3): 479- 485. doi: 10.1016/S1004-4132(08)60110-3
16	QU Z, DING Z, MOO P. Dual-side scheduling for radar resource management. Proc. of the International Radar Symposium, 2020: 260−263.
17	MIR H S, GUITOUNI A Variable dwell time task scheduling for multifunction radar. IEEE Trans. on Automation Science and Engineering, 2014, 11 (2): 463- 472. doi: 10.1109/TASE.2013.2285014
18	HUANG L, ZHANG Y, LI Q Y, et al Task-scheduling scheme based on greedy algorithm in integrated radar and communication systems. The Journal of Engineering, 2019, 2019 (19): 5864- 5867. doi: 10.1049/joe.2019.0213
19	YANG K W, LI J C, LIU M D, et al Complex systems and network science: a survey. Journal of Systems Engineering and Electronics, 2023, 34 (3): 543- 573. doi: 10.23919/JSEE.2023.000080
20	CHEN C, QUAN W, SHAO Z Aerial target threat assessment based on gated recurrent unit and self-attention mechanism. Journal of Systems Engineering and Electronics, 2024, 35 (2): 361- 373. doi: 10.23919/JSEE.2023.000116
21	ZHOU W, LIU M, XU Z S The dual-fuzzy convolutional neural network to deal with handwritten image recognition. IEEE Trans. on Fuzzy Systems, 2022, 30 (12): 5225- 5236. doi: 10.1109/TFUZZ.2022.3170657
22	JI H G, OH S, KIM J, et al. Integrating deep learning and machine translation for understanding unrefined languages. Computers, Materials & Continua, 2022, 70(1): 669-678.
23	YIN H C, ZHANG G Z, WU Q, et al A deep learning-based data-driven approach for predicting mining water inrush from coal seam floor using microseismic monitoring data. IEEE Trans. on Geoscience and Remote Sensing, 2023, 61, 1- 15.
24	YOU C X, LU J B, FILEV D, et al Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning. Robotics and Autonomous Systems, 2019, 114, 1- 18. doi: 10.1016/j.robot.2019.01.003
25	SHI Y C, JIU B, YAN J K, et al Data-driven radar selection and power allocation method for target tracking in multiple radar system. IEEE Sensors Journal, 2021, 21 (17): 19296- 19306. doi: 10.1109/JSEN.2021.3087747
26	LI S X, LIU G Y, ZHANG K, et al DRL-based joint path planning and jamming power allocation optimization for suppressing netted radar system. IEEE Signal Processing Letters, 2023, 30, 548- 552. doi: 10.1109/LSP.2023.3270762
27	LEE J, NIYATO D, GUAN Y L, et al Learning to schedule joint radar-communication with deep multi-agent reinforcement learning. IEEE Trans. on Vehicular Technology, 2022, 71 (1): 406- 422. doi: 10.1109/TVT.2021.3124810
28	LIU X W, ZHANG Q, LUO Y, et al Radar network time scheduling for multi-target ISAR task with game theory and multiagent reinforcement learning. IEEE Sensors Journal, 2021, 21 (4): 4462- 4473. doi: 10.1109/JSEN.2020.3029430
29	QU Z, DING Z, MOO P. A machine learning task selection method for radar resource management. Proc. of the International Conference on Information Fusion, 2019: 1−6.
30	AKBAR S, ADVE R S, DING Z, et al Transfer-based DRL for task scheduling in dynamic environments for cognitive radar. IEEE Trans. on Aerospace and Electronic Systems, 2024, 60 (1): 37- 50. doi: 10.1109/TAES.2023.3317819
31	XU L X, ZHANG T X. Reinforcement learning based dynamic task scheduling for multifunction radar network. Proc. of the IEEE Radar Conference, 2020: 1−5.
32	GEORGE T, WAGNER K, RADEMACHER P. Deep Q-network for radar task-scheduling problem. Proc. of the IEEE Radar Conference, 2022: 1−5.
33	QU Z, DING Z, MOO P. A neural network based algorithm selector for radar task scheduling. Proc. of the IEEE 19th International Conference on Cognitive Informatics & Cognitive Computing, 2020: 119−124.

Task type	p	Dwell number	Period/ms	l/ms	$ \Delta t $/ms
Confirmation	5	1	−	20	6
Precise tracking	4	1	500	20	5
Normal tracking	3	1	1000	25	6
Horizon searching	2	80	2 000	−	7
Airspace searching	1	120	3000	−	8

[1]	Ting CHENG, Luqing LIU, Siyu HENG. Dwell scheduling for MFIS with aperture partition and JRC waveform [J]. Journal of Systems Engineering and Electronics, 2025, 36(4): 951-961.
[2]	Yifan ZHANG, Tao DONG, Zhihui LIU, Shichao JIN. Multi-QoS routing algorithm based on reinforcement learning for LEO satellite networks [J]. Journal of Systems Engineering and Electronics, 2025, 36(1): 37-47.
[3]	Nanxun DUO, Qinzhao WANG, Qiang LYU, Wei WANG. Tactical reward shaping for large-scale combat by multi-agent reinforcement learning [J]. Journal of Systems Engineering and Electronics, 2024, 35(6): 1516-1529.
[4]	Guofei LI, Shituo LI, Bohao LI, Yunjie WU. Deep reinforcement learning guidance with impact time control [J]. Journal of Systems Engineering and Electronics, 2024, 35(6): 1594-1603.
[5]	Qi WANG, Zhizhong LIAO. Computational intelligence interception guidance law using online off-policy integral reinforcement learning [J]. Journal of Systems Engineering and Electronics, 2024, 35(4): 1042-1052.
[6]	Guang ZHAN, Kun ZHANG, Ke LI, Haiyin PIAO. UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience [J]. Journal of Systems Engineering and Electronics, 2024, 35(3): 644-665.
[7]	Yaozhong ZHANG, Zhuoran WU, Zhenkai XIONG, Long CHEN. A UAV collaborative defense scheme driven by DDPG algorithm [J]. Journal of Systems Engineering and Electronics, 2023, 34(5): 1211-1224.
[8]	Jiawei XIA, Xufang ZHU, Zhong LIU, Qingtao XIA. LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle [J]. Journal of Systems Engineering and Electronics, 2023, 34(5): 1343-1358.
[9]	Yunxiu ZENG, Kai XU. Recognition and interfere deceptive behavior based on inverse reinforcement learning and game theory [J]. Journal of Systems Engineering and Electronics, 2023, 34(2): 270-288.
[10]	Jie LI, Xiaoyu DANG, Sai LI. DQN-based decentralized multi-agent JSAP resource allocation for UAV swarm communication [J]. Journal of Systems Engineering and Electronics, 2023, 34(2): 289-298.
[11]	Yaozhong ZHANG, Yike LI, Zhuoran WU, Jialin XU. Deep reinforcement learning for UAV swarm rendezvous behavior [J]. Journal of Systems Engineering and Electronics, 2023, 34(2): 360-373.
[12]	Lu DONG, Zichen HE, Chunwei SONG, Changyin SUN. A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures [J]. Journal of Systems Engineering and Electronics, 2023, 34(2): 439-459.
[13]	Guangran CHENG, Lu DONG, Xin YUAN, Changyin SUN. Reinforcement learning-based scheduling of multi-battery energy storage system [J]. Journal of Systems Engineering and Electronics, 2023, 34(1): 117-128.
[14]	Peng LIU, Boyuan XIA, Zhiwei YANG, Jichao LI, Yuejin TAN. A deep reinforcement learning method for multi-stage equipment development planning in uncertain environments [J]. Journal of Systems Engineering and Electronics, 2022, 33(6): 1159-1175.
[15]	Bohao LI, Yunjie WU, Guofei LI. Hierarchical reinforcement learning guidance with threat avoidance [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 1173-1185.

Adaptive dwell scheduling based on Q-learning for multifunctional radar system

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 7

References 33

Related Articles 15

Recommended Articles

Metrics

Comments