Journal of Systems Engineering and Electronics ›› 2024, Vol. 35 ›› Issue (6): 1594-1603.doi: 10.23919/JSEE.2024.000111

• CONTROL THEORY AND APPLICATION • Previous Articles    

Deep reinforcement learning guidance with impact time control

Guofei LI1(), Shituo LI1(), Bohao LI2,*(), Yunjie WU3   

  1. 1 School of Astronautics, Northwestern Polytechnical University, Xi’an 710072, China
    2 Beijing Institute of Control and Electronic Technology, Beijing 100038, China
    3 School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
  • Received:2023-09-28 Online:2024-12-18 Published:2025-01-14
  • Contact: Bohao LI E-mail:liguofei1@126.com;lishituo1@163.com;libh08@buaa.edu.cn
  • About author:
    LI Guofei was born in 1991. He received his Ph.D. degree from the School of Automation Science and Electrical Engineering, Beihang University, Beijing, China, in 2020. From 2020 to 2021, he was a postdoctoral fellow of Zhuoyue Program in the School of Cyber Science and Technology, Beihang University, Beijing, China. Currently, he is an associate professor with the School of Astronautics, Northwestern Polytechnical University, Xi’an, China. His research interests include cooperative guidance, servo system control, and nonlinear control. E-mail: liguofei1@126.com

    LI Shituo was born in 2000. He received his B.E. degree from Chang’an University, Xi’an, China, in 2022. He is currently pursuing his M.S. degree in electronic and information engineering with Northwestern Polytechnical University, Xi’an, China. His research interests include cooperative guidance and nonlinear control. E-mail: lishituo1@163.com

    LI Bohao was born in 1990. He received his B.E. degree from Lanzhou University, Lanzhou, China, in 2012, M.S. degree from Lanzhou University of Technology, Lanzhou, China, in 2017, and Ph.D. degree from the School of Automation Science and Electrical Engineering, Beihang University, Beijing, China, in 2022. Currently, he is an engineer with Beijing Institute of Control and Electronic Technology. His research interests include deep learning, deep reinforcement learning, and guidance. E-mail: libh08@buaa.edu.cn

    WU Yunjie was born in 1969. She received her Ph.D. degree in navigation guidance and control from Beihang University in 2006. Currently, she is a professor with the School of Automation Science and Electrical Engineering, Beihang University, Beijing, China. Her research interests include system simulation, intelligent control, servo control, aircraft guidance and control technology. E-mail: wyjmip@ buaa.edu.cn
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (62003021;62373304), Industry-University-Research Innovation Fund for Chinese Universities (2021ZYA02009), Shaanxi Qinchuangyuan High-level Innovation and Entrepreneurship Talent Project (OCYRCXM-2022-136), Shaanxi Association for Science and Technology Youth Talent Support Program (XXJS202218), and the Fundamental Research Funds for the Central Universities (D5000210830).

Abstract:

In consideration of the field-of-view (FOV) angle constraint, this study focuses on the guidance problem with impact time control. A deep reinforcement learning guidance method is given for the missile to obtain the desired impact time and meet the demand of FOV angle constraint. On basis of the framework of the proportional navigation guidance, an auxiliary control term is supplemented by the distributed deep deterministic policy gradient algorithm, in which the reward functions are developed to decrease the time-to-go error and improve the terminal guidance accuracy. The numerical simulation demonstrates that the missile governed by the presented deep reinforcement learning guidance law can hit the target successfully at appointed arrival time.

Key words: impact time, deep reinforcement learning, guidance law, field-of-view (FOV) angle, deep deterministic policy gradient