Reinforcement survivable routing for resource-limited LEO satellite network

doi:10.23919/JSEE.2026.000142

Abstract

Abstract:

In order to provide low Earth orbit (LEO) satellite network survivability with limited resources, and effectively respond to the risk of on orbit operation, a survivable routing algorithm based on multi-agent reinforcement learning is proposed. The reinforcement survivable routing establishes the reward function from the perspective of network utility, and describes the improvement of survivability in the failure scenario as a joint optimization problem, which solves the online routing decision-making problems after LEO satellite network failure. Each cognitive user agent and satellite agent can independently determine the routing strategy for communication missions according to the time slot topology, node survival status, storage space and other environment information of the satellite network. The simulation results show that compared to the weighted semi-distributed routing algorithm (WSDRA) and Fibonacci multipath load balancing (FMLB) algorithm, the proposed multi-agent double deep Q-learning network (MADDQN) reinforcement survivable routing algorithm can adapt to satellite failure and topology changes, and improve network utility by 14% and 9% respectively in human attack scenario, and by 17% and 13% respectively in random failure scenario.

Key words: low Earth orbit (LEO) satellite network, survivable routing, reinforcement learning, network utility

Yuanyuan Nie, Zhigeng Fang, Su Gao, Jiajia Cai. Reinforcement survivable routing for resource-limited LEO satellite network[J]. Journal of Systems Engineering and Electronics, 2026, 37(4): 1341-1353.

Figures/Tables 9

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

Fig 8

Fig 9

References 41

1	Araniti G, Bezirgannidis N, Birrane E, et al Contact graph routing in DTN space networks: overview, enhancements and performance. IEEE Communications Magazine, 2015, 53 (3): 38. doi: 10.1109/MCOM.2015.7060480
2	Wang C H, Zhang Z S, Wu J Y An overview of protected satellite communications in intelligent age. Science China Information Sciences, 2021, 64 (6): 161301. doi: 10.1007/s11432-019-2928-9
3	Yan Y J, Han G J, Xu H H A survey on secure routing protocols for satellite network. Journal of Network and Computer Applications, 2019, 145, 102415. doi: 10.1016/j.jnca.2019.102415
4	Wang B, Liu X S, Zhang Y Q. Internet of Things and BDS application[M]. Singapore: Springer, 2022.
5	Li T, Li Y, Lan S W, et al A method of using back propagation neural network to estimate orbital lifetime of LEO satellites. Advances in Space Research, 2023, 72, 1961. doi: 10.1016/j.asr.2023.05.026
6	Ellison R, Linger R Survivable network system analysis: a case study. IEEE Software, 1999, 16 (4): 70. doi: 10.1109/52.776952
7	Wang Y X, Sun Z K, Zhang H Q Dynamic survivability of two-layer networks with different topologies. The European Physical Journal Plus, 2024, 139 (1): 94. doi: 10.1140/epjp/s13360-024-04906-9
8	Liu S T, Sun Z K, Zhao N N, et al Explosive transition in coupled oscillators through mixed attractive-repulsive interactions. International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 2022, 32 (2): 2250018. doi: 10.1142/s0218127422500183
9	Li S J, Zhang C J, Zhao C L, et al Robustness of LEO satellite hypernetworks under different network topologies and attack strategies. Physics Letters A, 2024, 526, 129952. doi: 10.1016/j.physleta.2024.129952
10	Zhou D, Sheng M, Liu R Z, et al Channel-aware mission scheduling in broadband data relay satellite networks. IEEE Journal on Selected Areas in Communications, 2018, 36 (5): 1052. doi: 10.1109/JSAC.2018.2832779
11	Li X, Xu W, Liu Y Link duration-aware shared backup path protection (LD-SBPP) against SRLG failure in LEO-satellite-terrestrial integrated optical networks. Optical Fiber Technology, 2023, 81, 103571. doi: 10.1016/j.yofte.2023.103571
12	Li H, Shi D C, Wang W Z, et al Secure routing for LEO satellite network survivability. Computer Networks, 2022, 211, 109011. doi: 10.1016/j.comnet.2022.109011
13	Zhou D, Sheng M, Wang X J, et al Mission aware contact plan design in resource-limited small satellite networks. IEEE Trans. on Communications, 2017, 65 (6): 2451. doi: 10.1109/TCOMM.2017.2685383
14	Qi X, Ma J, Wu D, et al A survey of routing techniques for satellite networks. Journal of Communications and Networks, 2017, 1 (4): 66. doi: 10.1007/bf03391581
15	Lu Y, Zhao Y J, Sun F C, et al Low-load survivable routing protocol for LEO satellite networks. International Journal of Wireless Information Networks, 2015, 22 (2): 73. doi: 10.1007/s10776-015-0265-0
16	Long F, Xiong N X, Vasilakos A V, et al A sustainable heuristic QoS routing algorithm for pervasive multi-layered satellite wireless networks. Wireless Networks, 2010, 16 (6): 1657. doi: 10.1007/s11276-009-0220-z
17	Lu Y, Zhao Y J, Sun F C, et al A survivable routing protocol for two-layered LEO/MEO satellite networks. Wireless Network, 2013, 20 (5): 871. doi: 10.1007/s11276-013-0656-z
18	Bhattacharya A, Sinha K An efficient protocol for load-balanced multipath routing in mobile ad hoc networks. Ad Hoc Networks, 2017, 63, 104. doi: 10.1016/j.adhoc.2017.05.008
19	Song G, Chao M Y, Yang B W, et al Research on multipath QOS routing strategy for the satellite network. Lecture Notes in Electrical Engineering, 2013, 187, 289.
20	Pasic A, Babarczi P, Tapolcai J, et al Minimum cost survivable routing algorithms for generalized diversity coding. IEEE-ACM Transactions on Networking, 2020, 28 (1): 289. doi: 10.1109/TNET.2019.2963574
21	Qi X X, Zhang B, Qiu Z L A distributed survivable routing algorithm for Mega-Constellations with inclined orbits. IEEE Access, 2020, 8, 219199. doi: 10.1109/ACCESS.2020.3041346
22	Din D R, Lien C Y Delay-constrained survivable multicast routing problem on WDM networks for node failure case. Computer Communications, 2017, 103, 165. doi: 10.1016/j.comcom.2017.01.018
23	Pairaa S, Chatterjeeb M, Bhattacharya U On survivable energy-efficient and crosstalk-aware routing, spectrum and core allocation schemes for dynamic multiclass traffic in SDM-EONs. Optical Switching and Networking, 2021, 42, 100630. doi: 10.1016/j.osn.2021.100630
24	Halder J, Acharya T, Chatterjee M, et al E-S-RSM-RSA: a novel energy and spectrum efficient regenerator aware multipath based survivable RSA in offline EON. IEEE Trans. on Green Communications and Networking, 2021, 5 (3): 1451. doi: 10.1109/TGCN.2021.3075616
25	Caini C, Cruickshank H, Farrell S, et al Delay- and disruption-tolerant networking (DTN): an alternative solution for future satellite networking applications. Proceedings of the IEEE, 2011, 99 (11): 1980. doi: 10.1109/JPROC.2011.2158378
26	Kaneko K, Nishiyama H, Kato N, et al Construction of a flexibility analysis model for flexible high-throughput satellite communication systems with a digital channelizer. IEEE Trans. on Vehicular Technology, 2018, 67 (3): 2097. doi: 10.1109/TVT.2017.2736010
27	Liu R, Sheng M, Lui K S, et al An analytical framework for resource-limited small satellite networks. IEEE Communications Letters, 2016, 20 (2): 388. doi: 10.1109/LCOMM.2015.2509993
28	Seo J, Ham D, Kwak J. Dynamic content-cached satellite selection and routing for power minimization in LEO satellite networks[J]. ICT Express, 2024, 10(6): 1199.
29	Tang R F, Yi D Y, Luo Q, et al Fast simulation algorithm for visibility of LEO satellites. Journal of System Simulation, 2018, 20 (18): 4850.
30	Li H, Yu J Y, Cao L L, et al Multi-agent reinforcement learning based computation offloading and resource allocation for LEO satellite edge computing networks. Computer Communications, 2024, 222, 268. doi: 10.1016/j.comcom.2024.05.008
31	Lin X, Chen Y, Xue J H, et al Large-volume LEO satellite imaging data networked transmission scheduling problem: model and algorithm. Expert Systems with Applications, 2024, 249, 123649. doi: 10.1016/j.eswa.2024.123649
32	Zhang R D, Zhang J D, Wang X, et al Utility optimization for computation offloading and splitting in time-varying HAP and LEO satellite integrated MEC networks. Computer Networks, 2024, 255, 110866. doi: 10.1016/j.comnet.2024.110866
33	Nie Y Y, Fang Z G, Gao S Q-GERT survivability assessment of LEO satellite constellation. Wireless Networks, 2021, 27 (1): 249. doi: 10.1007/s11276-020-02452-7
34	Nguyen H T, Choong S H, Han Z, et al Optimal pricing effect on equilibrium behaviors of delay-sensitive users in cognitive radio networks. IEEE Journal on Selected Areas in Communications, 2013, 31 (11): 2266. doi: 10.1109/jsac.2013.131134
35	Zhang J D, Zhang R D, Shi W X LEO satellites selection-based computation offloading algorithm in aircraft-satellite multi-access edge computing networks. Computer Communications, 2024, 223, 115. doi: 10.1016/j.comcom.2024.05.011
36	Geng S Y, Liu S F, Fang Z G Resilient communication model for satellite networks using clustering technique. Reliability Engineering and System Safety, 2021, 215, 107850. doi: 10.1016/j.ress.2021.107850
37	Hasselt H V, Guez A, Silver D. Deep reinforcement learning with double Q-learning[C]//30th AAAI Conference on Artificial Intelligence, 2015: 2094.
38	Wan S M. Research on the invulnerability of space information networks topology[D]. Chengdu: University of Electronic Science and Technology of China, 2021. (in Chinese)
39	Pasic A, Babarczi P, Korosi A K Diversity coding-based survivable routing with QoS and differential delay bounds. Optical Switching and Networking, 2017, 23, 118. doi: 10.1016/j.osn.2016.06.005
40	Guo Z, Yan Z A weighted semi-distributed routing algorithm for LEO satellite networks. Journal of Network and Computer Applications, 2018, 58, 1. doi: 10.2514/6.2014-4187
41	Naseem M, Kumar C Congestion-aware Fibonacci sequence based multipath load balancing routing protocol for MANETS. Wireless Personal Communications, 2015, 84 (4): 2955. doi: 10.1007/s11277-015-2775-6

[1]	Shijun Xie, Hao Liang, Liping Li, Jintian Xiong, Sili Liu. Design and optimization for polar coded cooperative transmission in satellite-based NTN [J]. Journal of Systems Engineering and Electronics, 2026, 37(4): 1177-1185.
[2]	Jingbo WANG, Liaoyuan ZHU, Shaojie XIA, Huibin LIU, Jing LIU, Chongxiao QU, Zhihuan SONG. Application of self-play reinforcement learning and explainable decision tree in intelligent air combat [J]. Journal of Systems Engineering and Electronics, 2026, 37(2): 616-635.
[3]	Hongtao ZHU, Xinyu WANG, Zhenyong WANG, Dezhi LI, Qing GUO. Satellite handover strategies based on minimum routing hops for mega LEO satellite networks [J]. Journal of Systems Engineering and Electronics, 2026, 37(1): 64-74.
[4]	Xiaoyu XING, Haoxiang XIA. A formation pursuit method integrated coordinated reciprocity for enhanced capture [J]. Journal of Systems Engineering and Electronics, 2026, 37(1): 211-224.
[5]	Shijie DENG, Yingxin KOU, Maolong LYU, Zhanwu LI, An XU. λ-return-based aircraft maneuvering for terminal defense and positioning guidance strategies [J]. Journal of Systems Engineering and Electronics, 2025, 36(6): 1692-1708.
[6]	Wenhao CHEN, Gang CHEN, Jichao LI, Jiang JIANG. Disintegration of heterogeneous combat network based on double deep Q-learning [J]. Journal of Systems Engineering and Electronics, 2025, 36(5): 1235-1246.
[7]	Rui ZHOU, Weichao ZHONG, Wenlong LI, Hao ZHANG. Self-play training and analysis for GEO inspection game with modular actions [J]. Journal of Systems Engineering and Electronics, 2025, 36(5): 1353-1373.
[8]	Hang DI, Tao DONG, Zhihui LIU, Shichao JIN. DEF-based energy consumption balancing optimization for LEO satellite networks [J]. Journal of Systems Engineering and Electronics, 2025, 36(4): 922-931.
[9]	Siyu HENG, Ting CHENG, Zishu HE, Yuanqing WANG, Luqing LIU. Adaptive dwell scheduling based on Q-learning for multifunctional radar system [J]. Journal of Systems Engineering and Electronics, 2025, 36(4): 985-993.
[10]	Yifan ZHANG, Tao DONG, Zhihui LIU, Shichao JIN. Multi-QoS routing algorithm based on reinforcement learning for LEO satellite networks [J]. Journal of Systems Engineering and Electronics, 2025, 36(1): 37-47.
[11]	Nanxun DUO, Qinzhao WANG, Qiang LYU, Wei WANG. Tactical reward shaping for large-scale combat by multi-agent reinforcement learning [J]. Journal of Systems Engineering and Electronics, 2024, 35(6): 1516-1529.
[12]	Guofei LI, Shituo LI, Bohao LI, Yunjie WU. Deep reinforcement learning guidance with impact time control [J]. Journal of Systems Engineering and Electronics, 2024, 35(6): 1594-1603.
[13]	Tingting ZHANG, Zijian GUO, Bin LI, Yuan FENG, Qi FU, Mingyu HU, Yunbo QU. Delay-optimal multi-satellite collaborative computation offloading supported by OISL in LEO satellite network [J]. Journal of Systems Engineering and Electronics, 2024, 35(4): 805-814.
[14]	Qi WANG, Zhizhong LIAO. Computational intelligence interception guidance law using online off-policy integral reinforcement learning [J]. Journal of Systems Engineering and Electronics, 2024, 35(4): 1042-1052.
[15]	Guang ZHAN, Kun ZHANG, Ke LI, Haiyin PIAO. UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience [J]. Journal of Systems Engineering and Electronics, 2024, 35(3): 644-665.