DOA estimation via a Newton-like method under unknown mutual coupling

doi:10.23919/JSEE.2026.000110

Abstract

Abstract:

As is well known, mutual coupling between array elements has a significant negative impact on direction of arrival (DOA) estimation. To achieve DOA estimation under unknown mutual coupling, this paper proposes a low computational complexity Newton-like method. Firstly, a block sparse model based on the signal subspace is established, and the Lagrangian function is established according to the block sparse model. Secondly, since the Hessian matrix of the Lagrangian function cannot always ensure positive definiteness and the computational complexity of the inverse matrix of the Hessian matrix is enormous, the Newton method is no longer applicable. Therefore, this paper proposes a Newton-like method to achieve DOA estimation under mutual coupling and reduce the computational complexity by matrix inversion lemma. Finally, compared with existing methods of DOA estimation under array mutual coupling, the simulation results validate the effectiveness of the proposed method.

Key words: direction of arrival (DOA), mutual coupling, block sparse, Newton-like method

Jihui LYU, Shuai LIU, Ming JIN, Fenggang YAN, Lizhong SONG. DOA estimation via a Newton-like method under unknown mutual coupling[J]. Journal of Systems Engineering and Electronics, 2026, 37(3): 826-835.

Figures/Tables 9

Fig 1

Table 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

Fig 8

References 32

1	YIN Y T, WANG Y X, DAI T T, et al. DOA estimation based on smoothed sparse reconstruction with time modulated linear arrays. IEEE Trans. on Signal Processing, 2024. DOI: 10.1016/j.sigpro.2023.109229.
2	RAIGURU P, SUSANTA K S, SAHANI M, et al. Machine learning-aided sparse direction of arrival estimation. IEEE Sensors Journal, 2024, 24 (22): 38125–38134.
3	JIN J, HE D, SHUANG W, et al. Off-grid DOA estimation method based on sparse Bayesian learning with clustered structural-aware prior in formation. IEEE Trans. on Vehicular Technology, 2024, 73 (4): 5469–5483.
4	SUN S L, LIU S, WANG J, et al. Joint polarization and DOA estimation based on improved maximum likelihood estimator and performance analysis for conformal array. Journal of Systems Engineering and Electronics, 2023, 34 (6): 1490–1500.
5	WU X M, YANG Z, WEI Z Q, et al. Direction-of-arrival estimation for constant modulus signals using a structured matrix recovery technique. IEEE Trans. on Wireless Communications, 2024, 23 (4): 3117–3130.
6	PENG S, CHEN B, YANG M Joint sparse recovery for direction of arrival based on the generalized music criterion. Digital Signal Processing, 2022, 122, 103382. doi: 10.1016/j.dsp.2021.103382
7	LEITE W S, LAMARE R C, ZAKHAROV Y, et al Direction finding with sparse subarrays: Design, algorithms, and analysis. IEEE Trans. on Aerospace and Electronic Systems, 2024, 60 (6): 8149- 8165. doi: 10.1109/TAES.2024.3429474
8	REN A K, WU Q, LIANG P Y, et al Off-grid DOA estimation based on coherent accumulation and weighted block sparse Bayesian. Journal of Systems Engineering and Electronics, 2026, 37 (2): 327- 336. doi: 10.23919/JSEE.2025.000164
9	HAN K, NEHORAI A Improved source number detection and direction estimation with nested arrays and using jackknifing. IEEE Trans. on Signal Processing, 2013, 61 (23): 6118- 6128. doi: 10.1109/TSP.2013.2283462
10	MALIOUTOV D, CETIN M, WILLSKY A A Sparse signal reconstruction perspective for source localization with sensor arrays. IEEE Trans. on Signal Processing, 2005, 53 (8): 3010- 3022.
11	YANG Z, XIE L H, ZHANG C S Off-grid direction of arrival estimation using sparse Bayesian inference. IEEE Trans. on Signal Processing, 2013, 61 (1): 38- 43. doi: 10.1109/TSP.2012.2222378
12	CHENG P, CHEN Z M, CAO Z X, et al A new atomic norm for DOA estimation with gain-phase errors. IEEE Trans. on Signal Processing, 2020, 68, 4293- 4396. doi: 10.1109/TSP.2020.3010749
13	WANG X P, WANG W, LIU J, et al A sparse representation scheme for angle estimation in monostatic MIMO radar. Signal Processing, 2014, 104, 258- 263. doi: 10.1016/j.sigpro.2014.04.007
14	LIU Y L, YIN Y Z, LU H M, et al. A novel weighted block sparse DOA estimation based on signal subspace under unknown mutual coupling. Electronics, 2024, 13(9): 1790.
15	WEISS A, FRIEDLANDER B Mutual coupling effects on phase-only direction finding. IEEE Trans. on Antennas and Propagation, 1992, 40 (5): 535- 541. doi: 10.1109/8.142628
16	ZHENG Z, YANG C L Direction-of-arrival estimation of coherent signals under direction-dependent mutual coupling. IEEE Communications letters, 2021, 25 (1): 147- 151. doi: 10.1109/LCOMM.2020.3020897
17	YE Z F, LIU C On the resiliency of music direction finding against antenna sensor coupling. IEEE Trans. on Antennas and Propagation, 2008, 56 (2): 371- 380. doi: 10.1109/TAP.2007.915461
18	TIAN Y, WANG R, CHEN H, et al Real-valued DOA estimation utilizing enhanced covariance matrix with unknown mutual coupling. IEEE Communications Letters, 2022, 26 (4): 912- 916. doi: 10.1109/LCOMM.2022.3148260
19	LIAO B, ZHANG Z G, CHAN S C DOA estimation and tracking of ULAS with mutual coupling. IEEE Trans. on Aerospace and Electronic Systems, 2012, 48 (1): 891- 905.
20	GE Q C, ZHANG Y, WANG Y D A low complexity algorithm for direction of arrival estimation with direction-dependent mutual coupling. IEEE Communications Letters, 2020, 24 (1): 90- 94. doi: 10.1109/LCOMM.2019.2950029
21	DAI J S, ZHAO D, JI F X A sparse representation method for DOA estimation with unknown mutual coupling. IEEE Antennas and Wireless Propagation Letters, 2012, 11, 1210- 1213. doi: 10.1109/LAWP.2012.2223651
22	WANG Q, DOU T D, CHEN H, et al Effective block sparse representation algorithm for DOA estimation with unknown mutual coupling. IEEE Communications Letters, 2017, 21 (12): 2622- 2625. doi: 10.1109/LCOMM.2017.2747547
23	WANG X P, MENG D D, HUANG M X, et al Reweighted regularized sparse recovery for DOA estimation with unknown mutual coupling. IEEE Communications Letters, 2019, 23 (2): 290- 293. doi: 10.1109/LCOMM.2018.2884457
24	MENG D D, WANG X P, HUANG M X, et al Robust weighted subspace fitting for DOA estimation via block sparse recovery. IEEE Communications Letters, 2022, 24 (3): 563- 567.
25	MENG D D, LI X, WANG W Robust sparse recovery based vehicles location estimation in intelligent transportation system. IEEE Trans. on Intelligent Transportation Systems, 2024, 25 (1): 1023- 1032. doi: 10.1109/TITS.2023.3266703
26	MENG D D, WANG X P, HUANG M X, et al Block rank sparsity-aware DOA estimation with large-scale arrays in the presence of unknown mutual coupling. Digital Signal Processing, 2019, 94, 96- 104. doi: 10.1016/j.dsp.2019.06.013
27	HYDER M M, MAHATA K. A scalable distributed video coder using compressed sensing. Proc. of the Annual IEEE India Conference, 2009. DOI: 10.1109/INDCON.2009.5409352.
28	MOHIMANI H, BABAIE M A fast approach for overcomplete sparse decomposition based on smoothed l0 norm. IEEE Trans. on Signal Processing, 2009, 57 (1): 289- 301. doi: 10.1109/TSP.2008.2007606
29	HYDER M M, MAHATA K Direction-of-arrival estimation using a mixed ${l_{2, 0}}$ norm approximation. IEEE Trans. on Signal Processing, 2010, 58 (9): 4646- 4655. doi: 10.1109/TSP.2010.2050477
30	HYDER M M, MAHATA K An improved smoothed l₀ approximation algorithm for sparse representation. IEEE Trans. on Signal Processing, 2010, 58 (4): 2194- 2205. doi: 10.1109/TSP.2009.2040018
31	GORODNITSKY I, RAO B Sparse signal reconstruction from limited data using focuss: a re-weighted minimum norm algorithm. IEEE Trans. on Signal Processing, 1997, 45 (3): 600- 616. doi: 10.1109/78.558475
32	FRIEDLANDER B, WEISS A Direction finding in the presence of mutual coupling. IEEE Trans. on Antennas and Propagation, 1991, 39 (3): 273- 284. doi: 10.1109/8.76322

Method	Time/s	Computational complexity
The proposed	0.0656	${{O}}((N\tilde Q){M^2} + (N\tilde Q)MQ)$
SR	1.6014	${{O}}({(N\tilde Q)^3})$
BSR	1.6860	${{O}}({(N\tilde QQ)^3})$
RWS	1.6853	${{O}}({(N\tilde Q)^3})$
FOCUSS	0.0875	${{O}}(M(N\tilde Q)Q + (N\tilde Q){M^2})$
WBSS	1.9303	${{O}}({(N\tilde QQ)^3} + NPM(M - Q))$

[1]	Ankang REN, Qi WU, Pingye LIANG, Yuanyuan XU. Off-grid DOA estimation based on coherent accumulation and weighted block sparse Bayesian [J]. Journal of Systems Engineering and Electronics, 2026, 37(2): 327-336.
[2]	Jinghu SUN, Jiahuan LIU, Wenqiang WEI, Xianxiang YU, Guolong CUI, Xiuyin ZHANG. Far-field calibration of automotive millimeter wave radar via near-field implementation [J]. Journal of Systems Engineering and Electronics, 2025, 36(3): 694-700.
[3]	Hangui ZHU, Xixi CHEN, Teng MA, Yongliang WANG. Deep unfolded amplitude-phase error self-calibration network for DOA estimation [J]. Journal of Systems Engineering and Electronics, 2025, 36(2): 353-361.
[4]	Zeqi YANG, Yiheng LIU, Hua ZHANG, Shuai MA, Kai CHANG, Ning LIU, Xiaode LYU. DOA estimation of high-dimensional signals based on Krylov subspace and weighted l₁-norm [J]. Journal of Systems Engineering and Electronics, 2024, 35(3): 532-540.
[5]	Shuang WU, Ye YUAN, Weike ZHANG, Naichang YUAN. Super-resolution DOA estimation for correlated off-grid signals via deep estimator [J]. Journal of Systems Engineering and Electronics, 2022, 33(6): 1096-1107.
[6]	Luo CHEN, Xiangrui DAI, Xiaofei ZHANG. Joint angle and frequency estimation for linear array: an extended DOA-matrix method [J]. Journal of Systems Engineering and Electronics, 2022, 33(4): 887-895.
[7]	Shenghua WANG, Yunhe CAO, Yutao LIU. A method of Robust low-angle target height and compound reflection coefficient joint estimation [J]. Journal of Systems Engineering and Electronics, 2022, 33(2): 322-329.
[8]	Ping LI, Jianfeng LI, Gaofeng ZHAO. Low complexity DOA estimation for massive UCA with single snapshot [J]. Journal of Systems Engineering and Electronics, 2022, 33(1): 22-27.
[9]	Junpeng SHI, Fangqing WEN, Yongxiang LIU, Tianpeng LIU, Zhen LIU. High-order extended coprime array design for direction of arrival estimation [J]. Journal of Systems Engineering and Electronics, 2021, 32(4): 748-755.
[10]	Shuai SHAO, Aijun LIU, Changjun YU, Quanrui ZHAO. Polarization quaternion DOA estimation based on vector MISC array [J]. Journal of Systems Engineering and Electronics, 2021, 32(4): 764-778.
[11]	Yanan DU, Hongyuan GAO, Menghan CHEN. Direction of arrival estimation method based on quantum electromagnetic field optimization in the impulse noise [J]. Journal of Systems Engineering and Electronics, 2021, 32(3): 527-537.
[12]	Huihui MA, Haihong TAO. Joint 2D-DOA and polarization estimation for sparse nonuniform rectangular array composed of spatially spread electromagnetic vector sensor [J]. Journal of Systems Engineering and Electronics, 2020, 31(6): 1116-1127.
[13]	Chenghu CAO, Yongbo ZHAO, Xiaojiao PANG, Baoqing XU, Sheng CHEN. A method based on Chinese remainder theorem with all phase DFT for DOA estimation in sparse array [J]. Journal of Systems Engineering and Electronics, 2020, 31(1): 1-11.
[14]	Kunlai XIONG, Zhangmeng LIU, Pei WANG. SAGE-based algorithm for DOA estimation and array calibration in the presence of sensor location errors [J]. Journal of Systems Engineering and Electronics, 2019, 30(6): 1074-1080.
[15]	Shun He, Zhiwei Yang, and Guisheng Liao. DOA estimation of wideband signals based on iterative spectral reconstruction [J]. Journal of Systems Engineering and Electronics, 2017, 28(6): 1039-1045.