Dual-stream coupling network with wavelet transform for cross-resolution person re-identification

doi:10.23919/JSEE.2023.000028

Journal of Systems Engineering and Electronics ›› 2023, Vol. 34 ›› Issue (3): 682-695.doi: 10.23919/JSEE.2023.000028

• ELECTRONICS TECHNOLOGY • Previous Articles

Dual-stream coupling network with wavelet transform for cross-resolution person re-identification

Rui SUN¹^,²(), Zi YANG¹^,²^,*(), Zhenghui ZHAO¹^,²(), Xudong ZHANG¹^,²()

¹ Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education), Hefei University of Technology, Hefei 230601, China
² School of Computer and Information, Hefei University of Technology, Hefei 230601, China

Received:2021-07-20 Online:2023-06-15 Published:2023-06-30
Contact: Zi YANG E-mail:sunrui@hfut.edu.cn;yangzi@mail.hfut.edu.cn;904150289@qq.com;xudong@hfut.edu.cn
About author:
SUN Rui was born in 1976. He received his B.S. degree from the Central South University of China, in 1998, M.S. degree from Harbin Engineering University of China, in 2000, and Ph.D. degree from Huazhong University of Science and Technology of China, in 2003. He worked as a senior software engineer with TCL Mobile Communication Company, China, from 2003 to 2005. He was a visiting scholar with the Computer Science Department, University of Missouri, Columbia, MO, USA, from 2010 to 2011. He was a postdoctoral researcher with Chery automobile Company, China, from 2012 to 2014. He is currently a professor with Hefei University of Technology, China. His research interests include object recognition and tracking, computer vision, and machine learning. E-mail: sunrui@hfut.edu.cn

YANG Zi was born in 1997. She received her B.S. degree from West Anhui University, in 2015. She is currently pursuing her M.S. degree with Hefei University of Technology. Her research interests include object recognition and tracking, computer vision, and image processing. E-mail: yangzi@mail.hfut.edu.cn

ZHAO Zhenghui was born in 1994. She received her B.S. degree from Hefei University of Technology, in 2015. She is currently pursuing her M.S. degree with Hefei University of Technology. Her research interests include object recognition and tracking, computer vision, and visible-infrared image processing. E-mail: 904150289@qq.com

ZHANG Xudong was born in 1966. He received his B.S. degree from Hefei University of Technology in 1989, M.S. degree from Hefei University of Technology in 1992, and Ph.D. degree from University of Science and Technology of China in 2005. He spent three months in collaborative research at Heilbronn University in Germany in 2006. Currently, he is a professor at Hefei University of Technology, China. His main research interests include image processing, pattern recognition, and intelligent information processing. E-mail: xudong@hfut.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61471154;61876057), and the Key Research and Development Program of Anhui Province-Special Project of Strengthening Science and Technology Police (202004D07020012).

Abstract

Abstract:

Person re-identification is a prevalent technology deployed on intelligent surveillance. There have been remarkable achievements in person re-identification methods based on the assumption that all person images have a sufficiently high resolution, yet such models are not applicable to the open world. In real world, the changing distance between pedestrians and the camera renders the resolution of pedestrians captured by the camera inconsistent. When low-resolution (LR) images in the query set are matched with high-resolution (HR) images in the gallery set, it degrades the performance of the pedestrian matching task due to the absent pedestrian critical information in LR images. To address the above issues, we present a dual-stream coupling network with wavelet transform (DSCWT) for the cross-resolution person re-identification task. Firstly, we use the multi-resolution analysis principle of wavelet transform to separately process the low-frequency and high-frequency regions of LR images, which is applied to restore the lost detail information of LR images. Then, we devise a residual knowledge constrained loss function that transfers knowledge between the two streams of LR images and HR images for accessing pedestrian invariant features at various resolutions. Extensive qualitative and quantitative experiments across four benchmark datasets verify the superiority of the proposed approach.

Key words: cross-resolution, feature invariant learning, person re-identification, residual knowledge transfer, wavelet transform

Rui SUN, Zi YANG, Zhenghui ZHAO, Xudong ZHANG. Dual-stream coupling network with wavelet transform for cross-resolution person re-identification[J]. Journal of Systems Engineering and Electronics, 2023, 34(3): 682-695.

Figures/Tables 10

Fig 1

Fig 2

Fig 3

Fig 4

Table 1

Table 2

Table 3

Table 4

Fig 5

Table 5

References 46

1	ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: past, present and future. https://arxiv.53yu.com/abs/1610.02984.
2	YE M, SHEN J B, LIN G L, et al Deep learning for person re-identification: a survey and outlook. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2021, 44 (6): 2872- 2893.
3	BAI S, TANG P, TORR P H S, et al Re-ranking via metric fusion for object retrieval and person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 740- 749.
4	CHANG X, HOSPEDALES T M, XIANG T Multi-level factorisation net for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, 2109- 2118.
5	CHEN Y C, ZHU X, ZHENG W S, et al Person re-identification by camera correlation aware feature augmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2017, 40 (2): 392- 408.
6	LI M X, ZHU X T, GONG S G Unsupervised tracklet person re-identification. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019, 42 (7): 1770- 1782.
7	SHEN Y T, LI H S, YI S, et al Person re-identification with deep similarity-guided graph neural network. Proc. of the European Conference on Computer Vision, 2018, 486- 504.
8	SONG J, YANG Y, SONG Y Z, et al Generalizable person re-identification by domain-invariant mapping network. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 719- 728.
9	XIAO T, LI H S, OUYANG W L, et al Learning deep feature representations with domain guided dropout for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 1249- 1258.
10	CHENG Z Y, DONG Q, GONG S G, et al Inter-task association critic for cross-resolution person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 2605- 2615.
11	JIAO J, ZHENG W S, WU A, et al. Deep low-resolution person re-identification. Proc. of the AAAI Conference on Artificial Intelligence. 2018, 32(1). DOI: https://doi.org/10.1609/aaai.v32i1.12284.
12	MAO S, ZHANG S, YANG M. Resolution-invariant person re-identification. https://arxiv.53yu.com/abs/1906.09748.
13	CHEN Y C, LI Y J, DU X, et al. Learning resolution-invariant deep representations for person re-identification. Proc. of the AAAI Conference on Artificial Intelligence, 2019, 33(1). DOI: http://doi.org/10.1609/aaai.v33i01.33018215.
14	LI X, ZHENG W S, WANG X, et al Multi-scale learning for low-resolution person re-identification. Proc. of the IEEE International Conference on Computer Vision, 2015, 3765- 3773.
15	HUANG Y, ZHA Z J, FU X, et al Real-world person re-identification via degradation invariance learning. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 14084- 14094.
16	WANG X D, ZHENG Z D, HE Y, et al. Progressive local filter pruning for image retrieval acceleration. https://arxiv.53yu.com/abs/2001.08878.
17	QUAN R J, DONG X Y, WU Y, et al Auto-reid: Searching for a part-aware convnet for person re-identification. Proc. of the IEEE/CVF International Conference on Computer Vision, 2019, 3750- 3759.
18	ZHANG H Z, LAN C L, ZENG W J, et al Densely semantically aligned person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 667- 676.
19	SUN Y F, CHENG C M, ZHANG Y H, et al Circle loss: a unified perspective of pair similarity optimization. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 6398- 6407.
20	FAN H H, ZHENG L, YAN C G, et al Unsupervised person re-identification: clustering and fine-tuning. ACM Trans. on Multimedia Computing, Communications, and Applications, 2018, 14 (4): 83.
21	ZHONG Z, ZHENG L, LUO Z M, et al Invariance matters: exemplar memory for domain adaptive person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 598- 607.
22	JING X Y, ZHU X, WU F, et al Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, 695- 704.
23	DONG C, LOY C C, HE K, et al Image super-resolution using deep convolutional networks. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2015, 38 (2): 295- 307.
24	TONG T, LI G, LIU X J, et al Image super-resolution using dense skip connections. Proc. of the IEEE International Conference on Computer Vision, 2017, 4799- 4807.
25	LEDIG C, THEIS L, HUSZAR F, et al Photo-realistic single image super-resolution using a generative adversarial network. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, 4681- 4690.
26	CHU X X, ZHANG B, MA H L, et al Fast, accurate and lightweight super-resolution with neural architecture search. Proc. of the IEEE 25th International Conference on Pattern Recognition, 2021, 59- 64.
27	ZHOU R F, SUSSTRUNK S Kernel modeling super-resolution on real low-resolution images. Proc. of the IEEE/CVF International Conference on Computer Vision, 2019, 2433- 2443.
28	HE Y G, BU X Z, JIANG M, et al Low bit rate underwater video image compression and coding method based on wavelet decomposition. China Communications, 2020, 17 (9): 210- 219. doi: 10.23919/JCC.2020.09.016
29	RAMAMONJISOA M, FIRMAN M, WATSON J, et al Single image depth prediction with wavelet decomposition. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, 11089- 11098.
30	SHAHREZAEI I H, KIM H C Fractal analysis and texture classification of high-frequency multiplicative noise in SAR sea-ice images based on a transform-domain image decomposition method. IEEE Access, 2020, 8, 40198- 40223. doi: 10.1109/ACCESS.2020.2976815
31	WIRSING K. Time frequency analysis of wavelet and Fourier transform. MOHAMMADY S. Wavelet Theory. London, UK: IntechOpen, 2020.
32	WANG Z, SIMONCELLI E P, BOVIK A C Multiscale structural similarity for image quality assessment. Proc. of the IEEE 37th Asilomar Conference on Signals, Systems & Computers, 2003, 2, 1398- 1402.
33	ZHENG L, SHEN L Y, TIAN L, et al Scalable person re-identification: a benchmark. Proc. of the IEEE International Conference on Computer Vision, 2015, 1116- 1124.
34	ZHENG Z D, ZHENG L, YANG Y Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. Proc. of the IEEE International Conference on Computer Vision, 2017, 3754- 3762.
35	LI W, ZHAO R, XIAO T, et al Deepreid: deep filter pairing neural network for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, 152- 159.
36	CHENG D S, CRISTANI M, STOPPA M, et al. Custom pictorial structures for re-identification. Proc. of the British Machine Vision Conference, 2011. DOI: 10.5244/C.25.68.
37	WANG X, DORETTO G, SEBASTIAN T, et al Shape and appearance context modeling. Proc. of the IEEE 11th International Conference on Computer Vision, 2007, 1- 8.
38	RONNEBERGER O, FISCHER P, BROX T U-net: convolutional networks for biomedical image segmentation. Proc. of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, 234- 241.
39	IOFFE S, SZEGEDY C Batch normalization: accelerating deep network training by reducing internal covariate shift. Proc. of the International Conference on Machine Learning, 2015, 448- 456.
40	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770−778.
41	KINGMA D P, BA J. A method for stochastic optimization. https://doi.org/10.48550/arXiv.1412.6980.
42	KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646−1654.
43	ZHONG Z, ZHENG L, ZHENG Z D, et al. Camera style adaptation for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 5157−5166.
44	GE Y X, LI Z W, ZHAO H Y, et al Fd-gan: pose-guided feature distilling gan for robust person re-identification. Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018, 1230- 1241.
45	WANG Z, HU R M, YU Y, et al. Scale-adaptive low-resolution person re-identification via learning a discriminating surface. Proc. of the 25th International Joint Conference on Artificial Intelligence, 2016: 2669–2675.
46	WANG Z, YE M, YANG F, et al. Cascaded SR-GAN for scale-adaptive low resolution person re-identification. Proc. of the 27th International Joint Conference on Artificial Intelligence, 2018: 3891–3897.

Method	Rank1/%	Rank5/%	mAP/%	PSNR/dB
SRCNN [23]	83.8	91.2	78.6	23.4470
VDSR [42]	84.9	91.3	80.6	24.7244
PEN w/o OE	69.0	82.7	63.8	20.3300
PEN	86.0	93.3	82.3	20.3428

Method	Rank1	Rank5	mAP
Single ResNet50	82.4	89.3	78.5
Two ResNet50	83.2	90.2	78.7
HRN	86.0	93.3	82.3

Method	Rank1	Rank5	mAP
Proposed method w/o ${L_L}$	84.5	89.9	80.4
Proposed method w/o ${L_H}$	84.2	91.4	80.6
Proposed method w/o ${L_{{\rm{rec}}} }$	83.3	90.9	79.5
Proposed method w/o ${L_{{\rm{res}}} }$	85.0	90.6	79.9
Proposed method w/o ${L_{{\rm{id}}} }$	1.0	3.1	3.1
Proposed method	86.0	93.3	82.3

Input	Rank1	Rank5	mAP
${I_{{\rm{lr}}} }{\text{(noWT)} }$	79.8	84.3	80.5
${I_{{\rm{lr}}} } + F_{{\rm{lr}}}^L$	84.9	91.7	80.3
${I_{{\rm{lr}}} } + F_{{\rm{lr}}}^H$	83.2	89.6	78.9
${I_{{\rm{lr}}} } + F_{{\rm{lr}}}^L + F_{{\rm{lr}}}^H$	86.0	93.3	82.3

Model	MLR-CUHK03		MLR-Market1501		MLR-DukeMTMC-REID		CAVIAR
Model	Rank1	Rank5	Rank1	Rank5	Rank1	Rank5	Rank1	Rank5
CamStyle [43]	69.1	89.6	74.5	88.6	64.0	78.1	32.1	72.3
FD-GAN [44]	73.4	93.8	79.6	91.6	67.5	82.0	33.5	71.4
JUDEA [14]	26.2	58.0	−	−	−	−	22.0	60.1
SLD²L [22]	−	−	−	−	−	−	18.4	44.8
SDF [45]	22.2	48.0	−	−	−	−	14.3	37.5
SING [11]	67.7	90.7	74.4	87.8	65.2	80.1	33.5	72.7
CSR-GAN [46]	71.3	92.1	76.4	88.5	67.6	81.4	34.7	72.5
RAIN [13]	78.9	97.3	−	−	−	−	42.0	77.3
FFSR+RIFE [12]	73.3	92.6	82.0	92.0	66.0	78.1	36.4	72.0
Our proposed model	86.0	93.3	83.3	92.9	75.9	86.4	40.4	72.8

Dual-stream coupling network with wavelet transform for cross-resolution person re-identification

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 10

References 46

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Meiyan PAN, Jun SUN, Yuhao YANG, Dasheng LI, Sudao XIE, Shengli WANG, Jianjun CHEN. Improved TQWT for marine moving target detection [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 470-481.
[2]	Rui SUN, Qiheng HUANG, Wei FANG, Xudong ZHANG. Attributes-based person re-identification via CNNs with coupled clusters loss [J]. Journal of Systems Engineering and Electronics, 2020, 31(1): 45-55.
[3]	Junhua YAN, Xuehan BAI, Wanyi ZHANG, Yongqi XIAO, Chris CHATWIN, Rupert YOUNG, Phil BIRCH. No-reference image quality assessment based on AdaBoost BP neural network in wavelet domain [J]. Journal of Systems Engineering and Electronics, 2019, 30(2): 223-237.
[4]	Emad ELSHAZLY, Safey ABDELWAHAB, Refaat ABOUZAID, Osama ZAHRAN, Sayed ELARABY, Mohamed ELKORDY. A secure image steganography algorithm based on least significant bit and integer wavelet transform [J]. Journal of Systems Engineering and Electronics, 2018, 29(3): 639-649.
[5]	Zhaoyang Qiu, Pei Wang, Jun Zhu, and Bin Tang. NYFR output pulse radar signal TOA analysis using extended Fourier transform and its TOA estimation [J]. Systems Engineering and Electronics, 2017, 28(2): 212-223.
[6]	Chao Wu, Luping Xu, Hua Zhang, and Wenbo Zhao. Code acquisition method based on wavelet transform filtering [J]. Systems Engineering and Electronics, 2015, 26(6): 1169-1176.
[7]	Zhizhong Fu, Yanjing Yang, Chang Shu, Yuan Li, Honggang Wu, and Jin Xu. Improved single image dehazing using dark channel prior [J]. Journal of Systems Engineering and Electronics, 2015, 26(5): 1070-1079.
[8]	Fang Deng, Jie Chen, and Chen Chen. Adaptive unscented Kalman filter for parameter and state estimation of nonlinear high-speed objects [J]. Journal of Systems Engineering and Electronics, 2013, 24(4): 655-665.
[9]	Qian Huaming & Ma Jichen. Research on fiber optic gyro signal de-noising based on wavelet packet soft-threshold [J]. Journal of Systems Engineering and Electronics, 2009, 20(3): 607-612.
[10]	Shang Zhengguo, Zhao Chunhui & Wan Jian. Application of multi-resolution analysis in sonar image denoising [J]. Journal of Systems Engineering and Electronics, 2008, 19(6): 1082-1089.
[11]	Chen Shanxue, Li Fangwei & Zhu Weile. Fast encoding algorithm for vector quantization based on subvector L2-norm [J]. Journal of Systems Engineering and Electronics, 2008, 19(3): 611-617.
[12]	Kang Zhiwei, Liu Jing & He Yigang. Steganography based on wavelet transform and modulus function [J]. Journal of Systems Engineering and Electronics, 2007, 18(3): 628-632.
[13]	Wang Xuewei, Liu Songtao & Zhou Xiaodong. New algorithm for infrared small target image enhancement based on wavelet transform and human visual properties [J]. Journal of Systems Engineering and Electronics, 2006, 17(2): 268-273.
[14]	Xiong Chengyi, Tian Jinwen & Liu Jian. Low complexity reconfigurable architecture for the 5/3 and 9/7 discrete wavelet transform [J]. Journal of Systems Engineering and Electronics, 2006, 17(2): 303-308.
[15]	Li Jian, Zhou Jiliu, Wang Ybng & Zhi Yong. Synchronization of two 3-scroll hyperchaotic attractors using wavelet transform [J]. Journal of Systems Engineering and Electronics, 2006, 17(2): 387-389.