Dual-stream coupling network with wavelet transform for cross-resolution person re-identification

doi:10.23919/JSEE.2023.000028

Journal of Systems Engineering and Electronics ›› 2023, Vol. 34 ›› Issue (3): 682-695.doi: 10.23919/JSEE.2023.000028

• • 上一篇

收稿日期:2021-07-20 出版日期:2023-06-15 发布日期:2023-06-30

Dual-stream coupling network with wavelet transform for cross-resolution person re-identification

Rui SUN¹^,²(), Zi YANG¹^,²^,*(), Zhenghui ZHAO¹^,²(), Xudong ZHANG¹^,²()

¹ Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education), Hefei University of Technology, Hefei 230601, China
² School of Computer and Information, Hefei University of Technology, Hefei 230601, China

Received:2021-07-20 Online:2023-06-15 Published:2023-06-30
Contact: Zi YANG E-mail:sunrui@hfut.edu.cn;yangzi@mail.hfut.edu.cn;904150289@qq.com;xudong@hfut.edu.cn
About author:
SUN Rui was born in 1976. He received his B.S. degree from the Central South University of China, in 1998, M.S. degree from Harbin Engineering University of China, in 2000, and Ph.D. degree from Huazhong University of Science and Technology of China, in 2003. He worked as a senior software engineer with TCL Mobile Communication Company, China, from 2003 to 2005. He was a visiting scholar with the Computer Science Department, University of Missouri, Columbia, MO, USA, from 2010 to 2011. He was a postdoctoral researcher with Chery automobile Company, China, from 2012 to 2014. He is currently a professor with Hefei University of Technology, China. His research interests include object recognition and tracking, computer vision, and machine learning. E-mail: sunrui@hfut.edu.cn

YANG Zi was born in 1997. She received her B.S. degree from West Anhui University, in 2015. She is currently pursuing her M.S. degree with Hefei University of Technology. Her research interests include object recognition and tracking, computer vision, and image processing. E-mail: yangzi@mail.hfut.edu.cn

ZHAO Zhenghui was born in 1994. She received her B.S. degree from Hefei University of Technology, in 2015. She is currently pursuing her M.S. degree with Hefei University of Technology. Her research interests include object recognition and tracking, computer vision, and visible-infrared image processing. E-mail: 904150289@qq.com

ZHANG Xudong was born in 1966. He received his B.S. degree from Hefei University of Technology in 1989, M.S. degree from Hefei University of Technology in 1992, and Ph.D. degree from University of Science and Technology of China in 2005. He spent three months in collaborative research at Heilbronn University in Germany in 2006. Currently, he is a professor at Hefei University of Technology, China. His main research interests include image processing, pattern recognition, and intelligent information processing. E-mail: xudong@hfut.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61471154;61876057), and the Key Research and Development Program of Anhui Province-Special Project of Strengthening Science and Technology Police (202004D07020012).

摘要/Abstract

Abstract:

Person re-identification is a prevalent technology deployed on intelligent surveillance. There have been remarkable achievements in person re-identification methods based on the assumption that all person images have a sufficiently high resolution, yet such models are not applicable to the open world. In real world, the changing distance between pedestrians and the camera renders the resolution of pedestrians captured by the camera inconsistent. When low-resolution (LR) images in the query set are matched with high-resolution (HR) images in the gallery set, it degrades the performance of the pedestrian matching task due to the absent pedestrian critical information in LR images. To address the above issues, we present a dual-stream coupling network with wavelet transform (DSCWT) for the cross-resolution person re-identification task. Firstly, we use the multi-resolution analysis principle of wavelet transform to separately process the low-frequency and high-frequency regions of LR images, which is applied to restore the lost detail information of LR images. Then, we devise a residual knowledge constrained loss function that transfers knowledge between the two streams of LR images and HR images for accessing pedestrian invariant features at various resolutions. Extensive qualitative and quantitative experiments across four benchmark datasets verify the superiority of the proposed approach.

Key words: cross-resolution, feature invariant learning, person re-identification, residual knowledge transfer, wavelet transform

. [J]. Journal of Systems Engineering and Electronics, 2023, 34(3): 682-695.

Rui SUN, Zi YANG, Zhenghui ZHAO, Xudong ZHANG. Dual-stream coupling network with wavelet transform for cross-resolution person re-identification[J]. Journal of Systems Engineering and Electronics, 2023, 34(3): 682-695.

图/表 10

参考文献 46

1	ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: past, present and future. https://arxiv.53yu.com/abs/1610.02984.
2	YE M, SHEN J B, LIN G L, et al Deep learning for person re-identification: a survey and outlook. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2021, 44 (6): 2872- 2893.
3	BAI S, TANG P, TORR P H S, et al Re-ranking via metric fusion for object retrieval and person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 740- 749.
4	CHANG X, HOSPEDALES T M, XIANG T Multi-level factorisation net for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, 2109- 2118.
5	CHEN Y C, ZHU X, ZHENG W S, et al Person re-identification by camera correlation aware feature augmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2017, 40 (2): 392- 408.
6	LI M X, ZHU X T, GONG S G Unsupervised tracklet person re-identification. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019, 42 (7): 1770- 1782.
7	SHEN Y T, LI H S, YI S, et al Person re-identification with deep similarity-guided graph neural network. Proc. of the European Conference on Computer Vision, 2018, 486- 504.
8	SONG J, YANG Y, SONG Y Z, et al Generalizable person re-identification by domain-invariant mapping network. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 719- 728.
9	XIAO T, LI H S, OUYANG W L, et al Learning deep feature representations with domain guided dropout for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 1249- 1258.
10	CHENG Z Y, DONG Q, GONG S G, et al Inter-task association critic for cross-resolution person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 2605- 2615.
11	JIAO J, ZHENG W S, WU A, et al. Deep low-resolution person re-identification. Proc. of the AAAI Conference on Artificial Intelligence. 2018, 32(1). DOI: https://doi.org/10.1609/aaai.v32i1.12284.
12	MAO S, ZHANG S, YANG M. Resolution-invariant person re-identification. https://arxiv.53yu.com/abs/1906.09748.
13	CHEN Y C, LI Y J, DU X, et al. Learning resolution-invariant deep representations for person re-identification. Proc. of the AAAI Conference on Artificial Intelligence, 2019, 33(1). DOI: http://doi.org/10.1609/aaai.v33i01.33018215.
14	LI X, ZHENG W S, WANG X, et al Multi-scale learning for low-resolution person re-identification. Proc. of the IEEE International Conference on Computer Vision, 2015, 3765- 3773.
15	HUANG Y, ZHA Z J, FU X, et al Real-world person re-identification via degradation invariance learning. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 14084- 14094.
16	WANG X D, ZHENG Z D, HE Y, et al. Progressive local filter pruning for image retrieval acceleration. https://arxiv.53yu.com/abs/2001.08878.
17	QUAN R J, DONG X Y, WU Y, et al Auto-reid: Searching for a part-aware convnet for person re-identification. Proc. of the IEEE/CVF International Conference on Computer Vision, 2019, 3750- 3759.
18	ZHANG H Z, LAN C L, ZENG W J, et al Densely semantically aligned person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 667- 676.
19	SUN Y F, CHENG C M, ZHANG Y H, et al Circle loss: a unified perspective of pair similarity optimization. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 6398- 6407.
20	FAN H H, ZHENG L, YAN C G, et al Unsupervised person re-identification: clustering and fine-tuning. ACM Trans. on Multimedia Computing, Communications, and Applications, 2018, 14 (4): 83.
21	ZHONG Z, ZHENG L, LUO Z M, et al Invariance matters: exemplar memory for domain adaptive person re-identification. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 598- 607.
22	JING X Y, ZHU X, WU F, et al Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, 695- 704.
23	DONG C, LOY C C, HE K, et al Image super-resolution using deep convolutional networks. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2015, 38 (2): 295- 307.
24	TONG T, LI G, LIU X J, et al Image super-resolution using dense skip connections. Proc. of the IEEE International Conference on Computer Vision, 2017, 4799- 4807.
25	LEDIG C, THEIS L, HUSZAR F, et al Photo-realistic single image super-resolution using a generative adversarial network. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, 4681- 4690.
26	CHU X X, ZHANG B, MA H L, et al Fast, accurate and lightweight super-resolution with neural architecture search. Proc. of the IEEE 25th International Conference on Pattern Recognition, 2021, 59- 64.
27	ZHOU R F, SUSSTRUNK S Kernel modeling super-resolution on real low-resolution images. Proc. of the IEEE/CVF International Conference on Computer Vision, 2019, 2433- 2443.
28	HE Y G, BU X Z, JIANG M, et al Low bit rate underwater video image compression and coding method based on wavelet decomposition. China Communications, 2020, 17 (9): 210- 219. doi: 10.23919/JCC.2020.09.016
29	RAMAMONJISOA M, FIRMAN M, WATSON J, et al Single image depth prediction with wavelet decomposition. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, 11089- 11098.
30	SHAHREZAEI I H, KIM H C Fractal analysis and texture classification of high-frequency multiplicative noise in SAR sea-ice images based on a transform-domain image decomposition method. IEEE Access, 2020, 8, 40198- 40223. doi: 10.1109/ACCESS.2020.2976815
31	WIRSING K. Time frequency analysis of wavelet and Fourier transform. MOHAMMADY S. Wavelet Theory. London, UK: IntechOpen, 2020.
32	WANG Z, SIMONCELLI E P, BOVIK A C Multiscale structural similarity for image quality assessment. Proc. of the IEEE 37th Asilomar Conference on Signals, Systems & Computers, 2003, 2, 1398- 1402.
33	ZHENG L, SHEN L Y, TIAN L, et al Scalable person re-identification: a benchmark. Proc. of the IEEE International Conference on Computer Vision, 2015, 1116- 1124.
34	ZHENG Z D, ZHENG L, YANG Y Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. Proc. of the IEEE International Conference on Computer Vision, 2017, 3754- 3762.
35	LI W, ZHAO R, XIAO T, et al Deepreid: deep filter pairing neural network for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, 152- 159.
36	CHENG D S, CRISTANI M, STOPPA M, et al. Custom pictorial structures for re-identification. Proc. of the British Machine Vision Conference, 2011. DOI: 10.5244/C.25.68.
37	WANG X, DORETTO G, SEBASTIAN T, et al Shape and appearance context modeling. Proc. of the IEEE 11th International Conference on Computer Vision, 2007, 1- 8.
38	RONNEBERGER O, FISCHER P, BROX T U-net: convolutional networks for biomedical image segmentation. Proc. of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, 234- 241.
39	IOFFE S, SZEGEDY C Batch normalization: accelerating deep network training by reducing internal covariate shift. Proc. of the International Conference on Machine Learning, 2015, 448- 456.
40	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770−778.
41	KINGMA D P, BA J. A method for stochastic optimization. https://doi.org/10.48550/arXiv.1412.6980.
42	KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646−1654.
43	ZHONG Z, ZHENG L, ZHENG Z D, et al. Camera style adaptation for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 5157−5166.
44	GE Y X, LI Z W, ZHAO H Y, et al Fd-gan: pose-guided feature distilling gan for robust person re-identification. Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018, 1230- 1241.
45	WANG Z, HU R M, YU Y, et al. Scale-adaptive low-resolution person re-identification via learning a discriminating surface. Proc. of the 25th International Joint Conference on Artificial Intelligence, 2016: 2669–2675.
46	WANG Z, YE M, YANG F, et al. Cascaded SR-GAN for scale-adaptive low resolution person re-identification. Proc. of the 27th International Joint Conference on Artificial Intelligence, 2018: 3891–3897.

Method	Rank1/%	Rank5/%	mAP/%	PSNR/dB
SRCNN [23]	83.8	91.2	78.6	23.4470
VDSR [42]	84.9	91.3	80.6	24.7244
PEN w/o OE	69.0	82.7	63.8	20.3300
PEN	86.0	93.3	82.3	20.3428

Method	Rank1	Rank5	mAP
Single ResNet50	82.4	89.3	78.5
Two ResNet50	83.2	90.2	78.7
HRN	86.0	93.3	82.3

Method	Rank1	Rank5	mAP
Proposed method w/o ${L_L}$	84.5	89.9	80.4
Proposed method w/o ${L_H}$	84.2	91.4	80.6
Proposed method w/o ${L_{{\rm{rec}}} }$	83.3	90.9	79.5
Proposed method w/o ${L_{{\rm{res}}} }$	85.0	90.6	79.9
Proposed method w/o ${L_{{\rm{id}}} }$	1.0	3.1	3.1
Proposed method	86.0	93.3	82.3

Input	Rank1	Rank5	mAP
${I_{{\rm{lr}}} }{\text{(noWT)} }$	79.8	84.3	80.5
${I_{{\rm{lr}}} } + F_{{\rm{lr}}}^L$	84.9	91.7	80.3
${I_{{\rm{lr}}} } + F_{{\rm{lr}}}^H$	83.2	89.6	78.9
${I_{{\rm{lr}}} } + F_{{\rm{lr}}}^L + F_{{\rm{lr}}}^H$	86.0	93.3	82.3

Model	MLR-CUHK03		MLR-Market1501		MLR-DukeMTMC-REID		CAVIAR
Model	Rank1	Rank5	Rank1	Rank5	Rank1	Rank5	Rank1	Rank5
CamStyle [43]	69.1	89.6	74.5	88.6	64.0	78.1	32.1	72.3
FD-GAN [44]	73.4	93.8	79.6	91.6	67.5	82.0	33.5	71.4
JUDEA [14]	26.2	58.0	−	−	−	−	22.0	60.1
SLD²L [22]	−	−	−	−	−	−	18.4	44.8
SDF [45]	22.2	48.0	−	−	−	−	14.3	37.5
SING [11]	67.7	90.7	74.4	87.8	65.2	80.1	33.5	72.7
CSR-GAN [46]	71.3	92.1	76.4	88.5	67.6	81.4	34.7	72.5
RAIN [13]	78.9	97.3	−	−	−	−	42.0	77.3
FFSR+RIFE [12]	73.3	92.6	82.0	92.0	66.0	78.1	36.4	72.0
Our proposed model	86.0	93.3	83.3	92.9	75.9	86.4	40.4	72.8

Dual-stream coupling network with wavelet transform for cross-resolution person re-identification

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 46

相关文章 0

编辑推荐

Metrics

本文评价