Attributes-based person re-identification via CNNs with coupled clusters loss

doi:10.21629/JSEE.2020.01.06

Journal of Systems Engineering and Electronics ›› 2020, Vol. 31 ›› Issue (1): 45-55.doi: 10.21629/JSEE.2020.01.06

• Electronics Technology • Previous Articles Next Articles

Attributes-based person re-identification via CNNs with coupled clusters loss

Rui SUN^1,^2,*(), Qiheng HUANG^1,²(), Wei FANG^1,²(), Xudong ZHANG^1,²()

¹ Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education), Hefei University of Technology, Hefei 230601, China
² School of Computer and Information, Hefei University of Technology, Hefei 230601, China

Received:2019-04-02 Online:2020-02-20 Published:2020-02-25
Contact: Rui SUN E-mail:sunrui@hfut.edu.cn;jchqh123@163.com;1204764020@qq.com;xudong@hfut.edu.cn
About author:SUN Rui was born in 1976. He is a Ph.D. and a professor in Hefei University of Technology. His research interests are computer vision, intelligent information processing and machine learning. E-mail: sunrui@hfut.edu.cn|HUANG Qiheng was born in 1994. He received his master degree from Hefei University of Technology. He is a R & D engineer in the China Mobile Anhui Branch. His research interests are computer vision and deep learning. E-mail: jchqh123@163.com|FANG Wei was born in 1993. He received his master degree from Hefei University of Technology. He is a R & D engineer in the ArcSoft Corporation Limited now. His research interests are computer vision and deep learning. E-mail: 1204764020@qq.com|ZHANG Xudong was born in 1966. He is a Ph.D. and a professor in Hefei University of Technology. His research interests are pattern recognition and intelligent information processing. E-mail: xudong@hfut.edu.cn
Supported by:
the National Natural Science Foundation of China(61471154);the National Natural Science Foundation of China(61876057);the Fundamental Research Funds for Central Universities(JZ2018YYPY0287);This work was supported by the National Natural Science Foundation of China (61471154; 61876057) and the Fundamental Research Funds for Central Universities (JZ2018YYPY0287)

Abstract

Abstract:

Person re-identification (re-id) involves matching a person across nonoverlapping views, with different poses, illuminations and conditions. Visual attributes are understandable semantic information to help improve the issues including illumination changes, viewpoint variations and occlusions. This paper proposes an end-to-end framework of deep learning for attribute-based person re-id. In the feature representation stage of framework, the improved convolutional neural network (CNN) model is designed to leverage the information contained in automatically detected attributes and learned low-dimensional CNN features. Moreover, an attribute classifier is trained on separate data and includes its responses into the training process of our person re-id model. The coupled clusters loss function is used in the training stage of the framework, which enhances the discriminability of both types of features. The combined features are mapped into the Euclidean space. The L2 distance can be used to calculate the distance between any two pedestrians to determine whether they are the same. Extensive experiments validate the superiority and advantages of our proposed framework over state-of-the-art competitors on contemporary challenging person re-id datasets.

Key words: person re-identification (re-id), convolutions neural network (CNN), attributes, coupled clusters loss (CCL)

Rui SUN, Qiheng HUANG, Wei FANG, Xudong ZHANG. Attributes-based person re-identification via CNNs with coupled clusters loss[J]. Journal of Systems Engineering and Electronics, 2020, 31(1): 45-55.

Figures/Tables 18

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

Fig 8

Fig 9

Fig 10

Table 1

Table 2

Fig 11

Fig 12

Table 3

Table 4

Table 5

Fig 13

References 42

1	GONG S, CRISTANI M, YAN S, et al. Person re-identification. London: Springer, 2014.
2	BEDAGKAR-GALA A, SHAH K S. A survey of approaches and trends in person re-identification. Image and Vision Computing, 2014, 32 (4): 270- 286. doi: 10.1016/j.imavis.2014.02.001
3	ZHAO R, OYANG W L, WANG X G. Person re-identification by saliency learning. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2017, 39 (2): 356- 370. doi: 10.1109/TPAMI.2016.2544310
4	GHEISSARI N, SEBASTIAN T, HARTLEY R. Person reidentification using spatiotemporal appearance. Proc. of the Computer Vision and Pattern Recognition, 2006, 1528- 1532.
5	FARENZENA M, BAZZANI L, PERINA A, et al. Person re-identification by symmetry-driven accumulation of local features. Proc. of Computer Vision and Pattern Recognition, 2010, 2- 8.
6	WAN X, GE D, LI P. Person re-identification by features fusion. Proc of the IEEE Information Technology, Networking, Electronic and Automation Control Conference, 2016, 285- 289.
7	KOESTINGER M, HIRZER M, WOHLHART P, et al. Large scale metric learning from equivalence constraints. Proc. of the Computer Vision and Pattern Recognition, 2012, 2288- 2295.
8	WEINBERGER Q K, BLITZER J, SAUL K L. Distance metric learning for large margin nearest neighbor classification. Proc. of the Neural Information Processing Systems, 2005, 1473- 1480.
9	ZHEN L, CHANG S Y, LIANG F, et al. Learning locally-adaptive decision functions for person verification. Proc. of the Computer Vision and Pattern Recognition, 2013, 3610- 3617.
10	HU J, LU J, TAN P Y. Deep transfer metric learning. IEEE Trans. on Image Processing, 2016, 25 (12): 5576- 5588. doi: 10.1109/TIP.2016.2612827
11	SUDOWE P, SPITZER H, LEIBE B. Person attribute recognition with a jointly-trained holistic cnn model. Proc. of the IEEE International Conference on Computer Vision Workshops, 2015, 87- 95.
12	HUANG J, FERIS S R, CHEN Q, et al. Cross-domain image retrieval with a dual attribute-aware ranking network. Proc. of the IEEE International Conference on Computer Vision, 2015, 1062- 1070.
13	MATSUKAWA T, SUZUKI E. Person re-identification using CNN features learned from combination of attributes. Proc. of the 23rd International Conference on Pattern Recognition, 2016, 2428- 2433.
14	LI W, ZHAO R, XIAO T, et al. Deepreid: deep filter pairing neural network for person re-identification. Proc. of the Computer Vision and Pattern Recognition, 2014, 152- 159.
15	CHENG D, GONG Y, ZHOU S, et al. Person re-identification by multi-channel parts-based CNN with improved triplet loss function. Proc. of the Computer Vision and Pattern Recognition, 2016, 1335- 1344.
16	VARIOR R R, HALOI M, WANG G. Gated siamese convolutional neural network architecture for human reidentification. Proc. of the European Conference on Computer Vision, 2016, 791- 808.
17	FARHADI A, ENDRES I, HOIEM D, et al. Describing objects by their attributes. Proc. of the Computer Vision and Pattern Recognition, 2009, 1778- 1785.
18	LAMPERT H C, NICKISCH H, HARMELING S. Attribute-based classification for zero-shot visual object categorization. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2014, 36 (3): 453- 465. doi: 10.1109/TPAMI.2013.140
19	RUSSAKOVSKY O, LI F F. Attribute learning in large-scale datasets. Proc. of the European Conference on Computer Vision, 2010, 1- 14.
20	JAYARAMAN D, SHA F, AND GRAUMAN K. Decorrelating semantic visual attributes by resisting the urge to share. Proc. of the Computer Vision and Pattern Recognition, 2014, 1629- 1636.
21	ZHANG H, YANG Y, LUAN H, et al. Start from scratch: Towards automatically identifying, modeling, and naming visual attributes. Proc. of the ACM Multimedia, 2014, 187- 196.
22	CHEN Y, HSU H W, LIAO M H. Automatic training image acquisition and effective feature selection from community-contributed photos for facial attribute detection. IEEE Trans. on Multimedia, 2013, 15 (6): 1388- 1399. doi: 10.1109/TMM.2013.2250492
23	LIN C, CHEN Y, CHEN B, et al. Facial attribute space compression by latent human topic discovery. Proc. of the ACM Multimedia, 2014, 1157- 1160.
24	ROMERO A, GATTA C, CAMPS-VALLS G. Unsupervised deep feature extraction for remote sensing image classification. IEEE Trans. on Geoscience and Remote Sensing, 2016, 54 (3): 1349- 1362. doi: 10.1109/TGRS.2015.2478379
25	YI D, LEI Z, LI S Z. Deep metric learning for practical person re-identification. Proc. of the International Conference on Pattern Recognition, 2014, 1- 8.
26	AHMED E, JONES M, MARKS T K. An improved deep learning architecture for person re-identification. Proc. of the Computer Vision and Pattern Recognition, 2015, 3908- 3916.
27	DING S, LIN L, WANG G, et al. Deep feature learning with relative distance comparison for person re-identification. Pattern Recognition, 2015, 48 (10): 2993- 3003. doi: 10.1016/j.patcog.2015.04.005
28	LIU H, FENG J, QI M, et al. End-to-end comparative attention networks for person re-identification. IEEE Trans. on Image Processing, 2017, 26 (7): 3492- 3506. doi: 10.1109/TIP.2017.2700762
29	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions. Proc. of the Computer Vision and Pattern Recognition, 2015, 1- 9.
30	DENG Y, LUO P, LOY C C, et al. Pedestrian attribute recognition at far distance. Proc. of the ACM Multimedia, 2014, 234- 241.
31	LIN D, ZHANG Z, CHEN X, et al. A richly annotated dataset for pedestrian attribute recognition. https://arXiv.org/abs/1603.07054.
32	SUDOWE P, SPITZER H, LEIBE B. Person attribute recognition with a jointly-trained holistic CNN model. Proc. of the International Conference on Computer Vision, 2015, 329- 334.
33	LI D, CHEN X, HUANG K. Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios. Proc. of the Asian Conference on Pattern Recognition, 2015, 111- 115.
34	YU K, LENG B, ZHANG Z, et al. Weakly supervised learning of mid-level features for pedestrian attribute recognition and localization. https://arXiv.org/abs/1611.05603.
35	WANG T, GONG S, ZHU X, et al. Person re-identification by video ranking. Proc. of the European Conference on Computer Vision, 2014, 688- 703.
36	LIU K, MA B, ZHANG W, et al. A spatio-temporal appearance representation for video-based pedestrian re-identification. Proc. of the International Conference on Computer Vision, 2015, 3810- 3818.
37	MCLAUGHLIN N, MARTINEZDEL RINCON J, MILLER P. Recurrent convolutional network for video-based person re-identification. Proc. of the Computer Vision and Pattern Recognition, 2016, 1325- 1334.
38	YOU J, WU A, LI X, et al. Top-push video-based person re-identification. Proc. of the Computer Vision and Pattern Recognition, 2016, 1345- 1353.
39	GAO C, WANG J, LIU L, et al. Temporally aligned pooling representation for video-based person re-identification. Proc. of the International Conference on Image Processing, 2016, 4284- 4288.
40	ZHU X, JING X Y, WU F, et al. Video-based person re-identification by simultaneously learning intra-video and inter-video distance metrics. IEEE Trans. on Image Processing, 2018, 28 (11): 5683- 5695.
41	XU S, CHENG Y, GU K, et al. Jointly attentive spatial-temporal pooling networks for video-based person re-identification. Proc. of the International Conference on Computer Vision, 2017, 4743- 4752.
42	DAI J, ZHANG P, WANG D, et al. Video person re-identification by temporal residual learning. IEEE Trans. on Image Processing, 2019, 28 (3): 1366- 1377.

Method	mA/%
ACN	81.15
DeepMAR	82.89
WPAL-GoogleNet-GMP	85.50
Ours	87.61

Attribute	mA/%
AgeAbove61	92.8
Male	92.3
Long hair	90.5
Stripes	76.4
Carry other	76.1
Logo	72.1
Hat	89.4
Muffler	89.3
Short sleeve	88.3
Sandals	76.1
Sunglasses	69.3
V-neck	63.5

Dataset	Person	Camera	Images	Size
iLIDS-VID	300	2	42 495	128$\times$64
PRID2011	200	2	24 541	128$\times$64

Method	$r=1$	$r=5$	$r=10$	$r=20$
STA [36]	44.3	71.7	83.7	91.7
RCNN [37]	58.3	84.6	92.0	96.7
TDL [38]	56.3	87.6	95.6	98.3
TAPR [39]	55.0	87.5	93.8	97.2
SI$^{2}$DL [40]	48.7	81.1	89.2	97.3
ASTPN [41]	62.0	86.0	94.0	98.0
TRL [42]	57.7	81.7	91.0	94.1
Ours	63.2	86.2	93.7	98.5

Method	$r=1$	$r=5$	$r=10$	$r=20$
STA [36]	64.1	87.3	89.9	92.0
RCNN [37]	70.6	92.3	95.3	97.3
TDL [38]	56.7	80.0	87.6	93.6
TAPR [39]	58.2	85.8	93.4	97.9
SI$^{2}$DL [40]	76.7	95.6	96.7	98.7
ASTPN [41]	74.0	95.0	99.0	99.0
TRL [42]	87.8	97.4	99.0	99.3
Ours	88.1	96.7	99.2	99.6

Attributes-based person re-identification via CNNs with coupled clusters loss

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 18

References 42

Related Articles 1

Recommended Articles

Metrics

Comments