A combined algorithm of K-means and MTRL for multi-class classification

doi:10.21629/JSEE.2019.05.06

Journal of Systems Engineering and Electronics ›› 2019, Vol. 30 ›› Issue (5): 875-885.doi: 10.21629/JSEE.2019.05.06

收稿日期:2018-10-15 出版日期:2019-10-08 发布日期:2019-10-09

A combined algorithm of K-means and MTRL for multi-class classification

Mengfan XUE(), NLei HA(), Dongliang PENG*()

School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China

Received:2018-10-15 Online:2019-10-08 Published:2019-10-09
Contact: Dongliang PENG E-mail:xuemf@hdu.edu.cn;hanlei52001@qq.com;dlpeng@hdu.edu.cn
About author:XUE Mengfan was born in 1990. She received her Ph.D. degree from Xidian University in 2016. She is currently a lecturer in Hangzhou Dianzi University. Her research interests are signal processing and machine learning. E-mail: xuemf@hdu.edu.cn|HAN Lei was born in 1994. He received his M.S. degree from Hangzhou Dianzi University. His research interests are machine learning and automatic target recognition. E-mail: hanlei52001@qq.com|PENG Dongliang was born in 1977. He received his B.S. and M.S. degrees in flight vehicle design and engineering from the Harbin Institute of Technology, Harbin, China, in 1998 and 2000, respectively, and his Ph.D. degree in control science and engineering from Zhejiang University, Hangzhou, China, in 2013. In 2003, he joined Hangzhou Dianzi University, Hangzhou, China, where he is currently a professor in the School of Automation. His research interests include information fusion and estimated theory. E-mail: dlpeng@hdu.edu.cn
Supported by:
the National Natural Science Foundation of China(61703131);the National Natural Science Foundation of China(61703129);the National Natural Science Foundation of China(61701148);the National Natural Science Foundation of China(61703128);This work was supported by the National Natural Science Foundation of China (61703131; 61703129; 61701148; 61703128)

摘要/Abstract

Abstract:

The basic idea of multi-class classification is a disassembly method, which is to decompose a multi-class classification task into several binary classification tasks. In order to improve the accuracy of multi-class classification in the case of insufficient samples, this paper proposes a multi-class classification method combining K-means and multi-task relationship learning (MTRL). The method first uses the split method of One vs. Rest to disassemble the multi-class classification task into binary classification tasks. K-means is used to down sample the dataset of each task, which can prevent over-fitting of the model while reducing training costs. Finally, the sampled dataset is applied to the MTRL, and multiple binary classifiers are trained together. With the help of MTRL, this method can utilize the inter-task association to train the model, and achieve the purpose of improving the classification accuracy of each binary classifier. The effectiveness of the proposed approach is demonstrated by experimental results on the Iris dataset, Wine dataset, Multiple Features dataset, Wireless Indoor Localization dataset and Avila dataset.

Key words: machine learning, multi-class classification, K-means, multi-task relationship learning (MTRL), over-fitting

. [J]. Journal of Systems Engineering and Electronics, 2019, 30(5): 875-885.

Mengfan XUE, NLei HA, Dongliang PENG. A combined algorithm of K-means and MTRL for multi-class classification[J]. Journal of Systems Engineering and Electronics, 2019, 30(5): 875-885.

图/表 14

参考文献 31

1	ZHOU Z H. Machine learning. Beijing: Tsinghua University Press, 2016.
2	KNERR S, PERSONNAZ L, DREYFUS G. Single-layer learning revisited:a stepwise procedure for building and training a neural network. Neurocomputing, 1990, 68 (11): 41- 50.
3	FRIEDMAN J H. Another approach to polychotomous classification. Technical Report, Departmet of Statistics, Stanford University, 1996.
4	KREBEL U H G. Pairwise classification and support vector machines. Advances in kernel methods: support vector learning. Cambridge, MA: MIT Press, 1999: 255-268.
5	DING S F, ZHANG X K, AN Y X, et al. Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification. Pattern Recognition, 2017, 67, 32- 46. doi: 10.1016/j.patcog.2017.02.011
6	ZHANG Y, LI B, LU H, et al. Sample-specific SVM learning for person re-identification. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 1278- 1287.
7	AN Y X, DING S F, SHI S H, et al. Discrete space reinforcement learning algorithm based on support vector machine classification. Pattern Recognition Letters, 2018, 111, 30- 35. doi: 10.1016/j.patrec.2018.04.012
8	DING S F, AN Y X, ZHANG X K, et al. Wavelet twin support vector machines based on glowworm swarm optimization. Neurocomputing, 2017, 225 (C): 157- 163.
9	DING S F, ZHANG X K, YU J Z. Twin support vector machines based on fruit fly optimization algorithm. International Journal of Machine Learning and Cybernetics, 2016, 7 (2): 193- 203. doi: 10.1007/s13042-015-0424-8
10	MUCHLINSKI D, SIROKY D, HE J R, et al. Comparing random forest with logistic regression for predicting classimbalanced civil war onset data. Political Analysis, 2017, 24 (1): 87- 103.
11	GUO H, LI Y, LI Y, et al. BPSO-adaboost-KNN ensemble learning algorithm for multi-class imbalanced data classification. Engineering Applications of Artificial Intelligence, 2016, 49 (C): 176- 193.
12	DONG G, KUANG G, WANG N, et al. SAR target recognition via joint sparse representation of monogenic signal. IEEE Journal of Selected Topics in Applied Earth Observations&Remote Sensing, 2015, 8 (7): 3316- 3328.
13	ZHENG H, GENG X, TAO D, et al. A multi-task model for simultaneous face identification and facial expression recognition. Neurocomputing, 2016, 171 (C): 515- 523.
14	LI C, BAO W, XU L, et al. Clustered multi-task learning for automatic radar target recognition. Sensors, 2017, 17 (10): 2218. doi: 10.3390/s17102218
15	BICKEL P J, RITOV Y, TSYBAKOV A B. Simultaneous analysis of Lasso and Dantzig selector. Annals of Statistics, 2009, 37 (4): 1705- 1732. doi: 10.1214/08-AOS620
16	ZU C, JIE B, LIU M, et al. Label-aligned multi-task feature learning for multimodal classification of Alzheimer's disease and mild cognitive impairment. Brain Imaging&Behavior, 2016, 10 (4): 1148- 1159.
17	LIU A, LU Y, NIE W, et al. HEp-2 cells classification via clustered multi-task learning. Neurocomputing, 2016, 195 (C): 195- 201.
18	ZHANG Y, YEUNG D Y. A regularization approach to learning task relationships in multitask learning. ACM Trans. on Knowledge Discovery from Data, 2014, 8 (3): 1- 31.
19	EVGENIOU T, PONTIL M. Regularized multi-task learning. Proc. of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, 109- 117.
20	ARGYRIOU A, EVGENIOU T, PONTIL M. Multi-task feature learning. Proc. of the 20th Annual Conference on Neural Information Processing Systems, 2007, 41- 48.
21	ARGYRIOU A, EVGENIOU T, PONTIL M. Convex multitask feature learning. Machine Learning, 2008, 73 (3): 243- 272. doi: 10.1007/s10994-007-5040-8
22	KEERTHI S S, SHEVADE S K. SMO algorithm for leastsquares SVM formulations. Neural Computation, 2003, 15 (2): 487- 507. doi: 10.1162/089976603762553013
23	PIATETSKY-SHAPIRO G. Advances in knowledge discovery and data mining. Menlo Park: AAAI Press, 1996.
24	DUDA R O, HART P E, STORK D G. Pattern classification and scene analysis. 2nd ed. New York: Wiley Interscience, 1995.
25	FELLER W. An introduction to probability theory and its applications. New York: Wiley, 1968.
26	JAIN A K. Data clustering:50 years beyond K-means. Pattern Recognition Letters, 2010, 31 (8): 651- 666. doi: 10.1016/j.patrec.2009.09.011
27	CHADHA A, KUMAR S. An improved K-means clustering algorithm: a step forward for removal of dependency on K. Proc. of the IEEE International Conference on Optimization, Reliabilty, and Information Technology, 2014: 136-140.
28	KAIE S S, BERE S S. An efficient K-means clustering algorithm. International Journal of Engineering Education and Technology, 2015, 3 (2): 1- 8.
29	EL AGHA M, ASHOUR W M. Efficient and fast initialization algorithm for k-means clustering. International Journal of Intelligent Systems and Applications, 2012, 4 (1): 21. doi: 10.5815/ijisa.2012.01.03
30	CHANG C C, LIN C J. LIBSVM: a library for support vector machines. ACM Trans. on Intelligent Systems and Technology, 2011, 2(3): 27.
31	GONG P, YE J, ZHANG C. Robust multi-task feature learning. Proc. of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2012: 895-903.

Class	task1	task2	task3
Positive class	class1	class2	class3
Negative class	class2, class3	class1, class3	class1, class2

Data set		Selected category		Feature dimension	Total number
Iris	setosa	versicolor	virginica	4	150
Wine	1	2	3	13	178
Multiple Features (Zernike moments)	0	1, 2, 3	4	47	1 000
WIL	1	2, 3	4	7	2 000
Avila	E	F, H, II	X	10	4 928(train) 931(test)

Reality	Prediction
Reality	Positive class	Negative dass
Positive class	TP	FN
Negative class	FP	TN

Method	Description
KNN	K-nearest neighbor classifier
SVM	Support vector machine
rMTFL	Select features based on related tasks and identify irrelevant tasks

Dataset	Without K-means/s	With K-means/s
Iris	0.342 9	0.129 0
Wine	0.624 0	0.061 7
WIL	244.083 4	19.982 7
Multiple Features	29.798 0	2.764 8
Avila	13 026	1 347.2

A combined algorithm of K-means and MTRL for multi-class classification

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 31

相关文章 0

编辑推荐

Metrics

本文评价