Abstract
The kernel function method in support vector machine (SVM) is an excellent tool for nonlinear classification. How to design a kernel function is difficult for an SVM nonlinear classification problem, even for the polynomial kernel function. In this paper, we propose a new kind of polynomial kernel functions, called semi-tensor product kernel (STP-kernel), for an SVM nonlinear classification problem by semi-tensor product of matrix (STP) theory. We have shown the existence of the STP-kernel function and verified that it is just a polynomial kernel. In addition, we have shown the existence of the reproducing kernel Hilbert space (RKHS) associated with the STP-kernel function. Compared to the existing methods, it is much easier to construct the nonlinear feature mapping for an SVM nonlinear classification problem via an STP operator.
Similar content being viewed by others
Change history
12 December 2022
The author portraits have been interchanged. This has been corrected.
References
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273–297.
Chapelle, O., Haffner, P., & Vapnik, V. N. (1999). Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks, 10(5), 1055–1064.
DeCoste, D., & Schölkopf, B. (2002). Training invariant support vector machines. Machine Learning, 46(1), 161–190.
Pradhan, S., Hacioglu, K., Krugler, V., Ward, W., Martin, J. H., & Jurafsky, D. (2005). Support vector learning for semantic argument classification. Machine Learning, 60(1), 11–39.
Statnikov, A., Hardin, D., & Aliferis, C. (2006). Using svm weight-based methods to identify causally relevant and non-causally relevant variables. Sign, 1(4), 474–484.
Maity, A. (2016). Supervised classification of radarsat-2 polarimetric data for different land features. arXiv preprint. arXiv:1608.00501.
Wenzel, F., Galy-Fajou, T., Deutsch, M., & Kloft, M. (2017). Bayesian nonlinear support vector machines for big data. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 307–322. Springer.
Zhou, C., Chase, J. G., & Rodgers, G. W. (2021). Support vector machines for automated modelling of nonlinear structures using health monitoring results. Mechanical Systems and Signal Processing, 149, 107201.
Tanveer, M., Rajani, T., Rastogi, R., Shao, Y.-H., & Ganaie, M. (2022). Comprehensive review on twin support vector machines. Annals of Operations Research. https://doi.org/10.1007/s10479-022-04575-w
Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions On Neural Networks, 10(5), 988–999.
Cheng, D. (2016). On equivalence of matrices. arXiv preprint. arXiv:1605.09523.
Cheng, D., & Qi, H. (2009). Controllability and observability of boolean control networks. Automatica, 45(7), 1659–1667.
Li, L., Chen, Y., Peng, H., & Yang, Y. (2020). Chaotic deep network for mobile d2d communication. IEEE Internet of Things Journal, 8(10), 8078–8096.
Cheng, D., Zhao, Y., & Mu, Y. (2010). Strategy optimization with its application to dynamic games. In: The 49th IEEE Conference on Decision and Control (CDC), pp. 5822–5827. IEEE.
Feng, J.-E., Lv, H., & Cheng, D. (2013). Multiple fuzzy relation and its application to coupled fuzzy control. Asian Journal of Control, 15(5), 1313–1324.
Cheng, D., Qi, H., & Zhao, Y. (2012). An Introduction to Semi-tensor Product of Matrices and Its Applications. Singapore: World Scientific.
Gao, Z., Fang, S.-C., Luo, J., & Medhin, N. (2021). A kernel-free double well potential support vector machine with applications. European Journal of Operational Research, 290(1), 248–262.
Berlinet, A., & Thomas-Agnan, C. (2011). Reproducing Kernel Hilbert Spaces in Probability and Statistics. New York: Springer.
Cucker, F., & Zhou, D. X. (2007). Learning Theory: an Approximation Theory Viewpoint (Vol. 24). Cambridge, MA: Cambridge University Press.
Mohammed, J., & Meira, W., Jr. (2020). Data Mining and Machine Learning: Fundamental Concepts and Algorithms. Cambridge, UK: Cambridge University.
Tauber, S. (1963). On multinomial coefficients. The American Mathematical Monthly, 70(10), 1058–1063.
Acknowledgements
We thank Prof. Daizhan Cheng for giving some valuable suggestions, and also the anonymous reviewers for their helpful comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Natural Science Foundation of China (61573288), the Key Programs in Shaanxi Province of China (2021JZ-12) and the Yulin Science and Technology Bureau project (2019-89-2).
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xue, S., Zhang, L. & Zhu, Z. Design of semi-tensor product-based kernel function for SVM nonlinear classification. Control Theory Technol. 20, 456–464 (2022). https://doi.org/10.1007/s11768-022-00120-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11768-022-00120-z