Abstract
In this paper, a distributed scheme is proposed for ensemble learning method of bagging, which aims to address the classification problems for large dataset by developing a group of cooperative logistic regression learners in a connected network. Moveover, each weak learner/agent can share the local weight vector with its immediate neighbors through diffusion strategy in a fully distributed manner. Our diffusion logistic regression algorithms can effectively avoid overfitting and obtain high classification accuracy compared to the non-cooperation mode. Furthermore, simulations with a real dataset are given to demonstrate the effectiveness of the proposed methods in comparison with the centralized one.
Similar content being viewed by others
References
R. M. Harbord, P. Whiting. Metand: meta-analysis of diagnostic accuracy using hierarchical logistic regression. The Stata Journal, 2009, 9(2): 211–229.
C. L. Tsien, H. S. Fraser, W. J. Long, et al. Using classification tree and logistic regression methods to diagnose myocardial infarction. Proceedings of the 9th World Congress on Medical Informatics, Seoul: IOS Press, 1998: 493–497.
T. Ayer, J. Chhatwal, O. Alagoz, et al. Comparison of logistic regression and artificial neural network models in breast cancer risk estimation. Adiographics, 2010, 30(1): 13–22.
Y. Gong, Y. Fang, Y. Guo. Private data analytics on biomedical sensing data via distributed computation. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2016, 13(3): 431–444.
D. Singh, D. Roy, C. K. Mohan. DiP-SVM: distribution preserving kernel support vector machine for big data. IEEE Transactions on Big Data, 2016, 3(1): 79–90.
T. G. Dietterich. Ensemble learning. The Handbook of Brain Theory and Neural Networks. 2nd ed. Cambridge: MIT Press, 2002: 110–125.
C. Zhang, Y. Ma (eds.). Ensemble Machine Learning: Methods and Applications. New York: Springer, 2012.
L. Breiman. Bagging predictors. Machine learning, 1996, 24(2): 123–140.
E. Bauer, R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 1999, 36(1/2): 105–139.
L. Breiman. Random forests. Machine Learning, 2001, 45(1): 5–32.
A. Liaw, M. Wiener. Classification and regression by random forest. R News, 2002, 2(3): 18–22.
T. N. Do, F. Poulet. Classifying one billion data with a new distributed SVM algorithm. International Conference on Research, Innovation and Vision for the Future, Vietnam: IEEE, 2006: 59–66.
S. S. Ram, A. Nedic, V. V. Veeravalli. Distributed stochastic subgradient projection algorithms for convex optimization. Journal of Optimization Theory and Applications, 2010, 147(3): 516–545.
D. Caragea, A. Silvescu, V. Honavar. Agents that learn from distributed dynamic data sources. Proceedings of the Workshop on Learning Agents, 2000: 53–61.
F. Pourpanah, C. J. Tan, C. P. Lim, et al. A Q-learning-based multi-agent system for data classification. Applied Soft Computing, 2017, 52: 519–531.
Z. Tu, Y. Wang, Y. Hong. Distributed boosting algorithm over multi-agent networks. Proceedings of the 37th Chinese Control Conference, Wuhan: IEEE, 2018: 7153–7157.
N. Couellan, S. Jan, T. Jorquera, et al. Self-adaptive support vector machine: a multi-agent optimization perspective. Expert Systems with Applications, 2015, 42(9): 4284–4298.
Y. Wang, P. Lin, H. Qin. Distributed classification learning based on nonlinear vector support machines for switching networks. Kybernetika, 2017, 53(4): 595–611.
J. Chen, A. H. Sayed. Diffusion adaptation strategies for distributed optimization and learning over networks. IEEE Transactions on Signal Processing, 2012, 60(8): 4289–4305.
C. G. Lopes, A. H. Sayed. Diffusion least-mean squares over adaptive networks: Formulation and performance analysis. IEEE Transactions on Signal Processing, 2008, 56(7): 3122–3136.
F. S. Cattivelli, A. H. Sayed. Diffusion LMS strategies for distributed estimation. IEEE Transactions on Signal Processing, 2009, 58(3): 1035–1048.
S. Y. Tu, A. H. Sayed. Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks. IEEE Transactions on Signal Processing, 2012, 60(12): 6217–6234.
A. H. Sayed. Adaptation, learning, and optimization over networks. Foundations and Trends in Machine Learning, 2014, 7(4/5): 311–801.
S. Boyd, L. Vandenberghe. Convex Optimization. Cambridge: Cambridge University Press, 2004.
T. P. Minka. A comparison of numerical optimizers for logistic regression: https://www.microsoft.com/en-us/research/publication/comparison-numerical-optimizers-logistic-regression/.
D. Dua, C. Graff. UCI Machine Learning Repository: https://archive.ics.uci.edu/ml/index.php.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported in part by the National Natural Science foundation of China (No. 41927801).
Yan DU received her B.Sc. degree in Communication Engineering from Wuhan University of Technology, China in 2018. From 2018 to 2021, she is pursuing a M.Sc. degree at Beijing Institute of Technology. Her research interests include distributed signal processing, estimation and machine learning.
Lijuan JIA received her Ph.D. degree in Electric and Electronic Engineering from Kyushu University, Japan, in 2002. From 2002 to 2005, she was a lecturer at the Department of Electrical and Electronic Engineering, Kyushu University, Japan. Since 2005, she has been an associate professor of the School of Information and Electronics, Beijing Institute of Technology. From 2013 to 2014, she was a visiting scholar at the Department of Electrical Engineering, University of California, Los Angeles, U.S.A. Her research interests include multi-agent system theory, distributed adaptive networks, and statistical signal processing.
Shunshoku KANAE (M’00) received the Dr. Eng. degree from Kyushu University, Fukuoka, Japan, in 1995. From 1995 to 1998, he was a Research Associate at Kyushu Institute of Technology. Since October 1998, he has been a Research Associate in the Graduate School of Information Science and Electrical Engineering, Kyushu University. He is now a Professor with the Department of Medical Engineering, Faculty of Health Science, Junshin Gakune University, in Fukuoka-city, Japan. His research interests include system identification, mechatronics system control, and soft computing.
Zijiang YANG received his Dr. Eng. degree in 1992 from Kyushu University. From 1996 to 2000, he was an Associate Professor in the Faculty of Computer Engineering and System Science, Kyushu Institute of Technology, Japan. From 2000 to 2009, He was an Associate Professor in the Department of Electrical and Electronic Systems Engineering, Kyushu University, Japan. Since 2009, he has been a Professor of the Department of Mechanical Systems Engineering, Faculty of Engineering, Ibaraki University, Japan. His research interests include system identification and nonlinear system control.
Rights and permissions
About this article
Cite this article
Du, Y., Jia, L., Kanae, S. et al. Diffusion logistic regression algorithms over multiagent networks. Control Theory Technol. 18, 160–167 (2020). https://doi.org/10.1007/s11768-020-0009-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11768-020-0009-2