Abstract
In this paper, the neural network-based adaptive decentralized learning control is investigated for nonlinear interconnected systems with input constraints. Because the decentralized control of interconnected systems is related to the optimal control of each isolated subsystem, the decentralized control strategy can be established by a series of optimal control policies. A novel policy iteration algorithm is presented to solve the Hamilton–Jacobi–Bellman equation related to the optimal control problem. This algorithm is implemented under the actor-critic structure where both neural networks are simultaneously updated to approximate the optimal control policy and the optimal cost function, respectively. The additional stabilizing term is introduced and an improved weight updating law is derived, which relaxes the requirement of initial admissible control policy. Besides, the input constraints of interconnected systems are taken into account and the Hamilton–Jacobi–Bellman equation is solved in the presence of input constraints. The interconnected system states and the weight approximation errors of two neural networks are proven to be uniformly ultimately bounded by utilizing Lyapunov theory. Finally, the effectiveness of the proposed decentralized learning control method is verified by simulation results.
Similar content being viewed by others
References
Bakule, L. (2008). Decentralized control: An overview. Annual Reviews in Control, 32(1), 87–98.
Bian, T., Jiang, Y., & Jiang, Z. (2015). Decentralized adaptive optimal control of large-scale systems with application to power systems. IEEE Transactions on Industrial Electronics, 62(4), 2439–2447.
Wang, H. (2013). Task-space synchronization of networked robotic systems with uncertain kinematics and dynamics. IEEE Transactions on Automatic Control, 58(12), 3169–3174.
Tang, Y., Tomizuka, M., Guerrero, G., & Montemayor, G. (2000). Decentralized robust control of mechanical systems. IEEE Transactions on Automatic Control, 45(4), 771–776.
Saberi, A. (1988). On optimality of decentralized control for a class of nonlinear interconnected systems. Automatica, 24(1), 101–104.
Mu, C., Sun, C., Wang, D., Song, A., & Qian, C. (2018). Decentralized adaptive optimal stabilization of nonlinear systems with matched interconnections. Soft Computing, 22(8), 2705–2715.
Li, S. (2017). Towards to dynamic optimal control for large-scale distributed systems. Control Theory and Technology, 15(2), 158–160.
Yang, X., & He, H. (2019). Adaptive critic learning and experience replay for decentralized event-triggered control of nonlinear interconnected systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(11), 4043–4055.
Mu, C., Wang, K., & Sun, C. (2021). Learning control supported by dynamic event communication applying to industrial systems. IEEE Transactions on Industrial Informatics, 17(4), 2325–2335.
Kang, W., & Wilcox, L. C. (2017). Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. Computational Optimization and Applications, 68(2), 289–315.
Mu, C., & Zhang, Y. (2020). Learning-based robust tracking control of quadrotor with time-varying and coupling uncertainties. IEEE Transactions on Neural Networks and Learning Systems, 31(1), 259–273.
Fairbank, M., Alonso, E., & Prokhorov, D. (2013). An equivalence between adaptive dynamic programming with a critic and backpropagation through time. IEEE Transactions on Neural Networks and Learning Systems, 24(12), 2088–2100.
Pang, B., Bian, T., & Jiang, Z. (2019). Adaptive dynamic programming for finite-horizon optimal control of linear time-varying discrete-time systems. Control Theory and Technology, 17(1), 73–84.
Guo, C., Xie, X., & Hou, Z. (2020). Removing feasibility conditions on adaptive neural tracking control of nonlinear time-delay systems with time-varying powers, input, and full-state constraints. IEEE Transactions on Cybernetics. https://doi.org/10.1109/TCYB.2020.3003327.
Xu, H., & Jagannathan, S. (2013). Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming. IEEE Transactions on Neural Networks and Learning Systems, 24(3), 471–484.
Liu, D., Wei, Q., & Yan, P. (2015). Generalized policy iteration adaptive dynamic programming for discrete-time nonlinear systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45(12), 1577–1591.
Vamvoudakis, K. G., & Lewis, F. L. (2010). Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica, 46(5), 878–888.
Fan, Q., & Yang, G. (2016). Adaptive actor-critic design-based integral sliding-mode control for partially unknown nonlinear systems with input disturbances. IEEE Transactions on Neural Networks and Learning Systems, 27(1), 165–177.
Wu, H., & Luo, B. (2012). Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear \({\rm H}_{\infty }\) control. IEEE Transactions on Neural Networks and Learning Systems, 23(12), 1884–1895.
Modares, H., Lewis, F. L., & Naghibi-Sistani, M. (2013). Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Transactions on Neural Networks and Learning Systems, 24(10), 1513–1525.
Wang, T., Wang, B., & Liang, Y. (2020). Multi-agent graphical games with input constraints: an online learning solution. Control Theory and Technology, 18(2), 148–159.
Chen, Y., Li, Z., Kong, H., & Ke, F. (2019). Model predictive tracking control of nonholonomic mobile robots with coupled input constraints and unknown dynamics. IEEE Transactions on Industrial Informatics, 15(6), 3196–3205.
Zhao, Z., Ahn, K., & Li, H. (2019). Boundary anti-disturbance control of a spatially nonlinear flexible string system. IEEE Transactions on Industrial Electronics, 67(6), 4846–4856.
Gao, S., Dong, H., Ning, B., & Yao, X. (2017). Single-parameter-learning-based fuzzy fault-tolerant output feedback dynamic surface control of constrained-input nonlinear systems. Information Sciences, 385, 378–394.
Luo, B., Liu, D., & Wu, H. (2018). Adaptive constrained optimal control design for data-based nonlinear discrete-time systems with critic-only structure. IEEE Transactions on Neural Networks and Learning Systems, 29(6), 2099–2111.
Khalaf, M. A., & Lewis, F. L. (2005). Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica, 41(5), 779–791.
Nodland, D., Zargarzadeh, H., & Jagannathan, S. (2013). Neural network-based optimal adaptive output feedback control of a helicopter UAV. IEEE Transactions on Neural Networks and Learning Systems, 24(7), 1061–1073.
Zhang, H., Cui, L., & Luo, Y. (2013). Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Transactions on Cybernetics, 43(1), 206–216.
Acknowledgements
This work was supported by the National Key R&D Program of China (No. 2018AAA0101400) and the National Natural Science Foundation of China (Nos. 62022061, 61921004).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mu, C., Luo, H., Wang, K. et al. Neural network-based adaptive decentralized learning control for interconnected systems with input constraints. Control Theory Technol. 19, 392–404 (2021). https://doi.org/10.1007/s11768-021-00056-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11768-021-00056-w