Abstract
In this paper, optimal switching and control approaches are investigated for switched systems with infinite-horizon cost functions and unknown continuous-time subsystems. At first, for switched systems with autonomous subsystems, the optimal solution based on the finite-horizon HJB equation is proposed and a data-driven optimal switching algorithm is designed. Then, for the switched systems with subsystem inputs, a data-driven optimal control approach based on the finite-horizon HJB equation is proposed. The data-driven approaches approximate the optimal solutions online by means of the system state data instead of the subsystem models. Moreover, the convergence of the two approaches is analyzed. Finally, the validity of the two approaches is demonstrated by simulation examples.
Similar content being viewed by others
References
Liberzon, D. (2003). Switching in systems and control. Basel: Birkhäuser.
Sun, Z. (2006). Switched linear systems: Control and design. London: Springer.
Sun, Z. (2017). Recent advances on analysis and design of switched linear systems. Control Theory Technology, 15(3), 242–244.
Xu, X., & Antsaklis, P.-J. (2004). Optimal control of switched systems based on parameterization of the switching instants. IEEE Transactions on Automatic Control, 49(1), 2–16.
Bengea, S. C., & Decarlo, R.-A. (2005). Optimal control of switching systems. Automatica, 41(1), 11–27.
Zhu, F., & Antsaklis, P.-J. (2015). Optimal control of hybrid switched systems: A brief survey. Discrete Event Dynamic Systems, 25(3), 345–364.
Yathisha, L., & Patilkulkarni, S. (2018). LQR and LQG based optimal switching techniques for PSS and UPFC in power systems. Control Theory and Technology, 16(1), 25–37.
Vrabie, D., Pastravanu, O., Abu-Khalaf, M., & Lewis, F. L. (2009). Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica, 45(2), 477–484.
Jiang, Y., & Jiang, Z.-P. (2012). Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica, 48(10), 2699–2704.
Bian, T., Jiang, Y., & Jiang, Z.-P. (2014). Adaptive dynamic programming and optimal control of nonlinear nonaffine systems. Automatica, 50(10), 2624–2632.
Bian, T., & Jiang, Z.-P. (2016). Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design. Automatica, 71, 348–360.
Bian, T., & Jiang, Z.-P. (2016). Value iteration, adaptive dynamic programming, and optimal control of nonlinear systems. In IEEE 55th Conference on Decision and Control (pp. 3375–3380). Las Vegas, NV, USA.
Heydari, A., & Balakrishnan, S. (2014). Optimal switching between autonomous subsystems. Journal of the Franklin Institute, 351(5), 2675–2690.
Heydari, A. (2015). Optimal scheduling for reference tracking or state regulation using reinforcement learning. Journal of the Franklin Institute, 352(8), 3285–3303.
Heydari, A. (2015). Feedback solution to optimal switching problems with switching cost. IEEE Transactions on Neural Networks and Learning Systems, 27(10), 2009–2019.
Heydari, A. (2017). Optimal switching with minimum dwell time constraint. Journal of the Franklin Institute, 354(11), 4498–4518.
Sardarmehni, T., & Heydari, A. (2017). Sub-optimal switching in antilock brake systems using approximate dynamic programming. IET Control Theory & Applications, 13(9), 1413–1424.
Sardarmehni, T., & Heydari, A. (2016). Policy iteration for optimal switching with continuous-time dynamics. In International Joint Conference on Neural Networks (pp. 3536–3543). Vancouver, Canada.
Sardarmehni, T., & Heydari, A. (2018). Sub-optimal scheduling in switched systems with continuous-time dynamics: A gradient descent approach. Neurocomputing, 285, 10–22.
Sardarmehni, T., & Heydari, A. (2018). Suboptimal scheduling in switched systems with continuous-time dynamics: A least squares approach. IEEE Transactions on Neural Networks and Learning Systems, 29(6), 2167–2178.
Heydari, A., & Balakrishnan, S.-N. (2014). Optimal switching and control of nonlinear switching systems using approximate dynamic programming. IEEE Transactions on Neural Networks and Learning Systems, 25(6), 1106–1117.
Qin, C., Zhang, H., Luo, Y., & Wang, B. (2014). Finite horizon optimal control of non-linear discrete-time switched systems using adaptive dynamic programming with epsilon-error bound. International Journal of Systems Science, 45(8), 1683–1693.
Heydari, A., & Balakrishnan, S.-N. (2015). Optimal switching between controlled subsystems with free mode sequence. Neurocomputing, 149, 1620–1630.
Pang, B., & Jiang, Z.-P. (2021). Adaptive optimal control of linear periodic systems: An off-policy value iteration approach. IEEE Transactions on Automatic Control, 66(2), 888–894.
Heydari, A. (2020). Optimal impulsive control using adaptive dynamic programming and its application in spacecraft rendezvous. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2020.3021037.
Liu, D., Yu, W., Baldi, S., Cao, J., & Huang, W. (2020). A switching-based adaptive dynamic programming method to optimal traffic signaling. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(11), 4160–4170.
Pang, B., Bian, T., & Jiang, Z.-P. (2019). Adaptive dynamic programming for finite-horizon optimal control of linear time-varying discrete-time systems. Control Theory and Technology, 17(1), 73–84.
Lu, W., & Ferrari, S. (2013). An approximate dynamic programming approach for model-free control of switched systems. In IEEE 52nd Annual Conference Conference on Decision and Control (pp. 3837–3844). Florence, Italy.
Lu, W., Zhu, P., & Ferrari, S. (2016). A hybrid-adaptive dynamic programming approach for the model-free control of nonlinear switched systems. IEEE Transactions on Automatic Control, 61(10), 3203–3208.
Zhang, C., Gan, M., & Zhao, J. (2019). Data-driven optimal control of switched linear autonomous systems. International Journal of Systems Science, 50(6), 1275–1289.
Gan, M., Zhang, C., & Zhao, J. (2019). Data-driven optimal switching of switched systems. Journal of the Franklin Institute, 356(10), 5193–5221.
Sontag, E.-D. (1990). Mathematical control theory: Deterministic finite dimensional systems. New York: Springer.
Powell, M.-J.-D. (1981). Approximation theory and methods. Cambridge: Cambridge University Press.
Funding
This work was supported by the National Natural Science Foundation of China (no. 61673065).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, C., Gan, M. & Xue, C. Data-driven optimal switching and control of switched systems. Control Theory Technol. 19, 299–314 (2021). https://doi.org/10.1007/s11768-021-00054-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11768-021-00054-y