Skip to main content
SpringerLink
Log in

Adaptive output regulation for cyber-physical systems under time-delay attacks

  • Research Article
  • Published:
Control Theory and Technology Aims and scope Submit manuscript

Abstract

In this paper, we present an output regulation method for unknown cyber-physical systems (CPSs) under time-delay attacks in both the sensor-to-controller (S-C) channel and the controller-to-actuator (C-A) channel. The proposed approach is designed using control inputs and tracking errors which are accessible data. Reinforcement learning is leveraged to update the control gains in real time using policy or value iterations. A thorough stability analysis is conducted and it is found that the proposed controller can sustain the convergence and asymptotic stability even when two channels are attacked. Finally, comparison results with a simulated CPS verify the effectiveness of the proposed output regulation method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Mahmoud, M. S., Hamdan, M. M., & Baroudi, U. A. (2019). Modeling and control of cyber-physical systems subject to cyber attacks: A survey of recent advances and challenges. Neurocomputing, 338, 101–115.

    Article  Google Scholar 

  2. Jazdi, N. (2014). Cyber physical systems in the context of Industry 4.0. In IEEE International Conference on Automation, Quality and Testing, Robotics (pp. 1–4). Cluj-Napoca, Romania.

  3. Cintuglu, M. H., Mohammed, O. A., Akkaya, K., & Uluagac, A. S. (2016). A survey on smart grid cyber-physical system testbeds. IEEE Communications Surveys& Tutorials, 19(1), 446–464.

    Article  Google Scholar 

  4. Xiong, G., Zhu, F., Liu, X., Dong, X., Huang, W., Chen, S., & Zhao, K. (2015). Cyber-physical-social system in intelligent transportation. IEEE/CAA Journal of Automatica Sinica, 2(3), 320–333.

    Article  MathSciNet  Google Scholar 

  5. Zhang, Y., Qiu, M., Tsai, C. W., Hassan, M. M., & Alamri, A. (2015). Health-CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Systems Journal, 11(1), 88–95.

    Article  Google Scholar 

  6. Wu, G., Sun, J., & Chen, J. (2016). A survey on the security of cyber-physical systems. Control Theory and Technology, 14(1), 2–10.

    Article  MathSciNet  Google Scholar 

  7. NSFOCUS (2020). An Observation on Cyber Security Incidents. https://nsfocusglobal.com/2020-an-observation-on-cyber-security-incidents.

  8. Langner, R. (2011). Stuxnet: Dissecting a cyberwarfare weapon. IEEE Security& Privacy, 9(3), 49–51.

    Article  Google Scholar 

  9. Sargolzaei, A., Yen, K. K., & Abdelghani, M. N. (2016). Preventing time-delay switch attack on load frequency control in distributed power systems. IEEE Transactions on Smart Grid, 7(2), 1176–1185.

    Google Scholar 

  10. Rahimi, K., Parchure, A., Centeno, V., & Broadwater, R. (2015). Effect of communication time-delay attacks on the performance of automatic generation control. In 2015 North American Power Symposium (NAPS) (pp. 1–6). Charlotte, NC, USA.

  11. Lou, X., Tran, C., Tan, R., Yau, D. K., Kalbarczyk, Z. T., Banerjee, A. K., & Ganesh, P. (2020). Assessing and mitigating impact of time delay attack: case studies for power grid controls. IEEE Journal on Selected Areas in Communications, 38(1), 141–155.

    Article  Google Scholar 

  12. Bianchin, G., & Pasqualetti, F. (2018). Time-delay attacks in network systems. In Cyber-Physical Systems Security (pp. 157–174). Springer

  13. Chen, B., Ho, D. W., Zhang, W. A., & Yu, L. (2019). Distributed dimensionality reduction fusion estimation for cyber-physical systems under DoS attacks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(2), 455–468.

    Article  Google Scholar 

  14. Lai, S., Chen, B., Li, T., & Yu, L. (2018). Packet-based state feedback control under DoS attacks in cyber-physical systems. IEEE Transactions on Circuits and Systems II: Express Briefs, 66(8), 1421–1425.

    Article  Google Scholar 

  15. Zhang, H., & Zheng, W. X. (2018). Denial-of-service power dispatch against linear quadratic control via a fading channel. IEEE Transactions on Automatic Control, 63(9), 3032–3039.

    Article  MathSciNet  Google Scholar 

  16. Gao, L., Chen, B., & Yu, L. (2020). Fusion-based FDI attack detection in cyber-physical systems. IEEE Transactions on Circuits and Systems II: Express Briefs, 67(8), 1487–1491.

    Article  Google Scholar 

  17. Li, Q., Xu, B., Li, S., Liu, Y., & Cui, D. (2018). Reconstruction of measurements in state estimation strategy against deception attacks for cyber physical systems. Control Theory and Technology, 16(1), 1–13.

    Article  MathSciNet  Google Scholar 

  18. Chen, B., Ho, D. W., Hu, G., & Yu, L. (2018). Secure fusion estimation for bandwidth constrained cyber-physical systems under replay attacks. IEEE Transactions on Cybernetics, 48(6), 1862–1876.

    Article  Google Scholar 

  19. Nilsson, J., Bernhardsson, B., & Wittenmark, B. (1998). Stochastic analysis and control of real-time systems with random time delays. Automatica, 34(1), 57–64.

    Article  MathSciNet  Google Scholar 

  20. Qi, Q., & Zhang, H. (2017). Output feedback control and stabilization for networked control systems with packet losses. IEEE Transactions on Cybernetics, 47(8), 2223–2234.

    Article  Google Scholar 

  21. Zhang, W. A., & Yu, L. (2010). A robust control approach to stabilization of networked control systems with short time-varying delays. Acta Automatica Sinica, 36(1), 87–91.

    MathSciNet  MATH  Google Scholar 

  22. Zhang, H., Zhang, Z., Wang, Z., & Shan, Q. (2015). New results on stability and stabilization of networked control systems with short time-varying delay. IEEE Transactions on Cybernetics, 46(12), 2772–2781.

    Article  Google Scholar 

  23. Zhang, W. A., Yu, L., & Yin, S. (2011). A switched system approach to \({\rm H}_{\infty }\) control of networked control systems with time-varying delays. Journal of the Franklin Institute, 348(2), 165–178.

    Article  MathSciNet  Google Scholar 

  24. Jin, D., Yu, L., & Zhang, W. A. (2017). Stabilization of networked interconnected systems. Journal of Advanced Computational Intelligence and Intelligent Informatics, 21(2), 251–257.

    Article  Google Scholar 

  25. Gao, W., & Jiang, Z. P. (2019). Adaptive optimal output regulation of time-delay systems via measurement feedback. IEEE Transactions on Neural Networks and Learning Systems, 30(3), 938–945.

    Article  MathSciNet  Google Scholar 

  26. Liu, Y., Zhang, H., Yu, R., & Qu, Q. (2018). Data-driven optimal tracking control for discrete-time systems with delays using adaptive dynamic programming. Journal of the Franklin Institute, 355(13), 5649–5666.

    Article  MathSciNet  Google Scholar 

  27. Zhang, H., Liu, Y., Xiao, G., & Jiang, H. (2020). Data-based adaptive dynamic programming for a class of discrete-time systems with multiple delays. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(2), 432–441.

    Article  Google Scholar 

  28. Fan, J., Wu, Q., Jiang, Y., Chai, T., & Lewis, F. L. (2020). Model-free optimal output regulation for linear discrete-time lossy networked control systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(11), 4033–4042.

    Article  Google Scholar 

  29. Pang, Z. H., Liu, G. P., Zhou, D., & Sun, D. (2016). Data-based predictive control for networked nonlinear systems with network-induced delay and packet dropout. IEEE Transactions on Industrial Electronics, 63(2), 1249–1257.

    Article  Google Scholar 

  30. Abouheaf, M. I., Lewis, F. L., Mahmoud, M. S., & Mikulski, D. G. (2015). Discrete-time dynamic graphical games: model-free reinforcement learning solution. Control Theory and Technology, 13(1), 55–69.

    Article  MathSciNet  Google Scholar 

  31. Zhao, X., Yi, P., & Li, L. (2020). Distributed policy evaluation via inexact ADMM in multi-agent reinforcement learning. Control Theory and Technology, 18(4), 362–378.

    Article  MathSciNet  Google Scholar 

  32. Huang, X., & Dong, J. (2020). ADP-based robust resilient control of partially unknown nonlinear systems via cooperative interaction design. IEEE Transactions on Systems, Man, and Cybernetics: Systems. https://doi.org/10.1109/TSMC.2020.2970040

    Article  Google Scholar 

  33. Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. Cambridge: MIT Press.

    MATH  Google Scholar 

  34. Lewis, F. L., & Vrabie, D. (2009). Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits and Systems Magazine, 9(3), 32–50.

    Article  Google Scholar 

  35. Jiang, H., Zhang, H., & Xie, X. (2020). Critic-only adaptive dynamic programming algorithms applications to the secure control of cyber—physical systems. ISA Transactions, 104, 138–144.

    Article  Google Scholar 

  36. Liu, R., Hao, F., & Yu, H. (2020). Optimal SINR-based DoS attack scheduling for remote state estimation via adaptive dynamic programming approach. IEEE Transactions on Systems, Man, and Cybernetics: Systems. https://doi.org/10.1109/TSMC.2020.2981478

    Article  Google Scholar 

  37. Huang, X., & Dong, J. (2021). Robust dynamic actuator failure compensation control of nonlinear systems via cooperative interaction design. IEEE Transactions on Cybernetics. https://doi.org/10.1109/TCYB.2021.3049508

    Article  Google Scholar 

  38. Bradbury, M., Leeke, M., & Jhumka, A. (2015). A dynamic fake source algorithm for source location privacy in wireless sensor networks. In IEEE Trustcom/BigDataSE/ISPA (pp. 531–538), Helsinki, Finland.

  39. Huang, J. (2004). Nonlinear Output Regulation: Theory and Applications. Philadelphia: SIAM.

    Book  Google Scholar 

  40. Lewis, F. L., Vrabie, D., & Syrmos, V. L. (2012). Optimal Control. New York: John Wiley & Sons Inc.

    Book  Google Scholar 

  41. Hewer, G. (1971). An iterative technique for the computation of the steady state gains for the discrete optimal regulator. IEEE Transactions on Automatic Control, 16(4), 382–384.

    Article  Google Scholar 

  42. Lancaster, P., & Rodman, L. (1995). Algebraic Riccati Equations. Oxford: Clarendon Press.

    MATH  Google Scholar 

  43. Wang, D., & Mu, C. (2018). Adaptive-critic-based robust trajectory tracking of uncertain dynamics and its application to a springCmassCdamper system. IEEE Transactions on Industrial Electronics, 65(1), 654–663.

    Article  MathSciNet  Google Scholar 

  44. Modares, H., & Lewis, F. L. (2014). Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica, 50(7), 1780–1792.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 61973277, 62073292) and in part by the Zhejiang Provincial Natural Science Foundation of China (No. LR20F030004).

Author information

Authors and Affiliations

  1. College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, China

    Dan Jin, Bo Chen & Li Yu

  2. Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, China

    Dan Jin, Bo Chen & Li Yu

  3. Department of Electronics, Carleton University, Ottawa, ON, K1S5B6, Canada

    Shichao Liu

Authors
  1. Dan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shichao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, D., Chen, B., Yu, L. et al. Adaptive output regulation for cyber-physical systems under time-delay attacks. Control Theory Technol. 20, 20–31 (2022). https://doi.org/10.1007/s11768-021-00072-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11768-021-00072-w

Keywords

Search

Navigation