Preprint

FZJ-2024-00909

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control

Mayfrank, D.FZJ*RWTH* ; Mitsos, A.FZJ*RWTH* ; Dahmen, M. (Corresponding author)FZJ*

2023
arXiv

This record in other databases:

Please use a persistent id in citations: doi:10.48550/ARXIV.2308.01674  doi:10.34734/FZJ-2024-00909

Abstract: (Economic) nonlinear model predictive control ((e)NMPC) requires dynamic system models that are sufficiently accurate in all relevant state-space regions. These models must also be computationally cheap enough to ensure real-time tractability. Data-driven surrogate models for mechanistic models can be used to reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum average prediction accuracy on simulation samples and perform suboptimally as part of actual (e)NMPC. We present a method for end-to-end reinforcement learning of dynamic surrogate models for optimal performance in (e)NMPC applications, resulting in predictive controllers that strike a favorable balance between control performance and computational demand. We validate our method on two applications derived from an established nonlinear continuous stirred-tank reactor model. We compare the controller performance to that of MPCs utilizing models trained by the prevailing maximum prediction accuracy paradigm, and model-free neural network controllers trained using reinforcement learning. We show that our method matches the performance of the model-free neural network controllers while consistently outperforming models derived from system identification. Additionally, we show that the MPC policies can react to changes in the control setting without retraining.

Keyword(s): Machine Learning (cs.LG) ; Systems and Control (eess.SY) ; FOS: Computer and information sciences ; FOS: Electrical engineering, electronic engineering, information engineering

---

Contributing Institute(s):

1. Modellierung von Energiesystemen (IEK-10)

Research Program(s):

1. 1121 - Digitalization and Systems Technology for Flexibility Solutions (POF4-112) (POF4-112)
2. HDS LEE - Helmholtz School for Data Science in Life, Earth and Energy (HDS LEE) (HDS-LEE-20190612) (HDS-LEE-20190612)

Appears in the scientific report 2023

---

Database coverage:

---