TY  - EJOUR
AU  - Mayfrank, Daniel
AU  - Mitsos, Alexander
AU  - Dahmen, Manuel
TI  - End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control
PB  - arXiv
M1  - FZJ-2024-00909
PY  - 2023
AB  - (Economic) nonlinear model predictive control ((e)NMPC) requires dynamic system models that are sufficiently accurate in all relevant state-space regions. These models must also be computationally cheap enough to ensure real-time tractability. Data-driven surrogate models for mechanistic models can be used to reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum average prediction accuracy on simulation samples and perform suboptimally as part of actual (e)NMPC. We present a method for end-to-end reinforcement learning of dynamic surrogate models for optimal performance in (e)NMPC applications, resulting in predictive controllers that strike a favorable balance between control performance and computational demand. We validate our method on two applications derived from an established nonlinear continuous stirred-tank reactor model. We compare the controller performance to that of MPCs utilizing models trained by the prevailing maximum prediction accuracy paradigm, and model-free neural network controllers trained using reinforcement learning. We show that our method matches the performance of the model-free neural network controllers while consistently outperforming models derived from system identification. Additionally, we show that the MPC policies can react to changes in the control setting without retraining.
KW  - Machine Learning (cs.LG) (Other)
KW  - Systems and Control (eess.SY) (Other)
KW  - FOS: Computer and information sciences (Other)
KW  - FOS: Electrical engineering, electronic engineering, information engineering (Other)
LB  - PUB:(DE-HGF)25
DO  - DOI:10.48550/ARXIV.2308.01674
UR  - https://juser.fz-juelich.de/record/1021653
ER  -