Abstract: Hydropower serves as a critical renewable energy source that is often used for essential peak-shaving and frequency regulation for power grids; the agility of hydropower units’ speed-governing system in response to load fluctuations directly impacts power quality and grid stability. However, the generating units usually operate across diverse conditions and suffer from severe nonlinearities, posing a huge challenge to the conventional method of proportional-integral-derivative (PID) control. To enhance system control performance and robustness, this paper describes a new intelligent control strategy that is based on the Soft Actor-Critic (SAC) reinforcement learning algorithm. By using the strategy and a nonlinear pump-turbine governing system, first a framework is constructed to train a network of Nonlinear Autoregressive with Exogenous Input Long Short-Term Memory (NARX-LSTM) as a surrogate model. It uses one module for LSTM-based error-correction to raise model accuracy. Then, this error-corrected NARX-LSTM environment is leveraged for iterative training of the SAC agent. Simulation results demonstrate that the new method outperforms traditional PID control in response speed and overshoot suppression across multiple operating points. And, the strategy exhibits superior resilience to operational transitions with minimal fluctuations. This study has verified the efficacy of reinforcement learning in achieving a complicated industrial control, and a promising new paradigm for hydropower speed regulation.

Key words: pump turbine, turbine speed control, reinforcement learning, soft actor-critic algorithm, nonlinear autoregressive with exogenous input long short-term memory network