Synergy between control theory and machine learning for building energy management

Name: Javier Arroyo Bastida

• Prof. dr. ir. Lieve Helsen (promotor)
• Dr. Fred Spiessens (co-promotor) , VITO NV - Energyville

The building sector is the largest energy-demanding sector responsible for over one-third of energy use globally and an equally important portion of carbon dioxide (CO₂) emissions. Nevertheless, the buildings' heating, ventilation, and air conditioning systems are typically operated using very basic, reactive controllers. Optimal controllers can enhance buildings' energy efficiency by taking forecast and uncertainties into account (e.g. weather and occupancy). Optimal controllers can also increase the share of Renewable Energy Sources (RES) by enabling demand response, i.e. flexible operation in response to the needs of the electrical or thermal grid. The former results in energy savings by making better use of energy systems within the buildings. The latter allows a higher share of renewable energy sources within the grid(s) leading to lower CO₂ emissions.

Even though the benefits of advanced optimal controllers have been demonstrated in several research studies and some demonstration cases, the adoption of these techniques in the built environment remains somewhat limited. One of the main reasons is that these novel control algorithms continue to be evaluated individually and the key performance indicators used are case-specific. This hampers the identification of best practices to deploy optimal control widely in the building sector. Additionally, both control and machine learning communities keep evolving independently, and so does their application for building energy management.With such parallel developments, it remains unclear how both communities could join forces.

The main subject of this research is to develop and compare novel algorithms that increase the buildings’ operational energy efficiency and use of RES aiming for the broad use of optimal control in practice. To this end, different software frameworks are developed that address the challenges above. Special attention is paid to (1) enabling evaluation and benchmarking of optimal controllers, (2) understanding and highlighting the similitudes and differences between the optimal control formulations, and (3) studying and comparing the control-oriented modeling paradigms.

The first main contribution of this work relates to the development of the Building Optimization Testing (BOPTEST) framework to provide a unified benchmark for advanced building control.The main novelty of BOPTEST is that it standardizes the key performance indicators and testing scenarios from a menu of high-fidelity emulator building models.This work also develops two software frameworks for implementing Model Predictive Control (MPC) and Reinforcement Learning (RL) in buildings.A synergetic approach is further proposed and implemented that merges the merits of these two control techniques. The new algorithm is called Reinforced Model Predictive Control (RL-MPC) and combines the operational safety of MPCwith the learning capabilities of RL.

The proposed methods are applied to different use cases to investigate and determine the best optimal building control practices. In a first use case, a dynamic coalition manager is presented that was developed in the frame of the Flexible Heat and Power project. This distributed optimization setup uses optimal control to harvest flexibility from a clusterof buildings to support the electricity grid. It is shown how this solution steers the flexibility of one hundred simulated buildings without compromising their privacy nor violating their indoor comfort.

Then, the focus is shifted towards the individual building level. Different variations of optimal control are investigated, e.g., by studying the impact of physics-based vs. data-driven modeling paradigms, the influence of different complexity levels of building modeling for control, or the direct comparison of MPC, RLand RL-MPC. An experimental study indicates that there is not necessarily a relationship between prediction and control performance, and that not modeling theheating system based on physical principles can seriously jeopardizethe control performance of data-driven models. On the other hand, the complexity overhead of physics-based models of the building envelope and their high exposure to uncertainty during operation suggest that this part of the building model may benefit from using operational data for training and making some simplifications.

Results from the BOPTEST framework show that contrary to what is stated in previous literature, model-free RL approaches poorly perform when tested in building environments with realistic system dynamics. Even when a model is available and simulation-based RL can be implemented, MPCmakes more effective use of the controller model for an equivalent formulation of the optimal control problem.The performance gap between both controllers reduces when using the RL-MPC algorithm that can handle constraints as in MPC while enabling adaptability as in RL.

• Prof. dr. ir. Lieve Helsen (promotor)
• Dr. Fred Spiessens (co-promotor) , VITO NV - Energyville
• Prof. dr. ir. Omer Van der Biest (voorzitter/chairman)
• Prof. dr. ir. Dirk Saelens (secretaris/secretary)
• Prof. dr. ir. Geert Deconinck
• Dr. David Blum , University of California
• Dr. Gowri Suryanarayana , VITO
• Prof. dr. Christian Veje , University of Southern Denmark