2000 character limit reached
Blackout Mitigation via Physics-guided RL
Published 17 Jan 2024 in eess.SY, cs.AI, and cs.SY | (2401.09640v2)
Abstract: This paper considers the sequential design of remedial control actions in response to system anomalies for the ultimate objective of preventing blackouts. A physics-guided reinforcement learning (RL) framework is designed to identify effective sequences of real-time remedial look-ahead decisions accounting for the long-term impact on the system's stability. The paper considers a space of control actions that involve both discrete-valued transmission line-switching decisions (line reconnections and removals) and continuous-valued generator adjustments. To identify an effective blackout mitigation policy, a physics-guided approach is designed that uses power-flow sensitivity factors associated with the power transmission network to guide the RL exploration during agent training. Comprehensive empirical evaluations using the open-source Grid2Op platform demonstrate the notable advantages of incorporating physical signals into RL decisions, establishing the gains of the proposed physics-guided approach compared to its black box counterparts. One important observation is that strategically~\emph{removing} transmission lines, in conjunction with multiple real-time generator adjustments, often renders effective long-term decisions that are likely to prevent or delay blackouts.
- “August 14, 200314200314,\;200314 , 2003 blackout: NERC actions to prevent and mitigate the impacts of future cascading blackouts,” February 2014, https://www.nerc.com/docs/docs/blackout/NERC_Final_Blackout_Report_07_13_04.pdf.
- A. N. Venkat, I. A. Hiskens, J. B. Rawlings, and S. J. Wright, “Distributed MPC strategies with application to power system automatic generation control,” IEEE Transactions on Control Systems Technology, vol. 16, no. 6, pp. 1192–1206, 2008.
- M. R. Almassalkhi and I. A. Hiskens, “Model-predictive cascade mitigation in electric power systems with storage and renewables—Part I: Theory and implementation,” IEEE Transactions on Power Systems, vol. 30, no. 1, pp. 67–77, 2014.
- ——, “Model-predictive cascade mitigation in electric power systems with storage and renewables—Part II: Case-Study,” IEEE Transactions on Power Systems, vol. 30, no. 1, pp. 78–87, 2014.
- Q. Huang, R. Huang, W. Hao, J. Tan, R. Fan, and Z. Huang, “Adaptive power system emergency control using deep reinforcement learning,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1171–1182, 2020.
- E. B. Fisher, R. P. O’Neill, and M. C. Ferris, “Optimal transmission switching,” IEEE Transactions on Power Systems, vol. 23, no. 3, pp. 1346–1355, 2008.
- A. Khodaei and M. Shahidehpour, “Transmission switching in security-constrained unit commitment,” IEEE Transactions on Power Systems, vol. 25, no. 4, pp. 1937–1945, 2010.
- J. D. Fuller, R. Ramasra, and A. Cha, “Fast heuristics for transmission-line switching,” IEEE Transactions on Power Systems, vol. 27, no. 3, pp. 1377–1386, 2012.
- P. Dehghanian, Y. Wang, G. Gurrala, E. Moreno-Centeno, and M. Kezunovic, “Flexible implementation of power system corrective topology control,” Electric Power Systems Research, vol. 128, pp. 79–89, 2015.
- B. Donnot, “Grid2Op - A Testbed Platform to Model Sequential Decision Making in Power Systems,” 2020. [Online]. Available: https://github.com/rte-france/grid2op
- M. Larsson, D. J. Hill, and G. Olsson, “Emergency voltage control using search and predictive control,” International Journal of Electrical Power & Energy Systems, vol. 24, no. 2, pp. 121–130, 2002.
- M. Zima, P. Korba, and G. Andersson, “Power systems voltage emergency control approach using trajectory sensitivities,” in Proc. IEEE Conference on Control Applications, Istanbul, Turkey, June 2003.
- I. Hiskens and B. Gong, “MPC-based load shedding for voltage stability enhancement,” in Proc. IEEE Conference on Decision and Control, Seville, Spain, December 2005.
- J. S. A. Carneiro and L. Ferrarini, “Preventing thermal overloads in transmission circuits via model predictive control,” IEEE Transactions on Control Systems Technology, vol. 18, no. 6, pp. 1406–1412, 2010.
- A. Kelly, A. O’Sullivan, P. de Mars, and A. Marot, “Reinforcement learning for electricity network operation,” arXiv:2003.07339, 2020.
- A. Marot, B. Donnot, C. Romero, B. Donon, M. Lerousseau, L. Veyrin-Forrer, and I. Guyon, “Learning to run a power network challenge for training topology controllers,” Electric Power Systems Research, vol. 189, p. 106635, 2020.
- A. Marot, B. Donnot, G. Dulac-Arnold, A. Kelly, A. O’Sullivan, J. Viebahn, M. Awad, I. Guyon, P. Panciatici, and C. Romero, “Learning to run a power network challenge: A retrospective analysis,” in Proc. NeurIPS Competition and Demonstration Track, December 2021.
- T. Lan, J. Duan, B. Zhang, D. Shi, Z. Wang, R. Diao, and X. Zhang, “AI-based autonomous line flow control via topology adjustment for maximizing time-series ATCs,” in Proc. IEEE Power and Energy Society General Meeting, QC, Canada, August 2020.
- Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in Proc. International Conference on Machine Learning, New York, NY, June 2016.
- D. Yoon, S. Hong, B.-J. Lee, and K.-E. Kim, “Winning the L2RPN challenge: Power grid management via semi-Markov afterstate actor-critic,” in Proc. International Conference on Learning Representations, May 2021.
- R. S. Sutton, D. Precup, and S. Singh, “Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning,” Artificial intelligence, vol. 112, no. 1-2, pp. 181–211, 1999.
- A. Chauhan, M. Baranwal, and A. Basumatary, “PowRL: A reinforcement learning framework for robust management of power networks,” in Proc. AAAI Conference on Artificial Intelligence, Washington, DC, June 2023.
- M. Dorfer, A. R. Fuxjäger, K. Kozak, P. M. Blies, and M. Wasserer, “Power grid congestion management via topology optimization with AlphaZero,” arXiv:2211.05612, 2022.
- D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, M. Lanctot et al., “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, 2016.
- A. Dwivedi and A. Tajer, “GRNN-based real-time fault chain prediction,” IEEE Transactions on Power Systems, vol. 39, no. 1, pp. 934–946, 2024.
- S. Paternain, L. Chamon, M. Calvo-Fullana, and A. Ribeiro, “Constrained reinforcement learning has zero duality gap,” in Proc. Advances in Neural Information Processing Systems, Vancouver, Canada, December 2019.
- V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
- J. Tsitsiklis and B. Van Roy, “An analysis of temporal-difference learning with function approximation,” IEEE Transactions on Automatic Control, vol. 42, no. 5, pp. 674–690, 1997.
- P. Sauer, K. Reinhard, and T. Overbye, “Extended factors for linear contingency analysis,” in Proc. Hawaii International Conference on System Sciences, Maui, Hawaii, January 2001.
- T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” in Proc. International Conference on Learning Representations, San Juan, Puerto Rico, May 2016.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. International Conference on Learning Representations, San Diego, CA, May 2015.
- M. Abadi et al., “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems,” 2015. [Online]. Available: https://www.tensorflow.org/
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
