Scalable stochastic optimization of complex energy systems

We present a scalable approach and implementation for solving stochastic programming problems, with application to the optimization of complex energy systems under uncertainty. Stochastic programming is used to make decisions in the present while incorporating a model of uncertainty about future events (scenarios). These problems present serious computational difficulties as the number of scenarios becomes large and the complexity of the system and planning horizons increase, necessitating the use of parallel computing. Our novel hybrid parallel implementation PIPS is based on interior-point methods and uses a Schur complement technique to obtain a scenario-based decomposition of the linear algebra. PIPS is applied to a stochastic economic dispatch problem that uses hourly wind forecasts and a detailed physical power flow model. Solving this problem is necessary for efficient integration of wind power with the Illinois power grid and real-time energy market. Strong scaling efficiency of 96% is obtained on 32 racks (131,072 cores) of the "Intrepid" Blue Gene/P system at Argonne National Laboratory.