Monday, August 31, 2015

Probabilistic Electric Load Forecasting: A Tutorial Review

Since 2012, Shu Fan and I have been trying to put together a review paper on load forecasting. We first started with a review on point load forecasting. Then both of us felt that our community may need a review paper on probabilistic load forecasting more than anything else, so we shifted the direction. The first submitted version was released at the beginning of GEFCom2014 for the contestants to have some basic idea about probabilistic load forecasting (see 10 Recommended Papers for GEFCom2014 Contestants). After going through two revisions, the paper went from 32 pages to 53 pages at its third submission. Many thanks to the valuable comments from Rob Hyndman, Pierre Pinson, Rafal Weron, and two anonymous reviewers, the quality of this review has gone up significantly. Today I'm very pleased to announce that this review paper on probabilistic load forecasting was just accepted by International Journal of Forecasting. We hope this paper is useful to the researchers and practitioners who are in the load forecasting community or interested in load forecasting.

Tao Hong and Shu Fan, "Probabilistic electric load forecasting: a tutorial review", International Journal of Forecasting, accepted. Working paper available online

Probabilistic Electric Load Forecasting: A Tutorial Review

Tao Hong and Shu Fan

Load forecasting is a fundamental business problem established since the inception of the electric power industry. Over the past 100 plus years, both research efforts and industry practices in this area are primarily focused on point load forecasting. In the recent decade, due to the increased market competition, aging infrastructure and renewable integration requirements, probabilistic load forecasting is becoming more and more important to energy systems planning and operations. This paper offers a tutorial review of probabilistic electric load forecasting, including notable techniques, methodologies, evaluation methods, and common misunderstandings. We also underline the need to invest in additional research, such as reproducible case studies, probabilistic load forecast evaluation and valuation, and consideration of emerging technologies and energy policies in probabilistic load forecasting process. 

Tuesday, August 18, 2015

Job Opening at NCEMC: Load Research Analyst

Posting a job for my friend at NCEMC. Its forecasting team has been at the leading edge of the load forecasting research and practice. Over the past several years, we have worked together to develop several load forecasting methodologies that are very useful in practice. Most of them are now being used by many other power companies worldwide. You may check out our recent papers on long term probabilistic load forecasting and normalization, weather station selection, and residual simulation for probabilistic load forecasting. We have also planned many exciting research activities for the next few years, of which some are listed in the job description below. I would like to add one more personal note about this job: I think the hiring manager is another reason that this position is so appealing. I haven't seen many managers who are as generous and supportive as this one in terms of professional development for the employees. If you are interested, please submit your resume online HERE.

Job Opening at NCEMC: Load Research Analyst

North Carolina Electric Membership Corporation, one of the largest generation and transmission cooperatives in the country, seeks a Load Research Analyst for its TSE Services Division.

The successful applicant will join a load forecasting team that is empowered with the latest technologies and cutting edge methods for both short- and long-term demand modelling.

Activities will also include measurement and verification of demand-response programs and the development of small area real-time forecasts for Distribution Systems Operations.

The position requires advanced knowledge of applied statistics, microeconomics, econometrics, time-series methods, statistical analysis of large data sets and load research methods.

A Bachelor of Science degree in engineering, economics, statistics, meteorology or equivalent is required and an advanced degree is preferred.

A minimum of five years professional experience in utility load forecasting or a related field and experience with forecasting tools from SAS is desired.

Applicants should submit their resume at

Friday, July 31, 2015

Reading for Writing

Writing skills are badly needed in the professional world, no matter in the industry or academia. I have seen many people (mostly international students, including myself) struggling with their writing skills. I don't believe there is any shortcut but constant and regular writing practice to improve writing skills. In addition, I think reading is a good compliment to writing practice. Here I'm putting together a list of recommended books.

The first two are on general writing (not necessarily scientific writing):
  • On Writing Well by William Zinsser
  • The Elements of Style by William Strunk and E. B. White
Then here are two books for scientific writing:
  • Scientific Writing and Communication: Papers, Proposals, and Presentations by  Angelika Hofmann
  • Writing Science: How to Write Papers That Get Cited and Proposals That Get Funded by Joshua Schimel 
I also got a list of classic novels from my colleague Jason Wilson (co-author of my TSG2014 paper):
  • Slaughterhouse Five by Kurt Vonnegut, Jr.
  • Gulliver’s Travels by Jonathan Swift
  • Ulysses by James Joyce
  • Hamlet by Shakespeare 
  • 1984 by George Orwell
  • Brave New World by Aldous Huxley
  • Animal Farm by George Orwell
  • The Grapes of Wrath by John Steinbeck
  • A Farewell to Arms by Ernest Hemingway
  • The Old Man and The Sea by Ernest Hemingway
  • Charlotte’s Web by E.B. White
  • War and Peace by Leo Tolstoy
  • Jurassic Park by Michael Crichton
Here I'm trying not to recommend a list of scientific papers in energy forecasting. I would rather suggest that we focus on the analytical and technical aspects of those papers.

Happy reading!

Saturday, July 25, 2015

Combining Load Forecasts from Independent Experts: Experience at NPower Forecasting Challenge 2015

Forecast combination is regarded as one of the best practices of forecasting. I think it is a straightforward and practical approach to improving existing forecasts. This paper describes the method my students took in the NPower Forecasting Challenge 2015. We will present the paper at the 47th North America Power Symposium.

Jingrui Xie, Bidong Liu, Xiaoqian Lyu, Tao Hong, and David Basterfield, "Combining load forecasts from independent experts: experience at NPower forecasting challenge 2015", the 47th North American Power Symposium (NAPS2015), October 4 - 6, 2015

Combining Load Forecasts from Independent Experts
Experience at NPower Forecasting Challenge 2015

Jingrui Xie, Bidong Liu, Xiaoqian Lyu, Tao Hong, and David Basterfield


The NPower Forecasting Challenge 2015 invited students and professionals worldwide to predict daily energy usage of a group of customers. The BigDEAL team from the Big Data Energy Analytics Laboratory landed a top 3 place in the final leaderboard. This paper presents a refined methodology based on the implementation during the competition. We first build the individual forecasts using several forecast techniques, such as Multiple Linear Regression (MLR), Autoregressive Integrated Moving Average (ARIMA), Artificial Neural Network (ANN) and Random Forests (RF). We then select a subset of the individual forecasts based on their performance on a validation period, a.k.a. post-sample. Finally we obtain the final forecast by averaging the selected individual forecasts. The forecast combination on average yields a better result than the forecast from a single technique.

Friday, July 24, 2015

Tao Hong: Be Honest

Below is my recent interview with T&D World Magazine. The original version is HERE.

Tao Hong: Be Honest

Tao Hong always sticks to his integrity, especially when it comes to energy forecasting. As graduate program director and EPIC assistant professor at the Systems Engineering and Engineering Management Department at the University of North Carolina at Charlotte, he said the best advice he has ever received is to always be honest.
Sometimes we are pressured to make the forecasts following someone else' personal agenda. Rather than modeling other people's mind and making fraudulent forecasts, we should always stick to our integrity.
Hong will be presenting Energy Forecasting in the Smart Grid Era (blog post) at the 2015 IEEE PES General Meeting, being held July 26-30, in Denver, Colorado. The full-day tutorial covers how wide-range deployment of smart grid technologies enables utilities to monitor the power systems and gather data on a much more granular level than ever before. While the utilities can potentially better understand the customers, design the demand response programs, forecast and control the loads, integrate renewable energy and plan the systems, etc., they are facing analytic issues with making sense and taking advantage of the "big data".

Monday, July 13, 2015

Job Opening at TEP: Customer Analytics & Data Forecasting Analyst

Posting a position for a friend at TEP. Click THIS LINK to see the details of the position

Customer Analytics & Data Forecasting Analyst

Company: Tucson Electric Power
Location: Tucson, AZ
Job Category: Rates and Revenue Requirements
Position Type: Unclassified

Friday, June 26, 2015

What's New in Energy Forecasting - Jun 2015

It's time for the mid-year report and summary of the exciting events in the energy forecasting community:

1. Global Energy Forecasting Competition 2014

The competition was launched in August 2014 and ended in December 2014. We are now in the stage of post-competition activities, such as organizing conference presentations and paper publications. Many finalists of the competition will gather at the IEEE PES General Meeting 2015. The papers are expected to be published in the IJF special issue on probabilistic energy forecasting in early 2016. To follow the update of this competition and future events, please join this LinkedIn group.

2. Activities at IEEE PES General Meeting

Wednesday, June 17, 2015

On Normality Assumption in Residual Simulation for Probabilistic Load Forecasting

I have been so fortunate to work with many talents in both industry and academia. Their involvement has added significant values to most, if not all, of my research projects. This paper is the result of one of those great examples.

The motivation of this project was simple. We were interested in improving the forecasts we produced for NCEMC a few years ago. (You can check our TSG2014 paper for the methodology we used to produce the previous forecasts. The same methodology was used in my recent EISPC/NARUC report.)

The initial idea was simple, too. Since all forecasts have errors, we would like to see if modeling and simulating the residuals can help improve the probabilistic forecasts. We expect an YES answer, because so many papers in the literature have reported a very similar approach. We also had a little doubt, because none of those papers really verified the approach via any formal error measures for probabilistic forecasting.

Monday, June 15, 2015

Energy Forecasting Talks at ISF2015

It's only one week toward the International Symposium on Forecasting 2015. I took a brief look at the program and found 5 sessions including 14 talks on energy forecasting. The at-a-glance program schedule is available HERE.

Wednesday, May 20, 2015

Energy Forecasting Activities at PESGM2015

IEEE Power & Energy Society just released the technical program agenda for PESGM2015. I'm pleased to highlight the two days on energy forecasting organized by our working group.

Please note that the Wednesday afternoon's panel session is in Plaza 3. We also added a reception on Wednesday evening.

Saturday, July 26, 2015

Energy Forecasting in the Smart Grid Era (full-day tutorial)
8:00 AM - 5:00 PM
Instructors: Tao Hong, Shu Fan, Hamidreza Zareipour, and Pierre Pinson

Wednesday, July 29, 2015