Saturday, February 28, 2015

Error Analysis in Load Forecasting: The Second Growing List

Over the past few years, THE most frequently asked question I have received is like the following:
My forecast error is x percent, how is the accuracy compared with my peers'?
Of course, each question was packaged with details about the jurisdictions and maybe model setups. At first I was trying to provide a customized response every time. Three years ago, I gave a webinar to cover very basic error measures used in load forecasting, which was apparently not enough to cover all the load forecast error/accuracy questions. Last week, after getting another email inquiry about forecasting accuracy, I decided to start this new series of blog posts. It will be the same style as the growing list of load forecasting terminology. The first 8 topics are listed below:
  1. How accurate is my forecast? (recorded webinar)
  2. Factors affecting load forecast accuracy
  3. More about weather variables
  4. How to measure load forecast errors?
  5. Accuracy of ISO/RTO short-term forecasts
  6. What a mess! Problems with reporting long term forecast errors
  7. Models for benchmarking
  8. Which techniques are "more accurate"?
I will add the links as I finish the posts. If you have any feedback, just email me or leave your thoughts in the comment field below. I will update the list accordingly. 

Friday, February 27, 2015

Electric Load Forecasting with Recency Effect: a Big Data Approach

When I first wrote the CFP for the special issue on Analytics for Energy Forecasting with Applications to Smart Grid in 2012, I used the term big data, with a quotation mark. Nowadays, big data is no longer new to the utility industry. In fact the utilities have been working with big data since it was called just "data" - we witness the growth of data to big data in this smart grid era. To collect the most recent progress and advancements in big data analytics, we just issued another CFP for the special issue on Big Data Analytics for Grid Modernization.
What is big data analytics, deep learning, high-performance computing and petabyte size? 
I have three simple criteria:
  1. The data size is larger than what typical data analysis tools can handle. If you are using MS Excel to do some data analysis, then a data file with 1.1 million rows is big data. 
  2. The computing time is longer than the analysis time. Let's say it takes you a few days to think of a design of an algorithm. If testing the algorithm takes a few weeks, then it is big data. 
  3. The problem requires analysis at a higher level of granularity than usual. If your typical load forecasting process rely on monthly data, moving to daily or hourly data may bring you the big data challenge. 
Although these three criteria do not have to be met at the same time to qualify big data analytics, they are indeed connected to each other. Analyzing high resolution data often requires advanced data analysis tools and significant computing time. 

This paper has big data in its title, because it covers the latter two criteria. The regression models we developed in this paper contain up to thousands of variables, which require significant amount of time for parameter estimation, much longer than our thought process. Moreover, we customized the models based on each zone of a geographic hierarchy and each node (hour) of the temporal hierarchy. Of course the forecasting errors are reduced with our proposed approach, which also tells us the importance of powerful computers in load forecasting. 

The case study is based on the GEFCom2012 data published in my 2014 IJF paper Global Energy Forecasting Competition 2012. We compared the results with those in my 2015 IJF paper Weather Station Selection for Electric Load Forecasting.

The working paper is available HERE

Electric Load Forecasting with Recency Effect: a Big Data Approach

Tao Hong, Bidong Liu and Pu Wang


Temperature plays a key role in driving electricity demand. We adopt "recency effect", a term originated from psychology, to illustrate the fact that electricity demand is affected by the temperatures of preceding hours. In the load forecasting literature, the temperature variables are often constructed in the form of lagged hourly temperatures and moving average temperatures. Over the past decades, computing power has been limiting the amount of temperature variables that can be used in a load forecasting model. In this paper, we present a comprehensive study on modeling recency effect through a big data approach. We take advantage of the modern computing power to answer a fundamental question: how many lagged hourly temperatures and/or moving average temperatures are needed in a regression model to fully capture recency effect without compromising the forecasting accuracy? Using the case study based on data from the load forecasting track of the Global Energy Forecasting Competition 2012, we first demonstrate that a model with recency effect outperforms its counterpart in forecasting individual load series at aggregated level by 18% to 20%. We then apply recency effect modeling to customize load forecasting models at low level of a geographic hierarchy, again showing the superiority over a benchmark model by 13% to 15% on average. Finally, we discuss four different implementations of the recency effect modeling by hour of a day. 

Saturday, February 21, 2015

Keep Reading Every Day: Tao's Recommended Websites for Energy Forecasters

Last month, I moved all of my students to the newly renovated BigDEAL. At the same time, I started the BigDEAL seminar series. Part of the seminar is an information sharing session - each participant takes a 5-minute slot to share the most interesting readings of the week with the audience. The purposes are to:
  1. practice the story-telling skills; 
  2. broaden their views of the field; and 
  3. establish the habit of reading every day.

Thursday, February 12, 2015

Mark Your 2015 Calendar: Tao's Recommended Conferences for Energy Forecasters

Update (3/1/2015): Due to high demand of my load forecasting courses, we added an offering at the New York City in May. 
Update (2/18/2015): I just confirmed two more speakers for ISF2015. Now we have six presentations in two sessions. 
Recently I received many inquiries about recommended energy forecasting conferences in 2015. First of all, I have never attended a conference that is perfectly designed and organized for energy forecasters, which motivates me to organize the ultimate energy forecasting conference. If you have to wait for this ultimate one, close this page and stay tuned for another two years. Otherwise, keep reading. I will provide a list of 6 venues for you to consider, in the chronological order.

1. Tao's load forecasting courses (May 27-29, 2015, New York, NY)

I have taught the fundamental course 15 times. More than 150 energy forecasters have attended the course. (See some statistics based on the first 10 offerings) Recently I have developed a one-day advanced level course for those who want some in depth coverage of the subject and hands on experience of SAS. The links to the courses are listed below:
2. 35th International Symposium on Forecasting (ISF2015, June 21-24, 2015, Riverside, CA)

ISF2015 is a great conference if you want to learn the frontiers of forecasting. I'm organizing an energy forecasting session at ISF2015. The two sessions includes six talks with a balanced mix of state-of-the-art research and practice.

Session Title: Frontiers in Electricity Demand Forecasting I: The State of The Practice
Chair: Tao Hong (University of North Carolina at Charlotte, USA)
  • SAS Energy Forecasting: Hourly load forecasting for all horizons
    • Bradley Lawson (SAS, USA)
  • Combining sister load forecasts
    • Tao Hong (University of North Carolina at Charlotte, USA)
    • Bidong Liu (University of North Carolina at Charlotte, USA)
  • MEFM: An R package for long-term probabilistic forecasting of electricity demand
    • Rob J. Hyndman (Monash University, Australia)
Session Title: Frontiers in Electricity Demand Forecasting II: Probabilistic Electric Load Forecasting
Chair: Tao Hong (University of North Carolina at Charlotte, USA)
    • Quantile regression algorithms for forecasting uncertainty in electricity smart meters data
      • Souhaib Ben Taieb, King Abdullah University of Science and Technology, Saudi Arabia.
      • Rob J. Hyndman, Monash University, Australia
      • Marc G. Genton, King Abdullah University of Science and Technology, Saudi Arabia.
    • Electricity demand interval forecasting with Quantile Regression Averaging
      • Jakub Notowarski (Wrocław University of Technology, Poland)
    • The myths of residual simulation for probabilistic load forecasting 
      • Jingrui Xie (University of North Carolina at Charlotte, USA)
    3. 3rd International Conference Energy & Meteorology (ICEM2015, June 22-26, 2015, Denver, CO)

    Weather drives electricity demand and wind/solar power generation. The conference has a unique focus on the interdisciplinary field of energy and meteorology. For more information, please read Make a Difference in the Energy & Meteorology World written by the guest blogger and conference chair Alberto Troccoli.

    4. Modern Electric Power Systems Conference 2015 (MEPS2015, July 6-9, 2015, Wroclaw, Poland)

    If you are in the area of load and price forecasting, you must be familiar with Rafal Weron and his book "Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach" and the recent IJF open access paper "Electricity Price Forecasting: A Review of the State-of-the-art with a Look into the Future". Rafal is based in Wroclaw, Poland. He is also a key player on the organizing committee of MEPS2015. I'm sure he will present some nice forecasting work at the conference.

    5. IEEE Power and Energy Society General Meeting 2015 (PESGM2015, July 26-30, 2015, Denver, CO)

    We have organized two days of agenda on energy forecasting. A full-day tutorial on "Energy Forecasting in the Smart Grid Era", and a full-day session of GEFCom2014 finalist presentations. I will write another blog post when the schedule is finalized.

    6. AEIC/WLRA annual conference (time & location TBD)

    This is a joint conference organized by AEIC Load Research Committee and Western Load Research Association. The two groups used to organize conferences separately. Last year was the first time they held a joint conference. I hope they will continue the joint conference this fall.

    Mark your calendar and enjoy the trips!

    Tuesday, February 10, 2015

    Integrated Energy Forecasting: Improving T&D Planning and Operations

    Comparing with my Foresight paper Energy Forecasting: Past, Present and Future, which was targeting the forecasting community, this paper was mainly written for the T&D engineers and managers. The web/digital version is available HERE.

    Tao Hong, "Integrated Energy Forecasting: Improving T&D Planning and Operations", Electricity Today, pp. 58-62, January/February, 2015

    Friday, February 6, 2015

    Npower Forecasting Challenge 2015

    Apparently after two big runs of the Global Energy Forecasting Competitions, power companies are getting more interested than before in taking the competition approach to making their business decisions. I'm very glad to see things going toward this direction. As I said in Three Worst Marketing Buzzwords for Forecasting Solutions about forecasting accuracy,
    Before doing a pilot, nobody can claim the superior accuracy. Even after the pilot, we cannot guarantee the consistent superiority in the future. Nevertheless, a well-designed pilot can offer customers a comprehensive view of the performance of competing solutions.
    This time it's about hiring summer interns. The message below is from Npower:
    We have a launched a forecasting challenge to help attract students to our graduate programme, however we are also excepting entries from others - who just want to test themselves on some real industry data, they won't be eligible for the cash prize but will be able to see their performance on the leaderboard.
    From the T&C's:
    "Please note this competition is for University students only (see terms & conditions). However if you would still be interested in attempting the challenge please email You will not be eligible for the prize but your forecast will be scored and appear in our rankings."
    Here is the landing page and also a blog entry on the R site - entries close on Feb 13 with the competition kicking off on Feb 16.
    BTW, I did not participate in the design or operations of this challenge, so send all your questions to Npower. 

    Have fun!

    Monday, February 2, 2015

    Job Opening at Vaisala: Renewable Energy Forecasting Analyst

    Posting a job for a friend at Vaisala. Please visit Vaisala career website to apply.


    Vaisala is a global leader in environmental and industrial measurement. Building on 75 years of experience, Vaisala contributes to a better quality of life by providing a comprehensive range of innovative observation and measurement products and services for chosen weather-related and industrial markets.

    The Energy segment of Vaisala is working to speed the adoption of Wind and Solar power around the world through better and more cost effective measurement technologies and information services.  Forecasting services to help get more renewable energy scheduled onto the electric grid is a critical aspect of this mission.

    We are searching for a Renewable Energy Forecasting Analyst to strengthen our team. This position is located in Boston or work remotely within the United States.

    Friday, January 23, 2015

    Job Opening at PNM Resources: Economist

    To apply go to and read a full job description, register, upload a resume and
    answer all posting questions. Deadline is no later than February 8, 2015.

    Job Title: ECONOMIST
    Job ID: 6084993
    Location: PNM Resources, Albuquerque, NM
    Full/Part Time: Full-Time
    Regular/Temporary: Regular

    Sunday, January 11, 2015

    Job Openings at DTU: Assistant/Associate Professor in Energy Analytics & Markets

    This is the first academic job posting on my blog. I'm posting it for my friend and colleague Pierre Pinson. I consider his group a top research group in the area of energy analytics. Knowing Pierre in person and working with him closely on several initiatives, I trust that whomever takes the job will enjoy working with him and have a bright future in the academic world. The original posting is HERE on DTU website.

    Thursday, January 8, 2015

    Make a difference in the Energy & Meteorology World!

    Make a difference in the Energy & Meteorology World!

    Guest Blogger: Dr. Alberto Troccoli
    Commonwealth Scientific and Industrial Research Organization, Australia

    I’ve always felt the urge to see my scientific research come out of the computer screen to land on a practical everyday situation. It is often much easier said than done however. As scientists we like to think our research is going to be used by someone one day, but the truth is that most of the time we prefer the comfort of the "ivory tower". We may feel a sense of relief when, in project proposals, we include the odd sentence about the critical role of our research for society, often added in an attempt to get our proposal across the line.

    Oceanography, seasonal climate forecasts and more generally climate research has been a satisfying endeavor for me for many a year but the baby steps in this area of research, despite the size of the effort, had become less and less fulfilling. What a better time then to use weather and climate (collectively referred to as meteorology) observations and forecasts to directly assist society! And what better time to simultaneously assist society in the once-in-a-few-generation overhaul of the energy system! To contribute to the sustainability, resilience and efficiency of energy systems under ever changing weather and climate appeared (and it still does!) like a thrilling goal!