Announcing Global Energy Forecasting Competition 2021: Solar and Net Load Forecasting

Dear colleagues and readers of this blog,

Last year at ISF2019 in Thessaloniki, Greece, I mentioned the possibilities of running the next Global Energy Forecasting Competition in the SWEET membership meeting. Then the COVID-19 pandemic put everything on hold. 

Today, I'm pleased to announce GEFCom2021: Solar and Net Load Forecasting. 

GEFCom2021 will feature two tracks, solar irradiance forecasting and net load forecasting. Dr. Dazhi Yang will chair the solar irradiance forecasting track, while I'm taking care of the net load forecasting track.

This competition will inherit the bi-level setup of GEFCom2017. We will use a qualifying match math to bring together contestants from various domains, and to help people get familiar with solar and load forecasting problems. Then a final match will determine the winners. 

Additional details of GEFCom2021 will be released in Spring 2021. Please join the email list via this REGISTRATION FORM to get timely updates about GEFCom2021.

Hope you all stay safe and enjoy the holiday season!

Tao

Forecasting with High Frequency Data: M4 Competition and Beyond

M4 competition was a huge success. The International Journal of Forecasting just published a full issue covering all aspects of the competition. I was honored to be invited by the guest editors to write a commentary paper, which focused on the hourly series of the competition.

According to the organizers (see HERE), the M5 competition is coming soon!

Citation

Tao Hong, "Forecasting with high frequency data: M4 competition and beyond," International Journal of Forecasting, vol.36, no.1, pp.191-194, January, 2020. (ScienceDirect)

Forecasting with High Frequency Data: M4 Competition and Beyond

Tao Hong

Abstract

The M4 competition included 100,000 time series, with the frequencies ranging from yearly to hourly. The team rankings differ notably across frequencies for both point and probabilistic forecasting. I discuss the performances of these methods, with an emphasis on the hourly series of the M4 competition. I also discuss forecasting with high-frequency data in general.

Leaderboard for BFCom2018 Final Match!!!

The final match of the BigDEAL Forecasting Competition 2018 was on probability daily peak hour forecasting, a very important problem in today's electricity market but new to the academic literature. Even without any monetary prize, all 16 finalists from 5 countries submitted their forecasts. (See the qualifying match leaderboad HERE.)

The figure below shows the leaderboard for BFCom2018 Qualifying Match. The green highlighted ones are in-class students. I also created a naive forecast, which is highlighted in red. 

BigDEAL Forecasting Competition 2018 Final Match Leaderboard

One of my students Zehan Xu, who was auditing the class but got disqualified in the qualifying match, also worked on the final problem and submitted his forecast on time. I included his score on the leaderboard, but marked his ranking as "BR-6", which means bragging right for ranking #6. His ranking does not affect the rankings of the other teams. 

Congratulations to all the BFCom2018 finalists for completing this competition! 

To get updates about the follow-up events, please follow my twitter and/or connect to me on LinkedIn.

Winning Methods from BFCom2018 Qualifying Match

I invited the BFCom2018 finalists to share their methods used at the qualifying match. Here are the ones I've received so far.

#1. Geert Scholma

Team member: Geert Scholma

Software: Excel, R (dplyr, lubridate, ggplot2, plotly, tidyr, dygraphs, xts, nnls)

Core technique: Multiple Linear Regression.

The model includes the usual variables with some special recipe: 5 weekdays; federal holidays; strong bridge days (mo before / fr after); weak bridge days (others); 4th degree polynomials for exponentially weighted moving average temperatures on 3 timescales (roughly 1 day, 1 week, 1 month) with optimized decaying factors; 4th degree polynomial time trend for long term gradual changes, changing in a constant value after the last training date; 8th degree polynomial year day for yearly shape, with weekend interaction.

Core methodology: No data cleaning. 1 weighted weather station, based on the non negative linear regression coefficients of a second model step that combined the predictions of all the single weather station driven models of a first step.

Key reference: (Hong, Wang, & White, 2015).


#2. Redwood Coast Energy Authority

Team member: Allison Campbell, Redwood Coast Energy Authority and UNCC

Software: Python (SKLearn package LinearRegression, and the genetic algorithm package DEAP)

Core technique: Multiple Linear Regression.

I adapted the DEAP One Max Problem to optimize selection of weather stations. The bulk of my model is built from Tao's vanilla benchmark, with the inclusion of lagged temperature, weighted moving average of the last day's temperature, transformation of holidays to weekend/days, and exponentially weighted least squares.  Before the regression, I log transformed the load.  I also created 18 "sister" forecasts by redefining the number of months in a year to be 6 to 24.  This model was informed by Tao's doctoral thesis, Hong, Wang, White 2015 (Weather Stn Selection), Wang, Liu, Hong 2016 (Recency Big Data), Nowotarski, Liu, Weron, Hong 2016 (Combining Sisters), Xie, Hong 2018 (24 Solar Terms), and Arlot, Celisse 2009 (CV for model selection).


#5. Masoud_BigDEAL

Team member: Masoud Sobhani, UNCC

Software: SAS (proc GLM)

Core technique: Multiple Linear Regression

I work with Dr. Hong in BigDEAL lab and I am the TA of “Energy Analytics” course this semester. For the first few assignments of this class, we gave the same dataset to the student to make them improve the accuracy of their forecast after they learned different forecasting skills. Like previous classes, Dr. Hong asks me to prepare a benchmark forecast for the class. I built a model during the first lecture and we kept it as the benchmark for all assignments. Later, Dr. Hong decided to make a competition using the same dataset for the qualifying exam. My initial benchmark model was still in the leader board and fortunately qualified to the next round.

In this model, I did not do any data cleansing and I used the raw data for the forecasting. The core technique that I used was based on Vanilla Benchmark Model with recency (Wang, Liu, & Hong, 2016) and holiday effects (Hong, 2010). This model uses third order polynomials of temperature and calendar variables and interactions between them. I removed the Trend variable and used 14 lagged temperatures. For the weather station selection, I employed the exact method proposed in (Hong, Wang, & White, 2015).


#7. SaurabhSangamwar_BigDEAL

Team Member: Saurabh Sangamwar, UNCC

Software: SAS (proc GLM)

Core technique: Multiple Linear Regression

Methodology:

• Weather station selection using proposed approach mentioned in (Hong, Wang, & White, 2015)
• Used 24 solar terms to classify the data as proposed in (Xie & Hong, 2018)
• Added recency effect to Tao’s Vanilla Benchmark model as proposed in (Wang, Liu, & Hong, 2016)
• Used holiday effect (considering holiday as Sunday and day after holiday as Monday), weekend
• effect, trend variable (Increasing serial number), maximum and minimum temperature of day and its interaction with month, solar terms and hour is considered. While forecasting using solar terms solar month 5 and 4 are grouped together.
• Used 2 years of training period to train the model i.e.,year 2006 and 2007 to train and 2008 load data was forecasted.
• Used 3- fold cross validation and stepwise variable selection method to select the parameter, number of lagged effects.
• As there was different lagged effect for each year. Also, solar terms were good instead of using Gregorian calendars months as class variable and for some cases vice a versa. So, generated the point forecast from 11,12,13 and 14 lagged effect for solar terms and Gregorian calendar. So total 8-point forecasts were generated and finally submitted the average of 8 forecasts.

#10. YikeLi_BigDEAL

Team member: Yike Li, Accenture and UNCC 

Software: SAS (proc GLM)

Core techniques: Multiple Linear Regression

Core methodology:

• Weather station selection: A modified version of (Hong, Wang, & White, 2015) by evaluating all possible combinations of top selected weather stations. Selecting the virual station based on three-fold cross validation.
• Recency effect:  Performed a 2-dimensional forward stepwise analysis. Assumption is the MAPE results of each d-h combinations on the validation period (d=0~6, h=0~24) form a convex hull; Starting from d=0 and gradually adding the h terms to Tao’s vanilla model, until adding more temperature lags to the existing model won’t yield better MAPE; Keep the selected h value and gradually add d terms to the existing model, until adding more past daily average to the existing model won’t yield better MAPE. 

#13. 4C

Team members:

• Ilias Dimoulkas, KTH Royal Institute of Technology, Stockholm, Sweden
• Peyman Mazidi, Loyola Andalucia University, Seville, Spain
• Lars Herre, KTH Royal Institute of Technology, Stockholm, Sweden
• Nicholas-Gregory Baltas, Loyola Andalucia University, Seville, Spain

Software: Matlab / Matlab Neural Network Toolbox

Technique: Feed-forward Neural Networks

Methodology:

• Data cleansing. Missing values at the spring daylight saving hours were filled with the average of the previous and the following hours. Double values at the fall daylight saving hours were replaced by their average value. No other data cleansing or outlier detection was done.
• Weather station selection. The technique described in (Hong, Wang, & White, 2015) was used with the difference that neural networks were used to make the forecasts instead of multiple linear regression. 
• Feature selection. Forward sequential feature selection was used. The initial pool of variables consisted of time variables (year, month, hour, etc.), temperature related variables (temperature, power, lags, simple moving average) and cross effects between the temperature and the time variables. The pool contained 172 variables in total. The evaluation was also based on neural networks forecasts. The final feature set consisted of 31 variables.
• Forecast. 10 neural networks were trained on the whole data set (years 2005-2007). The forecast for year 2008 was the mean forecast of the 10 neural networks.

#13. AdG

Team member: Andrés M. Alonso, Universidad Carlos III de Madrid, Spain.

Software: Matlab (Statistics and Machine Learning toolbox)

Technique: support vector regression

In this project, I use SVM regressions to predict hourly loads using explanatory variables such as temperatures, day of the week, month, federal holidays, and a linear trend. As in Hong et al (2015), I made a selection of meteorological stations taking the loads of 2007 as a trial period. I selected the five meteorological stations with the best results from MAPE. In the final model, the five temperature measures were considered instead of using an aggregate measure. The local or focused approach consists in selecting days in the training sample that have a temperature behavior similar to the day to be predicted. In that way, the regression is estimated / trained using only similar days. That is, for 2007 (2008), I performed 365 (366) SVM regressions but trained in different samples. For 2007, the focused approach improves the overall approach that uses all data from the training set. 

References used by the finalists:

• Hong, T. (2010), “Short Term Electric Load Forecasting,” Ph.D. Dissertation, Graduate Program of Operation Research and Dept. of Electrical and Computer Engineering, North Carolina State University.
• Wang, P., Liu, B. and Hong, T. (2016) "Electric load forecasting with recency effect: a big data approach, "International Journal of Forecasting, vol.32, no.3, pp 585-597.
• Hong, T., Wang, P. and White, L. (2015) "Weather station selection for electric load forecasting, "International Journal of Forecasting, vol.31, no.2, pp 286-295.
• Tashman, L. J. (2000). Out-of-sample tests of forecasting accuracy: an analysis and review. International Journal of Forecasting, 16(4), 437-450.
• Arlot, S., & Celisse, A. (2010). A survey of cross-validation procedures for model selection. Statistics surveys,4, 40-79.
• Xie, J. and Hong, T. (2018) "Load forecasting using 24 solar terms," Journal of Modern Power Systems and Clean Energy, vol.6, no.2, pp 208-214
• Nowotarski, J., Liu, B., Weron, R. and Hong, T. (2016) "Improving short term load forecast accuracy via combining sister forecasts," Energy, vol.98, pp 40-49

BTW, I also created a new label "winning methods" so that audience of this blog can easily find the winning methods of previous competitions. 

Leaderboard for BFCom2018 Qualifying Match!!!

The forecast submission due date for Qualifying Match of BigDEAL Forecasting Competition 2018 was Nov 4, 2018. Out of 81 teams who registered the competition, 39 teams successfully submitted their forecasts by the due date. 10 teams will be advanced to the final match together with my Energy Analytics class of 2018 including 5 master and PhD students plus the teaching assistant Masoud Sobhani.

Two methods are used to calculate the MAPE of the forecasts. The first one is the direct calculation of Mean Absolute Percentage Error (MAPE) based on the raw forecast submitted by each team, which was originally announced The other is based on bias-adjusted forecast, which is calculated by dividing the hourly load forecast by the coincident monthly energy, and then multiplying it by the actual monthly energy of that month. For each measure, the MAPE of the last ranked in-class student is used as the qualifying bar. A team outperforming either bar can be advanced to the final match.

The figure below shows the leaderboard for BFCom2018 Qualifying Match. The green highlighted ones are in-class students, while the qualifying bar for each measure is in bold. The teams above the red line are the finalists. The "Ranking (BOTH)" column lists the rankings based on the sum of two rankings from both measures.

BigDEAL Forecasting Competition 2018 - Qualifying Match Leaderboard

Congratulations to the BFCom2018 finalists! A tougher problem is waiting for them in the final match :)

p.s., I will organize a series of follow-up events for the winners to present their methodologies. For more information about this qualifying match, please keep an eye on the FAQ page. 

FAQ for BFCom2018 Qualifying Match

After the two-week registration period, we officially kicked off the BigDEAL Forecasting Competition 2018 with 81 teams formed by 142 data scientists across 26 countries. This morning, I sent out the data and instructions to the contestants. If you are a registered contestant but have not yet receive the data and instructions, please contact me directly.

BFCom2018 attracted 142 data scientists from 26 countries.

This blog post lists the frequently asked questions for BEFCom2018. I'll be updating this post as the questions come along, so please stay tuned. 

Q: Which error measure are you going to use to rank the teams?

A: MAPE, mean absolute percentage error.

Q: Why are there 23 hours in Mar 9, 2008 and 25 hours in Nov 2, 2008?
A: They were observed daylight savings time. Similar observations were in the historical years. See THIS BLOG POST for more information. In the original submission template, the hours in Nov 2, 2008 were from 1 to 25. A new submission template was sent to the contestant on Oct 24, 2018, which had the 2nd hour of Nov 2, 2008 repeated twice, to match the temperature of 2008.

Q: There are 28 weather stations, but only one load series. Which weather stations shall I use?

A: That's part of the challenge. Read this weather station selection paper for more information. 

Q: I'm new to load forecasting. Where shall I get started?
A: This qualifying problem is very similar to the load forecasting track of GEFCom2012. Reading the papers from those winning teams should help.

Q: We are going to use multiple methods. Can we submit multiple forecasts?
A: No. You should only submit one forecast for grading. If you have multiple forecasts, you may consider combining them. This paper may give you some idea about forecast combination.

Q: The local economy information, which was not given in the data, may have some significant effects to the forecasting period. Would you provide the local economy information? (For details, see Geert Scholma's comment under the original BFCom2018 announcement.)
A: No. We will add an error measure that calculate MAPE on bias-adjusted load forecast. We will adjust the hourly forecast based on the coincidence monthly energy, so that your forecasted energy of each month equal to the actual monthly energy. Beating the last-ranked in-class student on either measure can secure the ticket to the final match.

Q: I did not pass the qualifying match bar, but I'm very interested in learning from the winners about their methodologies. Would you summarize their methods?
A: I will organize a series of webinars for the finalists to talk about their methods, though the webinars are not recorded. I will also invite the finalists to summarize their winning methods to post on the blog.

Q: I'm a PhD student just starting my research in energy forecasting. I've learned a lot from this competition. Will you organize this again?
A: Yes. This is not the first BigDEAL Forecasting Competition. It will not be the last either. You can follow my twitter, subscribe to this blog, and/or connect to me on LinkedIn to get updates about events like this.

(To be continued...)

BigDEAL Forecasting Competition 2018

[Update Oct 22, 2018] The registration is closed. 142 data scientists from 26 countries have formed 81 teams to join BFCom2018. See the news article from UNCC College of Engineering. An FAQ page is set up to address questions for the qualifying match.
=======================

This semester I'm teaching Energy Analytics for the fifth time. The course has earned its reputation on the UNC Charlotte campus and even around the utility industry, for its toughness, high withdraw rate, and challenging nature. Here are some comments from the students in 2015 and 2017. Nowadays, not many students even dare to register the course. 

After the first midterm exam last week, I have five students left in the class. These five "survivors" (out of more than a dozen students at the beginning of the semester) have completed two assignments and one exam. I am impressed by their submissions every time. I must confess that this is by far the most academically strong class I've ever had for this course, even stronger than the group that won several award plaques in GEFCom2014. 

Previously, I sent students of this course to the competitions, such as GEFCom2014 and NPower Forecasting Challenge, where they can solve some conventional energy forecasting problems while competing with others around the globe. 

This year, thanks to the outstanding performance of these students, I was spending a lot of time trying to figure out a challenge for them. Finally, I decided to give them a new load forecasting problem to solve. 

I'll keep the problem secret for now, but I can tell that a practical solution to this problem can save power companies a lot of money. To those who are interested in writing academic papers, a winning solution to this problem should greatly increase the likelihood of having the manuscript accepted by the top venues for energy forecasting papers, such as International Journal of Forecasting (IJF) and IEEE Transactions on Smart Grid (TSG). 

The competition is by invitation only. The ones who are interested in joining this competition should first pass the qualifying match. I will use the first homework problem of Energy Analytics for the qualifying match. A contestant has to beat the last-ranked student of my class to receive the invitation to BFCom2018. If nobody beats any of my students, I'll just run the competition with the in-class students. 

For the qualifying match, I'll provide three years of hourly load and temperature, and one year of hourly temperature for the fourth year. The contestants should submit the ex post load forecast for the fourth year. The temperature data is from 28 weather stations. To excel in the qualifying match, the contestants may want to read two of my IJF papers on weather station selection and recency effect. 

Important Dates

Oct 8, 2018 - Registration open. 

Oct 21, 2018 - Registration close. 

Oct 22, 2018 - Qualifying match data release.

Nov 4, 2018 - Qualifying match submission due. 

Nov 5, 2018 - Leaderboard published; BFCom2018 invitation sent. 

Dec 3, 2018 - BFCom2018 winners announced. 

Note: There is no monetary prize for this competition. The leaderboard will be published on this blog. I will consider providing research assistantships to the top three contestants if they are interested in joining my lab as PhD students.

If you are interested, please register HERE. See you in the game!

Which Model is the Best?

Recently Rob Hyndman blogged about the history of forecasting competitions. I have read the post three times already. I learned something new each time. I wish every reader of this blog can also read that post and learn something from the history. Nevertheless, I would like to highlight a paragraph here:

[...] This reveals a view commonly held (even today) that there is some single model that describes the data generating process, and that the job of a forecaster is to find it. This seems patently absurd to me — real data comes from much more complicated, non-linear, non-stationary processes than any model we might dream up — and George Box famously dismissed it saying “All models are wrong but some are useful”.

I have been in the field load forecasting for a little more than 10 years. During the past decade, I received many "the best" questions: Which model is the best? Which technique is the best? Which software is the best? Which variable is the best? ...

There are some variants to these questions: Do you use neural networks? Do you use population in long term load forecasting? Do you use normal weather? Do you think demand response can reduce peak demand? Do you think a MAPE value of 20% is too high? ...

While the folks who asked these questions may expect a crisp answer of the best model, technique, software, variable, or a YES/NO answer, I had to disappoint them with "depends", sometimes followed by a lengthy elaboration.

It depends on the data, the business needs, the production environment, and many other factors... 

I don't have a single model to sell to my clients as "the best model". I recommend proper methodologies to the clients after making a comprehensive evaluation of their situations.

I think the root of these "the best" questions is that commonly held view:

There is some single model that describes the data generating process. 

I disagree with this view, and don't know where the view is originally coming from. I wish someone can write another history article to explain the source.

In load forecasting, there is no universally best model. We need empirical studies, many empirical studies, to show some evidence that one method is superior in some aspect. That's why I have been promoting reproducible research and benchmarking data pool and models.

In computational complexity and optimization, there is actually a theorem, no free lunch theorem. 

RTE Forecasting Challenge 2018

RTE, the French TSO is organizing its forecasting challenge for the second time. This short-term winter electricity demand challenge includes two parts, point forecasting and probabilistic forecasting. The registration will open until January 21, 2018, followed by both parts of the challenge running simultaneously from January 22 to February 10, 2018.

If you are interested, you can register at datascience.net. For non-French speakers, if you see the website in French, you may click the UK flag on the top to view the English version.

According to Geert Scholma, who took the 4th place in the first RTE forecasting challenge, it was "the most exciting competition so far". I guess this one will be very competitive too.

It's  nice to kick of the new year with such an interesting competition, isn't it? 

UNC Charlotte Students Winning All Top 3 Spots of NPower Forecasting Challenge 2017

Every year, RWE npower, a large electricity generator and supplier of gas and electricity based in the United Kingdom hosts a forecasting competition to recruit summer interns. While the internships are only open to UK students, the competition is open to the world. Hundreds of students and working professionals have participated in these npower forecasting challenges in the past few years. Every time I sent a few students to the competition. Every time, they took a few the top spots (see 2015 electricity, 2015 gas, and 2016).

This year, 19 UK teams and 26 international teams joined the competition. Npower created a separate leaderboard for the UK students. The top 1 UK team would rank #9 among all teams. The screen shot below shows the top 8 teams. The official site is HERE.

For the first time, my students took all top 3 spots. They came from two of my classes: Technological Forecasting and Decision Making (Spring 2017) and Energy Analytics (Fall 2017). Most of them are currently enrolled in the master capstone projects under my supervision. My courses are among the most challenging ones in the college. The students had to spend tremendous amount of time to earn the credits. I'm glad that they have acquired some useful skills from the class and showed off their analytical capabilities through the competition. I asked the top teams to summarize their methodology in the comment field below.

Congratulations, 49ers!

NPower Forecasting Challenge 2017

It's time for npower forecasting challenge 2017! The registration will close on Nov 2nd, 2017. You don't have to be a UK student or citizen to join the game, but the prizes and internship opportunities are for the UK people only. The organizer also told me that you may register as a single-person team if you like. Since the registration form asks for multiple names, you may put your own name and contact information twice.

This is the list of blog posts about the previous npower forecasting challenges, where you can find our winning methodologies. 

Look forward to seeing you in the competition!

IEEE PES Announces Winning Teams for Global Energy Forecasting Competition 2017

More than 300 students and professionals from more than 30 countries formed 177 teams to compete on hierarchical probabilistic load forecasting, exploring opportunities from the big data world and tackling the analytical challenges.

PISCATAWAY, N.J., USA, August 18, 2017 – IEEE, the world's largest professional organization advancing technology for humanity, today announced the results of the Global Energy Forecasting Competition 2017 (GEFCom2017), which was organized and supported by the IEEE Power & Energy Society (IEEE PES) and the IEEE Working Group on Energy Forecasting (WGEF).

RTE Day-ahead Load Forecasting Competition 2017

For many years, the Transmission System Operator RTE has been building electricity demand forecasts, ensuring the ability to match supply and demand at all times and, consequently, guaranteeing power system reliability.

Leaderboard for GEFCom2017 Qualifying Match!!!

[Update 5/18/2017]: ISO NE just released the April load data two days ago. Jingrui and I have updated the leaderboard for the qualifying match. Please check the rankings and let us know by 5/26/2017 if there is any issue.

The six rounds of GEFCom2017 qualifying match just ended last week. I'm sure that the contestants are anxiously waiting for the leaderboard. Here is a brief report. I'll update this post as ISO New England releases its recent load data.

Out of 177 registered teams, 73 have submitted entries to the defined track, and 26 to the open track. After six rounds, 53 teams completed the defined track with at least 4 submissions, while 20 completed the open track. 

The due date of report and code is on March 10th, 2017. Please send them to hong.bigdeal@gmail.com. Follow the same protocol as the forecast submissions. Please follow THIS GUIDE to prepare the report.

Jingrui Xie created two benchmarks:

• Vanilla Benchmark, which has been used to calculate the scores of the teams in each round. See Q7 of THIS FAQ for more information.
• Rain Benchmark, which will be used to select the teams being advanced to the final match.  

(As an organizer of GEFCom2017, Jingrui Xie is not eligible for the prize.)

The spreadsheet with detailed scores can be accessed HERE. The higher the score is, the higher the rank is. 

Stay tuned :)

Mark Your 2017 Calendar: Tao's Recommended Conferences for Energy Forecasters

I didn't realize the overdue of this post until I just hit the road for my first trip of 2017. Here is the 2017 list of my recommended conferences for energy forecasters:

1. International Symposium on Energy Analytics (ISEA2017, Cairns, Australia, June 22-23, 2017)

Even if you missed all the other events down this list, you can still find the year rewarding by attending ISEA2017, the first-ever gathering of world-wide energy forecasters. Our generous sponsors, the International Institute of Forecasters (Super Sponsor), Tangent Works (Gigawatt Sponsor) and the State Grid Electric Power Research Institute (Kilowatt Sponsor), have helped bring the registration fees down. There are many reasons to join the party. You will meet the winners of GEFCom2017. You will hear the presentations from world-class energy forecasting researchers and practitioners. You will network with energy forecasting colleagues from more than a dozen countries. And of course, you will enjoy two World Heritage sites side-by-side.

2. Tao's courses

The next two SAS courses on load forecasting have been scheduled in Charlotte, March 27-29.

• Electric Load Forecasting: Fundamentals and Best Practices
• Electric Load Forecasting: Advanced Topics and Case Studies

In addition, I'm going to teach these three courses through EUCI:

• Fundamentals of Utility Analytics: Techniques, Applications and Case Studies
• Introduction to Energy Forecasting
• Long Term Load Forecasting

Stay tuned with the training page of Hong Analytics for the recent updates of all training courses.

3. Conferences from other professional organizations

I will attend the following three, as always:

• International Symposium on Forecasting (ISF2017, Cairns, Australia, June 25-28, 2017)
• IEEE PES General Meeting (PESGM2017, Chicago, IL, USA, July 16-20, 2017)
• AEIC/WLRA Annual Conference (Time & Location TBD)

Look forward to seeing you in these fantastic events!

Winning Methods from npower Forecasting Challenge 2016

RWE npower released the final leaderboard for its forecasting challenge 2016. I took a screen shot of the top teams. Interestingly, the international teams (colored in red) took over all of the top 6 places. Unfortunately, some of those top-notch UK load forecasters did not join the competition. I'm hoping that they can show up at the game to defend the country's legacy:)

RWE npower Forecasting Challenge 2016 Final Leaderboard (top 12 places)

In each of the previous two npower competitions, I asked my BigDEAL students to join the competition as a team. In both competitions, they were ranked top and beating all UK teams (see the blog posts HERE and HERE). We also published our winning methods for electricity demand forecasting and gas demand forecasting.

This year, instead of forming a BigDEAL team, I sent the students in my Energy Analytics class to the competition. The outcome is again very pleasing. The UNCC students took two of the top three places, and four of the top six places. What makes me, a professor, very happy is the fact that the research findings has been fully integrated into the teaching materials and smoothly transferred to the students in the class. (See my research-consulting-teaching circle HERE.)

OK, enough bragging...

I asked the top teams share their methodologies with the audience of my blog as what we did in BFCom2016s. Here they are:

FAQ for GEFCom2017 Qualifying Match

I have received many questions from GEFCom2017 contestants. Many thanks to those who raised the questions. This is a list of frequently asked questions. I will update it periodically if I get additional ones.

Q1. I can't open the link to the competition data. How to get access to the data?

A1. If you cannot access the data via the provided link directly, you may need a VPN service. There are many free VPN services available. Use Google to find one, or post the question on LinkedIn forum to see if your peer contestants can help.

Q2. Can the competition organizer re-post the data somewhere else?

A2. No. We are not going to re-post the data during the competition, because ISO New England updates the data periodically.

Q3. Are we forecasting the same forecasting period in both Round 2 and Round 3? And another same forecasting period in both Round 4 and Round 5?

A3. For GEFCom2017-D, ISO New England updates the data every month, typically in the first half of the month. In Round 2, you will be using the data as of Nov 30, 2016. In Round 3, the December 2016 data should be available as well. For GEFCom2017-O, the data is being updated in real-time. We would like to see if there is any improvement with half a month of information. This set up also gives some flexibility to the contestants. If the team is busy with other commitments during the competition, they may submit the same forecast for both Round 2 and Round 3.

Q4. Can the same team join both tracks?

A4. Yes. A team may even submit the same forecasts to both tracks. Nevertheless, we are expecting higher accuracy in the forecasts of GEFCom2017-O than those of GEFCom2017-D.

Q5. Can one person join two or more teams?

A5. No.

Q6. I'm with a vendor. I don't know if my company wants to put its name as the team name. Can I join the competition personally? If I win, can I add my company as my affiliation and/or change the team name to my company's name?

A6. You can join the competition with or without linking your team to your company. However, you need to make the decision before registration. Once you are in the game, we can not change your affiliation or team name.

Q7. Which benchmark method will be used?

A7. The benchmark method forecasts each zone individually. We will use the vanilla model as the underlying model, simulate the temperature by shifting 11 years of temperature data (2005 - 2015) 4 days forward and backward to come up with 99 scenarios, which will be used to extract 9 quantiles. See THIS PAPER for more details.

Q8. In GEFCom2017-D, are we required to process daylight savings time in a specific way?

A8. No. You can treat the daylight savings time any way you like. THIS POST elaborates my approach, which you don't have to follow.

Q9. In GEFCom2017-D, are we allowed to assume the knowledge of federal holidays before 2011?Can we give special treatments to the days before and after the holidays?

A9. Yes, and yes. The opm.gov website only publishes federal holidays starting from 2011. You can infer the federal holidays before 2011. You can model the days before and after holidays the way you like. I had a holiday effect section in my dissertation, which you don't have to follow. Keep in mind that you should not assume any knowledge about local events or local holidays, such as NBA final games and Saint Patrick's Day.

Q10. The sum of the 8 zones are slightly different from the total demand published by ISO New England. Which number will you use to evaluate the total demand?

A10. Column D of the "ISO NE CA" worksheet.

Q11. For GEFCom2017-D, are you going to provide weather forecasts that every team should use?

A11. No. It is an ex ante hierarchical probabilistic load forecasting problem. We do not provide weather forecasts. The contestants in the GEFCom2017-D track should not use any weather forecasts from other data sources. Nevertheless, the contestants may generate their own weather forecast if they want to. The weather forecasting methodology should be in the final report if they take this route.

Q12. No wind, solar or price forecasting in GEFCom2017? It's a pity!

A12. GEFCom2017 is a load forecasting competition. Unfortunately, we were not able to identify good datasets to set up wind, solar or price forecasting tracks to match the challenge level as this load forecasting problem. Nevertheless, in GEFCom2017-O, you may leverage other data sources to predict wind, solar and prices, which may be good for your load forecasts.

Q13. I'm a professor. Any advice if I want to leverage this competition in class?

A13. It would be nice to leverage the competition in your course. I did so two years ago in GEFCom2014. There will again be an institute prize in GEFCom2017. To aim for the institute prize, I would recommend that you sign up as many teams as possible to maximize the likelihood to win. What I did two years ago was to have each student form a single-person team, and tied the competition ranking to their grades. Anyway, if you are going to join the competition, it's better to have the students look into the data ASAP. The first round submission is due on 12/15/2016.

Q14. Any reference materials we should read before we dive into the competition problem?

A14. For probabilistic load forecasting, you should at least read this recent IJF review paper on probabilistic load forecasting and the relevant references. You can find my recent papers on probabilistic load forecasting HERE. The papers from winning entries of GEFCom2014 are HERE. For hierarchical forecasting, you can check out Hyndman and Athanasopoulos' BOOK and their PAPER. 

Instructions for GEFCom2017 Qualifying Match

The GEFCom2017 Qualifying Match means to attract and educate a large number of contestants with diverse background, and to prepare them for the final match. It includes two tracks: a defined-data track (GEFCom2017-D) and an open-data track (GEFCom2017-O). In both tracks, the contestants are asked to forecast the same thing: zonal and total loads of ISO New England. The only difference between the two tracks is on the input data.

Data 

The input data a participating team can use GEFCom2017-D should not go beyond the following:

1. Columns A, B, D, M and N in the worksheets of "YYYY SMD Hourly Data" files, where YYYY represents the year. These data files can be downloaded from ISO New England website via the zonal information page of the energy, load and demand reports. Contestants outside United States may need a VPN to access the data. 
2. US Federal Holidays as published via US Office of Personnel Management.

The contestants are assumed to have the general knowledge of Daylight Savings Time and inferring the day of week and month of year based on a date.

There is no limitation for the input data in GEFCom2017-O.

Forecasts

The forecasts should be in the form of 9 quantiles following the exact format provided in the template file. The quantiles are the 10th, 20th, ... 90th percentiles. The forecasts should be generated for 10 zones, including the 8 ISO New England zones, the Massachusetts (sum of three zones under Massachusetts), and the total (sum of the first 8 zones).

Timeline

GEFCom2017 Qualifying Match includes six rounds.

Round 1 due date: Dec 15, 2016; forecast period: Jan 1-31, 2017.
Round 2 due date: Dec 31, 2016; forecast period: Feb 1-28, 2017.
Round 3 due date: Jan 15, 2017; forecast period: Feb 1-28, 2017.
Round 4 due date: Jan 31, 2017; forecast period: Mar 1-31, 2017.
Round 5 due date: Feb 14, 2017; forecast period: Mar 1-31, 2017.
Round 6 due date: Feb 28, 2017; forecast period: Apr 1-30, 2017.
Report and code due date: Mar 10, 2017.

The deadline for each round is 11:59pm EST of the corresponding due date.

Submission

The submissions will be through email. Within two weeks of registration, the team leader should receive a confirmation email with the track name and team name in the email subject line. If the team registered both tracks, the team leader should receive two separate emails, one for each track.

The team lead should submit the forecast on behalf of the team by replying to the confirmation email.

The submission must be received before the deadline (based on the receipt time of the email system) to be counted in the leaderboard.

Template

The submissions should strictly follow the requirements below:

1. The file format should be *.xls;
2. The file name should be "TrackInitialRoundNumber-TeamName". For instance, Team "An Awesome Win" in the defined data track's round 3 should name the file as "D3-An Awesome Win".
3. The file should include 10 worksheets, named as CT, ME, NEMASSBOST, NH, RI, SEMASS, VT, WCMASS, MASS, TOTAL. Please arrange the worksheets in the same order as listed above. 
4. In each worksheet, the first two columns should be date and hour, respectively, in chronological order.
5. The 3rdto the 11th columns should be Q10, Q20, ... to Q90. 

The template is HERE. The contestants should replace the date column to reflect the forecast period in each round.

Evaluation

In round i, for a forecast submitted by team j for zone k, the average Pinball Loss of the 9 quantiles will be used as the quantile score of the probabilistic forecast Sijk. A benchmark method will be used to forecast each of the 10 zones. We denote the quantile score of the benchmark method in round i for zone k as Bik.

In round i, we will calculate the relative improvement (1 - Sijk/Bik) for each zone. The average improvement over all zones team j accomplishes will be the rating for team j, denoted as Rij. The rank of team j in round i is RANKij.

The weighted average of the rankings from all 6 rounds will be used to rank the teams in the qualifying match leaderboard. The first 5 rounds will be weighted equally, while the weight for the 6th round is doubled.

A team completing four or more rounds is eligible to for the prizes. The ratings for the missing rounds will be imputed before calculating the weighted average of the ratings.

Prizes

Institute Prize (up to 3 universities): $1000
1st place in each track: $2000
2nd place in each track: $1000
3rd place in each track: $500
1st place in each round of each track: $200

For more information about GEFCom2017, please visit www.gefcom.org.

GEFCom2017: Hierarchical Probabilistic Load Forecasting

IEEE Working Group on Energy Forecasting invites you to join the Global Energy Forecasting Competition 2017 (GEFCom2017): Hierarchical Probabilistic Load Forecasting.

Background

Emerging technologies, such as microgrids, electric vehicles, rooftop solar panels and intelligent batteries, are challenging the traditional operational practices of the power industry. While uncertainties on the demand side are pushing the operational excellence toward the edge of the grid, probabilistic load forecasting at various levels of the power system hierarchy is becoming increasingly important.

GEFCom2017 will bring together state-of-the-art techniques and methodologies for hierarchical probabilistic energy forecasting. The competition features a bi-level setup: a three-month qualifying match that includes two tracks, and a one-month final match on a large-scale problem.

Qualifying match

The qualifying match means to attract and educate a large number of contestants with diverse background, and to prepare them for the final match. The qualifying match includes two tracks, both on forecasting the zonal and total loads of ISO New England (the "DEMAND" column) for the next month in real-time on rolling basis.

The defined-data track (GEFCom2017-D) restricts the data used by the contestants. The data cannot go beyond the calendar data, load (the "DEMAND" column) and temperature data (the "DryBulb" and "DewPnt" columns) provided by ISO New England via the zonal information page of the energy, load and demand reports,  plus the US Federal Holidays as published via US Office of Personnel Management. The contestants may infer day of week and Federal Holidays based on the aforementioned data.

The open-data track (GEFCom2017-O)encourages the contestants to explore various public and private data sources and bring the necessary data into the load forecasting process. The data may include, but is not limited to the data published by ISO New England, the weather forecast data from any weather service providers, the local economy information, the penetration of solar PV published by US government websites.

Final match

The final match (GEFCom2017-F) will be open to the top entries from the qualifying match, tackling a more challenging, larger scale problem than the qualifying match problems. The final match includes one-track only, forecasting the load of a few hundred delivery points of a U.S. utility. The data is from the real world, so the contestants should expect many data issues, such as load transfers and anomalies. Details of the final match will be released on March 15, 2017.

Submission method

To save competition platform costs and implement more sophisticated evaluation methods, the submission will be via email. Within two weeks of the registration, the contestants will receive an email with the instructions about how to submit the forecasts.

Evaluation 

The "DEMAND" column published by ISO New England will be used to evaluate the skills of the probabilistic forecasts. Note that the "DEMAND" data may be revised during the settlement process. The version at the time of evaluation will be used to score the forecasts.

The evaluation metric is quantile score. For each forecasted period, the quantile score of a submitted forecast will be compared with the quantile score of the benchmark. The relative improvement over the benchmark will be used to rate and rank the teams.

World Energy Forecaster Rankings (WEFR)

Many contestants who joined GEFCom2012 also participated in GEFCom2014. To encourage the continuous investments in energy forecasting and recognize those who excel in these competitions, we will start building the World Energy Forecaster Rankings.

The contestants of GEFCom2017 will be eligible to participate in WEFR. We hope the rankings can help reward the participants with career opportunities and tickets to future competitions. In addition, editors of relevant journals can also leverage WEFR to enhance the peer review process.

Prize

IEEE Power and Energy Society budgeted 20,000 for this competition. The prize pool is $18,000, to be shared among the winning teams and institutions from qualifying match and final match.

Publication

Winning teams will be invited to submit papers to a special issue of the International Journal of Forecasting. 

Registration

The maximum team size is three. The team leader should register on behalf of the team. The registration period is from Oct 14, 2016 to Jan 14, 2017. Please register via THIS LINK if you want to join the competition.

Competition timeline

• Competition Problems Release  --  Oct 14, 2016
• Qualifying Match Starts  --  Dec 1, 2016
• Qualifying Match Ends  --  Feb 28, 2017
• Final Match Data Release  --  Mar 15, 2017
• Final Match Submission Due  --  May 15, 2017

Additional rules

• One email account per person. 
• Team mergers are not allowed. 
• Each team leader should have a profile on LinkedIn and join the LinkedIn Group on Global Energy Forecasting Competition. 
• Private sharing data or code outside teams is not allowed, though it is OK to publicly share information with other teams through the LinkedIn Group on Global Energy Forecasting Competition. 
• A winning team should present the methodology at the International Symposium on Energy Analytics (ISEA2017) to claim the full prize money. 
• The GEFCom2017 host has the right to publicly disseminate any entries or disqualify any teams.
• The GEFCom2017 host reserves the right to publicly change these terms and conditions at any time.

For any questions or comments, please put them in the comment field below. Please link your name to your LinkedIn profile. 

NPower Forecasting Challenge 2016

RWE npower is running its forecasting challenge again this year. The purpose is to recruit summer interns from UK schools. Nevertheless, the competition will be open to people outside UK as well.

In 2015, BigDEAL participated in both competitions, one on electric load forecasting, and the other on gas load forecasting. We summarized our methods into two papers (electricity; gas), which may give you some idea about the previous competitions.

The registrations are now open until November 1, 2016. Have fun!