Who's Who in Energy Forecasting: Geert Scholma

I got to know Geert Scholma from NPower Forecasting Challenge 2015, where he outperformed my BigDEAL students on the leaderboard. Since then, he has been topping the NPower leaderboard every time. Recently, as a winner of the qualifying match of GEFCom2017, he presented his methodology at ISEA2017.

Geert lives in Rotterdam, The Netherlands. He has a strong focus on data science and the energy transition, with a masters degree in physics and 5 years experience as an Energy Forecaster for Energy Retail Company and E.On spin-off Uniper Benelux.

IEEE PES Announces Winning Teams for Global Energy Forecasting Competition 2017

More than 300 students and professionals from more than 30 countries formed 177 teams to compete on hierarchical probabilistic load forecasting, exploring opportunities from the big data world and tackling the analytical challenges.

PISCATAWAY, N.J., USA, August 18, 2017 – IEEE, the world's largest professional organization advancing technology for humanity, today announced the results of the Global Energy Forecasting Competition 2017 (GEFCom2017), which was organized and supported by the IEEE Power & Energy Society (IEEE PES) and the IEEE Working Group on Energy Forecasting (WGEF).

7 Reasons to Attend ISEA2017

The International Symposium on Energy Analytics (ISEA2017) is coming in 7 weeks. If you are still wondering whether you should join the event or not, here are 7 reasons for you to attend ISEA2017:

1. Grow your international network

ISEA2017 is truly international. The early registrations came from 16 countries. As a conference attendee, you will hear 20+ presentations describing methodologies and insights gained from various places in the world. You will also share your experience and expertise with this diverse audience and get their critique and compliment.

2. Check out the winning methods of GEFCom2017

Selected GEFCom2017 teams will be presenting their methodologies at ISEA2017. You will witness the recognition of GEFCom2017 winners and have the face-to-face discussion with them. Rather than reading thousands of energy forecasting papers published every year and wondering which ones work well, you can grasp the secret sauce of the most effective methods during ISEA2017.

3. Experience a novel peer review process 

Whether we like today's peer review system or not, we have to live with it, at least for the next few years before a better one is in place. We have tied ISEA2017 to an IJF special section on energy forecasting, where we try to implement a new peer review process. The ISEA2017 attendees will have the opportunity to experience this new process and help improve it.

4. Peek and shape the future of energy analytics 

If you are struggling with the topic for your next paper, ISEA2017 is a must-attend conference for you. We will discuss the emerging topics as well as the research agenda for the future. Rather than guessing where the future goes, you can contribute to the plan!

5. Attend International Symposium on Forecasting

The 37th International Symposium on Forecasting (ISF) will be held two days after ISEA2017, right at the same location. ISF is the only major scientific forecasting conference I know of. I find it very rewarding to attend ISF, where I hear forecasting topics from various industries, as well as the methodological breakthroughs in general. Many of them could be applied to the energy forecasting problems. Extending the trip to include ISF in your travel plan would be a wise choice.

6. Two world heritage sites in one place

The World Heritage Centre has a list of about 1000 world heritage sites around the globe. Two of them (Great Barrier Reef and Daintree Rainforest) are in Cairns, Australia, making Cairns the only place in this planet with two world heritage sites side by side. ISF organizers have planned the social program including numerous social events and tour opportunities for delegates, their friends and family.

7. Low registration fees

Our sponsors, the International Institute of Forecasters, Tangent Works and Journal of Modern Power Systems and Clean Energy, have generously contributed to the organization of ISEA2017, helping significantly subsidize the registration fees. If you attend both ISEA and ISF, there is an additional discount. To register both ISEA and ISF, click HERE. To register ISEA only, click HERE.

ISEA2017 will be held in Cairns, Australia, June 22-23, 2017. Look forward to seeing you there!

Leaderboard for GEFCom2017 Qualifying Match!!!

[Update 5/18/2017]: ISO NE just released the April load data two days ago. Jingrui and I have updated the leaderboard for the qualifying match. Please check the rankings and let us know by 5/26/2017 if there is any issue.

The six rounds of GEFCom2017 qualifying match just ended last week. I'm sure that the contestants are anxiously waiting for the leaderboard. Here is a brief report. I'll update this post as ISO New England releases its recent load data.

Out of 177 registered teams, 73 have submitted entries to the defined track, and 26 to the open track. After six rounds, 53 teams completed the defined track with at least 4 submissions, while 20 completed the open track. 

The due date of report and code is on March 10th, 2017. Please send them to hong.bigdeal@gmail.com. Follow the same protocol as the forecast submissions. Please follow THIS GUIDE to prepare the report.

Jingrui Xie created two benchmarks:

• Vanilla Benchmark, which has been used to calculate the scores of the teams in each round. See Q7 of THIS FAQ for more information.
• Rain Benchmark, which will be used to select the teams being advanced to the final match.  

(As an organizer of GEFCom2017, Jingrui Xie is not eligible for the prize.)

The spreadsheet with detailed scores can be accessed HERE. The higher the score is, the higher the rank is. 

Stay tuned :)

FAQ for GEFCom2017 Qualifying Match

I have received many questions from GEFCom2017 contestants. Many thanks to those who raised the questions. This is a list of frequently asked questions. I will update it periodically if I get additional ones.

Q1. I can't open the link to the competition data. How to get access to the data?

A1. If you cannot access the data via the provided link directly, you may need a VPN service. There are many free VPN services available. Use Google to find one, or post the question on LinkedIn forum to see if your peer contestants can help.

Q2. Can the competition organizer re-post the data somewhere else?

A2. No. We are not going to re-post the data during the competition, because ISO New England updates the data periodically.

Q3. Are we forecasting the same forecasting period in both Round 2 and Round 3? And another same forecasting period in both Round 4 and Round 5?

A3. For GEFCom2017-D, ISO New England updates the data every month, typically in the first half of the month. In Round 2, you will be using the data as of Nov 30, 2016. In Round 3, the December 2016 data should be available as well. For GEFCom2017-O, the data is being updated in real-time. We would like to see if there is any improvement with half a month of information. This set up also gives some flexibility to the contestants. If the team is busy with other commitments during the competition, they may submit the same forecast for both Round 2 and Round 3.

Q4. Can the same team join both tracks?

A4. Yes. A team may even submit the same forecasts to both tracks. Nevertheless, we are expecting higher accuracy in the forecasts of GEFCom2017-O than those of GEFCom2017-D.

Q5. Can one person join two or more teams?

A5. No.

Q6. I'm with a vendor. I don't know if my company wants to put its name as the team name. Can I join the competition personally? If I win, can I add my company as my affiliation and/or change the team name to my company's name?

A6. You can join the competition with or without linking your team to your company. However, you need to make the decision before registration. Once you are in the game, we can not change your affiliation or team name.

Q7. Which benchmark method will be used?

A7. The benchmark method forecasts each zone individually. We will use the vanilla model as the underlying model, simulate the temperature by shifting 11 years of temperature data (2005 - 2015) 4 days forward and backward to come up with 99 scenarios, which will be used to extract 9 quantiles. See THIS PAPER for more details.

Q8. In GEFCom2017-D, are we required to process daylight savings time in a specific way?

A8. No. You can treat the daylight savings time any way you like. THIS POST elaborates my approach, which you don't have to follow.

Q9. In GEFCom2017-D, are we allowed to assume the knowledge of federal holidays before 2011?Can we give special treatments to the days before and after the holidays?

A9. Yes, and yes. The opm.gov website only publishes federal holidays starting from 2011. You can infer the federal holidays before 2011. You can model the days before and after holidays the way you like. I had a holiday effect section in my dissertation, which you don't have to follow. Keep in mind that you should not assume any knowledge about local events or local holidays, such as NBA final games and Saint Patrick's Day.

Q10. The sum of the 8 zones are slightly different from the total demand published by ISO New England. Which number will you use to evaluate the total demand?

A10. Column D of the "ISO NE CA" worksheet.

Q11. For GEFCom2017-D, are you going to provide weather forecasts that every team should use?

A11. No. It is an ex ante hierarchical probabilistic load forecasting problem. We do not provide weather forecasts. The contestants in the GEFCom2017-D track should not use any weather forecasts from other data sources. Nevertheless, the contestants may generate their own weather forecast if they want to. The weather forecasting methodology should be in the final report if they take this route.

Q12. No wind, solar or price forecasting in GEFCom2017? It's a pity!

A12. GEFCom2017 is a load forecasting competition. Unfortunately, we were not able to identify good datasets to set up wind, solar or price forecasting tracks to match the challenge level as this load forecasting problem. Nevertheless, in GEFCom2017-O, you may leverage other data sources to predict wind, solar and prices, which may be good for your load forecasts.

Q13. I'm a professor. Any advice if I want to leverage this competition in class?

A13. It would be nice to leverage the competition in your course. I did so two years ago in GEFCom2014. There will again be an institute prize in GEFCom2017. To aim for the institute prize, I would recommend that you sign up as many teams as possible to maximize the likelihood to win. What I did two years ago was to have each student form a single-person team, and tied the competition ranking to their grades. Anyway, if you are going to join the competition, it's better to have the students look into the data ASAP. The first round submission is due on 12/15/2016.

Q14. Any reference materials we should read before we dive into the competition problem?

A14. For probabilistic load forecasting, you should at least read this recent IJF review paper on probabilistic load forecasting and the relevant references. You can find my recent papers on probabilistic load forecasting HERE. The papers from winning entries of GEFCom2014 are HERE. For hierarchical forecasting, you can check out Hyndman and Athanasopoulos' BOOK and their PAPER. 

Instructions for GEFCom2017 Qualifying Match

The GEFCom2017 Qualifying Match means to attract and educate a large number of contestants with diverse background, and to prepare them for the final match. It includes two tracks: a defined-data track (GEFCom2017-D) and an open-data track (GEFCom2017-O). In both tracks, the contestants are asked to forecast the same thing: zonal and total loads of ISO New England. The only difference between the two tracks is on the input data.

Data 

The input data a participating team can use GEFCom2017-D should not go beyond the following:

1. Columns A, B, D, M and N in the worksheets of "YYYY SMD Hourly Data" files, where YYYY represents the year. These data files can be downloaded from ISO New England website via the zonal information page of the energy, load and demand reports. Contestants outside United States may need a VPN to access the data. 
2. US Federal Holidays as published via US Office of Personnel Management.

The contestants are assumed to have the general knowledge of Daylight Savings Time and inferring the day of week and month of year based on a date.

There is no limitation for the input data in GEFCom2017-O.

Forecasts

The forecasts should be in the form of 9 quantiles following the exact format provided in the template file. The quantiles are the 10th, 20th, ... 90th percentiles. The forecasts should be generated for 10 zones, including the 8 ISO New England zones, the Massachusetts (sum of three zones under Massachusetts), and the total (sum of the first 8 zones).

Timeline

GEFCom2017 Qualifying Match includes six rounds.

Round 1 due date: Dec 15, 2016; forecast period: Jan 1-31, 2017.
Round 2 due date: Dec 31, 2016; forecast period: Feb 1-28, 2017.
Round 3 due date: Jan 15, 2017; forecast period: Feb 1-28, 2017.
Round 4 due date: Jan 31, 2017; forecast period: Mar 1-31, 2017.
Round 5 due date: Feb 14, 2017; forecast period: Mar 1-31, 2017.
Round 6 due date: Feb 28, 2017; forecast period: Apr 1-30, 2017.
Report and code due date: Mar 10, 2017.

The deadline for each round is 11:59pm EST of the corresponding due date.

Submission

The submissions will be through email. Within two weeks of registration, the team leader should receive a confirmation email with the track name and team name in the email subject line. If the team registered both tracks, the team leader should receive two separate emails, one for each track.

The team lead should submit the forecast on behalf of the team by replying to the confirmation email.

The submission must be received before the deadline (based on the receipt time of the email system) to be counted in the leaderboard.

Template

The submissions should strictly follow the requirements below:

1. The file format should be *.xls;
2. The file name should be "TrackInitialRoundNumber-TeamName". For instance, Team "An Awesome Win" in the defined data track's round 3 should name the file as "D3-An Awesome Win".
3. The file should include 10 worksheets, named as CT, ME, NEMASSBOST, NH, RI, SEMASS, VT, WCMASS, MASS, TOTAL. Please arrange the worksheets in the same order as listed above. 
4. In each worksheet, the first two columns should be date and hour, respectively, in chronological order.
5. The 3rdto the 11th columns should be Q10, Q20, ... to Q90. 

The template is HERE. The contestants should replace the date column to reflect the forecast period in each round.

Evaluation

In round i, for a forecast submitted by team j for zone k, the average Pinball Loss of the 9 quantiles will be used as the quantile score of the probabilistic forecast Sijk. A benchmark method will be used to forecast each of the 10 zones. We denote the quantile score of the benchmark method in round i for zone k as Bik.

In round i, we will calculate the relative improvement (1 - Sijk/Bik) for each zone. The average improvement over all zones team j accomplishes will be the rating for team j, denoted as Rij. The rank of team j in round i is RANKij.

The weighted average of the rankings from all 6 rounds will be used to rank the teams in the qualifying match leaderboard. The first 5 rounds will be weighted equally, while the weight for the 6th round is doubled.

A team completing four or more rounds is eligible to for the prizes. The ratings for the missing rounds will be imputed before calculating the weighted average of the ratings.

Prizes

Institute Prize (up to 3 universities): $1000
1st place in each track: $2000
2nd place in each track: $1000
3rd place in each track: $500
1st place in each round of each track: $200

For more information about GEFCom2017, please visit www.gefcom.org.

GEFCom2017: Hierarchical Probabilistic Load Forecasting

IEEE Working Group on Energy Forecasting invites you to join the Global Energy Forecasting Competition 2017 (GEFCom2017): Hierarchical Probabilistic Load Forecasting.

Background

Emerging technologies, such as microgrids, electric vehicles, rooftop solar panels and intelligent batteries, are challenging the traditional operational practices of the power industry. While uncertainties on the demand side are pushing the operational excellence toward the edge of the grid, probabilistic load forecasting at various levels of the power system hierarchy is becoming increasingly important.

GEFCom2017 will bring together state-of-the-art techniques and methodologies for hierarchical probabilistic energy forecasting. The competition features a bi-level setup: a three-month qualifying match that includes two tracks, and a one-month final match on a large-scale problem.

Qualifying match

The qualifying match means to attract and educate a large number of contestants with diverse background, and to prepare them for the final match. The qualifying match includes two tracks, both on forecasting the zonal and total loads of ISO New England (the "DEMAND" column) for the next month in real-time on rolling basis.

The defined-data track (GEFCom2017-D) restricts the data used by the contestants. The data cannot go beyond the calendar data, load (the "DEMAND" column) and temperature data (the "DryBulb" and "DewPnt" columns) provided by ISO New England via the zonal information page of the energy, load and demand reports,  plus the US Federal Holidays as published via US Office of Personnel Management. The contestants may infer day of week and Federal Holidays based on the aforementioned data.

The open-data track (GEFCom2017-O)encourages the contestants to explore various public and private data sources and bring the necessary data into the load forecasting process. The data may include, but is not limited to the data published by ISO New England, the weather forecast data from any weather service providers, the local economy information, the penetration of solar PV published by US government websites.

Final match

The final match (GEFCom2017-F) will be open to the top entries from the qualifying match, tackling a more challenging, larger scale problem than the qualifying match problems. The final match includes one-track only, forecasting the load of a few hundred delivery points of a U.S. utility. The data is from the real world, so the contestants should expect many data issues, such as load transfers and anomalies. Details of the final match will be released on March 15, 2017.

Submission method

To save competition platform costs and implement more sophisticated evaluation methods, the submission will be via email. Within two weeks of the registration, the contestants will receive an email with the instructions about how to submit the forecasts.

Evaluation 

The "DEMAND" column published by ISO New England will be used to evaluate the skills of the probabilistic forecasts. Note that the "DEMAND" data may be revised during the settlement process. The version at the time of evaluation will be used to score the forecasts.

The evaluation metric is quantile score. For each forecasted period, the quantile score of a submitted forecast will be compared with the quantile score of the benchmark. The relative improvement over the benchmark will be used to rate and rank the teams.

World Energy Forecaster Rankings (WEFR)

Many contestants who joined GEFCom2012 also participated in GEFCom2014. To encourage the continuous investments in energy forecasting and recognize those who excel in these competitions, we will start building the World Energy Forecaster Rankings.

The contestants of GEFCom2017 will be eligible to participate in WEFR. We hope the rankings can help reward the participants with career opportunities and tickets to future competitions. In addition, editors of relevant journals can also leverage WEFR to enhance the peer review process.

Prize

IEEE Power and Energy Society budgeted 20,000 for this competition. The prize pool is $18,000, to be shared among the winning teams and institutions from qualifying match and final match.

Publication

Winning teams will be invited to submit papers to a special issue of the International Journal of Forecasting. 

Registration

The maximum team size is three. The team leader should register on behalf of the team. The registration period is from Oct 14, 2016 to Jan 14, 2017. Please register via THIS LINK if you want to join the competition.

Competition timeline

• Competition Problems Release  --  Oct 14, 2016
• Qualifying Match Starts  --  Dec 1, 2016
• Qualifying Match Ends  --  Feb 28, 2017
• Final Match Data Release  --  Mar 15, 2017
• Final Match Submission Due  --  May 15, 2017

Additional rules

• One email account per person. 
• Team mergers are not allowed. 
• Each team leader should have a profile on LinkedIn and join the LinkedIn Group on Global Energy Forecasting Competition. 
• Private sharing data or code outside teams is not allowed, though it is OK to publicly share information with other teams through the LinkedIn Group on Global Energy Forecasting Competition. 
• A winning team should present the methodology at the International Symposium on Energy Analytics (ISEA2017) to claim the full prize money. 
• The GEFCom2017 host has the right to publicly disseminate any entries or disqualify any teams.
• The GEFCom2017 host reserves the right to publicly change these terms and conditions at any time.

For any questions or comments, please put them in the comment field below. Please link your name to your LinkedIn profile. 

Announcing GEFCom2017: Join the Interest List

I'm sure readers of this blog are anxious for the next Global Energy Forecasting Competition. Today I'm pleased to announce the GEFCom2017, an upgraded version over GEFCom2012 and GEFCom2014.

To bring together contestants with diverse background and to dive deep into the challenging problems, GEFCom2017 will feature a bi-level setup: a three-month qualifying match and a one-month final match. The qualifying match means to attract and educate a large number of contestants with diverse background. The final match will be open to the top entries from the qualifying match, tackling a more challenging, larger scale problem.

Many contestants who joined GEFCom2012 also participated in GEFCom2014. To encourage the continuous investments in energy forecasting and recognize those who excel in these competitions, we will start building the World Energy Forecaster Rankings.

We will release the competition problems and the formal registration on 10/14/2016. Please join the interest list HERE to get timely updates about GEFCom2017.

Stay tuned!