IEEE Working Group on Energy Forecasting in Good Hands

At IEEE Power & Energy Society General Meeting 2010 (GM'10), I proposed a future panel session on Practical Aspects of Electric Load Forecasting. The session was approved by the Power Systems Planning and Implementation (PSPI) Committee.

The following year at GM'11, we hosted the session with 6 speakers and a fully packed room of audience. The discussion was quite engaging. We shared ideas, experience, concerns and visions of the future for this field. At the same conference, Shu Fan and I proposed to establish the IEEE Working Group on Energy Forecasting to tackle a long list of challenges in the field. The proposal was approved at the PSPI committee meeting. I became the Chair, and Shu became the Vice Chair. 

After GM'11, we started working with a few other key players on several initiatives, such as GEFCom2012, and a TSG special session on forecasting. We brought in Pierre Pinson to both initiatives. Both turned out to be unbelievably successful. GEFCom2012 was a game-changing event that produced many valuable assets for the energy forecasting community and started the . Our TSG special section collected many high quality and highly cited papers.

At GM'12, I met Hamidreza Zareipour for the first time. Within seconds, we both sensed each other's strong passion in forecasting. Then Hamid became the Secretary of our working group. 

After GM'12, Shu, Hamid, Pierre, and I started another initiative: a tutorial on energy forecasting. We taught the tutorial for the next four consecutive years, from 2013 to 2016. 

During the last 9 years, we have completed virtually every task we promised to do back in 2011, and much more: the series of three Global Energy Forecasting Competitions, two special sections for IEEE Transactions on Smart Grid and one special issue for Power & Energy Magazine, a 25-page literature review, a full-day tutorial, and more than a dozen panel sessions at PES General Meetings.

Since GEFCom2017, I have been thinking about replicating the success beyond the power sector. Fortunately, the International Institute of Forecasters offered a great platform to reach out to the fields of gas, water, environment, and climate science. Long story short, I founded SWEET, IIF Section on Water, Energy, and EnvironmenT. We just had our first meeting at the International Symposium on Forecasting 2019, with 43 talks in 13 sessions. Our next meeting will be at ISF2020 in Rio, Brazil, July 5-8, 2020.

Running SWEET requires a lot of thoughts and efforts. To focus on this new challenge, I decided to take off my hat as the Chair of the Energy Forecasting Working Group. I believed the group also needed a new leader for its next chapter. I found no better successor than Hamid to take over the Chair position. Hamid is a full professor of electrical and computer engineering at the University of Calgary. He started as the Secretary of the group in 2012. After Shu Fan left academia to pursue his career in trading, he stepped up as the Vice Chair. Hamid has been a crucial contributor to the success of the group since its infancy. I'm sure that he will bring the group up to the next level.

Hamid and I searched for a new secretary and found a rising star Yi Wang, a postdoc researcher at ETH Zurich. Yi received his PhD from Tsinghua University. He has published more than 30 journal papers, of which most are on load forecasting and smart meter data analytics. In the past, he has helped us review many energy forecasting manuscripts for top scholarly journals. This year at GM'19, Yi is chairing a panel session on probabilistic energy forecasting. Yi has already brought up several innovative ideas to grow the group. As the Past Chair, I'll support Hamid and Yi to ensure a smooth transition.

IEEE Working Group on Energy Forecasting is in good hands!

A Look Back at ISEA2017

About a year ago, I posted 7 reasons to attend ISEA2017. Last June, we had our very first International Symposium on Energy Analytics at the beautiful city Cairns, Australia. 50 researchers and practitioners from industry and academia across 16 countries attended the symposium. 36 of them also attended ISF2017.

The two-day agenda included 5 keynote presentations, 13 contributed presentations selected from 31 abstract submissions, and 5 GEFCom2017 finalists presentations. At the end of the first day, June 22, 2017, we hosted the GEFCom2017 award ceremony followed by a two-hour cocktail reception. The program agenda, presentations and pictures can be downloaded via THIS LINK.

We have been processing the full paper submissions since then. So far, three papers are currently in press with the Journal of Modern Power Systems and Clean Energy, one is in press with the Power & Energy Magazine. The remaining papers are going through the peer review system of the International Journal of Forecasting. The accepted ones together with the GEFCom2017 articles are going to be published in an energy forecasting special section edited by Pierre Pinson and myself.  

I want to thank many people who made ISEA2017 a successful event, the keynote speakers, the presenters, the GEFCom2017 winners, and all the attendees of ISEA2017. Special thanks to Rob Hyndman and George Athanasopoulos for the strong local support, our photographer Mitchell O'Hara-Wild for capturing the happy moments, and Pam Stroud for handling the registrations and helping keep things in order. 

Last but not least, big thanks to our sponsors, International Institute of Forecasters, Tangent Works and Journal of Modern Power Systems and Clean Energy, for their generous financial support that helped to keep the registration fees low. 

Many people have asked me about the next ISEA. I know there will be the next one, but I don't know when and where. If you have any idea or want to help with organizing ISEA, let me know! 

IEEE PES Announces Winning Teams for Global Energy Forecasting Competition 2017

More than 300 students and professionals from more than 30 countries formed 177 teams to compete on hierarchical probabilistic load forecasting, exploring opportunities from the big data world and tackling the analytical challenges.

PISCATAWAY, N.J., USA, August 18, 2017 – IEEE, the world's largest professional organization advancing technology for humanity, today announced the results of the Global Energy Forecasting Competition 2017 (GEFCom2017), which was organized and supported by the IEEE Power & Energy Society (IEEE PES) and the IEEE Working Group on Energy Forecasting (WGEF).

Mark Your 2017 Calendar: Tao's Recommended Conferences for Energy Forecasters

I didn't realize the overdue of this post until I just hit the road for my first trip of 2017. Here is the 2017 list of my recommended conferences for energy forecasters:

1. International Symposium on Energy Analytics (ISEA2017, Cairns, Australia, June 22-23, 2017)

Even if you missed all the other events down this list, you can still find the year rewarding by attending ISEA2017, the first-ever gathering of world-wide energy forecasters. Our generous sponsors, the International Institute of Forecasters (Super Sponsor), Tangent Works (Gigawatt Sponsor) and the State Grid Electric Power Research Institute (Kilowatt Sponsor), have helped bring the registration fees down. There are many reasons to join the party. You will meet the winners of GEFCom2017. You will hear the presentations from world-class energy forecasting researchers and practitioners. You will network with energy forecasting colleagues from more than a dozen countries. And of course, you will enjoy two World Heritage sites side-by-side.

2. Tao's courses

The next two SAS courses on load forecasting have been scheduled in Charlotte, March 27-29.

• Electric Load Forecasting: Fundamentals and Best Practices
• Electric Load Forecasting: Advanced Topics and Case Studies

In addition, I'm going to teach these three courses through EUCI:

• Fundamentals of Utility Analytics: Techniques, Applications and Case Studies
• Introduction to Energy Forecasting
• Long Term Load Forecasting

Stay tuned with the training page of Hong Analytics for the recent updates of all training courses.

3. Conferences from other professional organizations

I will attend the following three, as always:

• International Symposium on Forecasting (ISF2017, Cairns, Australia, June 25-28, 2017)
• IEEE PES General Meeting (PESGM2017, Chicago, IL, USA, July 16-20, 2017)
• AEIC/WLRA Annual Conference (Time & Location TBD)

Look forward to seeing you in these fantastic events!

2016 Greetings from IEEE Working Group on Energy Forecasting

Another Christmas is coming in few days. It's time to look back at 2016 and see what IEEE Working Group on Energy Forecasting has done:

• The LinkedIn group for IEEE WGEF has grown to 1642 members as of today, up from 1507 same time last year. LinkedIn has split the Global Energy Forecasting Competition group from WGEF. The GEFCom group now has 657 members, up from 485 same time last year. 
• IEEE Transactions on Smart Grid published the special section we edited on Big Data Analytics for Grid Modernization.
• International Journal of Forecasting published the special section Pierre Pinson and I edited on Probabilistic Energy Forecasting.
• Hamidreza Zareipour, Pierre Pinson and I taught the IEEE tutorial Energy Forecasting in the Smart Grid Era at PESGM 2016 in Boston for the fourth and last time. 
• Ning Zhang and Pierre Pinson prganized and chaired a panel session "Offering, Pricing and Managing Uncertainty and Variability in Power System Operations with Significant Renewable Energy Integration". The session room was over-crowded by 100+ people. 
• We kicked off the qualifying match of the Global Energy Forecasting Competition 2017, with 140+ teams from 30 countries. 

Next year will be even more exciting:

• We will hold the International Symposium on Energy Analytics (ISEA2017), the first-ever gathering of world-wide energy forecasters in Cairns, Australia, the only place on earth with two World Heritage sites side-by-side, Great Barrier Reef and the Daintree Rainforest.  
• We will conclude GEFCom2017 at ISEA2017 with the winner presentations and prizes. 
• A PESGM2017 panel session on multiple energy systems is being organized by Ning Zhang and myself. 
• I will be editing a special issue for the Power & Energy Magazine on big data analytics. The papers are by invitation only. If you have any good idea and would like to present it to thousands of PES members through this special issue, please let me know. 
• We didn't have the bandwidth for JREF this year. We will try to conduct the JREF survey next year. 

Happy Holidays and Happy Forecasting!

FAQ for GEFCom2017 Qualifying Match

I have received many questions from GEFCom2017 contestants. Many thanks to those who raised the questions. This is a list of frequently asked questions. I will update it periodically if I get additional ones.

Q1. I can't open the link to the competition data. How to get access to the data?

A1. If you cannot access the data via the provided link directly, you may need a VPN service. There are many free VPN services available. Use Google to find one, or post the question on LinkedIn forum to see if your peer contestants can help.

Q2. Can the competition organizer re-post the data somewhere else?

A2. No. We are not going to re-post the data during the competition, because ISO New England updates the data periodically.

Q3. Are we forecasting the same forecasting period in both Round 2 and Round 3? And another same forecasting period in both Round 4 and Round 5?

A3. For GEFCom2017-D, ISO New England updates the data every month, typically in the first half of the month. In Round 2, you will be using the data as of Nov 30, 2016. In Round 3, the December 2016 data should be available as well. For GEFCom2017-O, the data is being updated in real-time. We would like to see if there is any improvement with half a month of information. This set up also gives some flexibility to the contestants. If the team is busy with other commitments during the competition, they may submit the same forecast for both Round 2 and Round 3.

Q4. Can the same team join both tracks?

A4. Yes. A team may even submit the same forecasts to both tracks. Nevertheless, we are expecting higher accuracy in the forecasts of GEFCom2017-O than those of GEFCom2017-D.

Q5. Can one person join two or more teams?

A5. No.

Q6. I'm with a vendor. I don't know if my company wants to put its name as the team name. Can I join the competition personally? If I win, can I add my company as my affiliation and/or change the team name to my company's name?

A6. You can join the competition with or without linking your team to your company. However, you need to make the decision before registration. Once you are in the game, we can not change your affiliation or team name.

Q7. Which benchmark method will be used?

A7. The benchmark method forecasts each zone individually. We will use the vanilla model as the underlying model, simulate the temperature by shifting 11 years of temperature data (2005 - 2015) 4 days forward and backward to come up with 99 scenarios, which will be used to extract 9 quantiles. See THIS PAPER for more details.

Q8. In GEFCom2017-D, are we required to process daylight savings time in a specific way?

A8. No. You can treat the daylight savings time any way you like. THIS POST elaborates my approach, which you don't have to follow.

Q9. In GEFCom2017-D, are we allowed to assume the knowledge of federal holidays before 2011?Can we give special treatments to the days before and after the holidays?

A9. Yes, and yes. The opm.gov website only publishes federal holidays starting from 2011. You can infer the federal holidays before 2011. You can model the days before and after holidays the way you like. I had a holiday effect section in my dissertation, which you don't have to follow. Keep in mind that you should not assume any knowledge about local events or local holidays, such as NBA final games and Saint Patrick's Day.

Q10. The sum of the 8 zones are slightly different from the total demand published by ISO New England. Which number will you use to evaluate the total demand?

A10. Column D of the "ISO NE CA" worksheet.

Q11. For GEFCom2017-D, are you going to provide weather forecasts that every team should use?

A11. No. It is an ex ante hierarchical probabilistic load forecasting problem. We do not provide weather forecasts. The contestants in the GEFCom2017-D track should not use any weather forecasts from other data sources. Nevertheless, the contestants may generate their own weather forecast if they want to. The weather forecasting methodology should be in the final report if they take this route.

Q12. No wind, solar or price forecasting in GEFCom2017? It's a pity!

A12. GEFCom2017 is a load forecasting competition. Unfortunately, we were not able to identify good datasets to set up wind, solar or price forecasting tracks to match the challenge level as this load forecasting problem. Nevertheless, in GEFCom2017-O, you may leverage other data sources to predict wind, solar and prices, which may be good for your load forecasts.

Q13. I'm a professor. Any advice if I want to leverage this competition in class?

A13. It would be nice to leverage the competition in your course. I did so two years ago in GEFCom2014. There will again be an institute prize in GEFCom2017. To aim for the institute prize, I would recommend that you sign up as many teams as possible to maximize the likelihood to win. What I did two years ago was to have each student form a single-person team, and tied the competition ranking to their grades. Anyway, if you are going to join the competition, it's better to have the students look into the data ASAP. The first round submission is due on 12/15/2016.

Q14. Any reference materials we should read before we dive into the competition problem?

A14. For probabilistic load forecasting, you should at least read this recent IJF review paper on probabilistic load forecasting and the relevant references. You can find my recent papers on probabilistic load forecasting HERE. The papers from winning entries of GEFCom2014 are HERE. For hierarchical forecasting, you can check out Hyndman and Athanasopoulos' BOOK and their PAPER. 

Instructions for GEFCom2017 Qualifying Match

The GEFCom2017 Qualifying Match means to attract and educate a large number of contestants with diverse background, and to prepare them for the final match. It includes two tracks: a defined-data track (GEFCom2017-D) and an open-data track (GEFCom2017-O). In both tracks, the contestants are asked to forecast the same thing: zonal and total loads of ISO New England. The only difference between the two tracks is on the input data.

Data 

The input data a participating team can use GEFCom2017-D should not go beyond the following:

1. Columns A, B, D, M and N in the worksheets of "YYYY SMD Hourly Data" files, where YYYY represents the year. These data files can be downloaded from ISO New England website via the zonal information page of the energy, load and demand reports. Contestants outside United States may need a VPN to access the data. 
2. US Federal Holidays as published via US Office of Personnel Management.

The contestants are assumed to have the general knowledge of Daylight Savings Time and inferring the day of week and month of year based on a date.

There is no limitation for the input data in GEFCom2017-O.

Forecasts

The forecasts should be in the form of 9 quantiles following the exact format provided in the template file. The quantiles are the 10th, 20th, ... 90th percentiles. The forecasts should be generated for 10 zones, including the 8 ISO New England zones, the Massachusetts (sum of three zones under Massachusetts), and the total (sum of the first 8 zones).

Timeline

GEFCom2017 Qualifying Match includes six rounds.

Round 1 due date: Dec 15, 2016; forecast period: Jan 1-31, 2017.
Round 2 due date: Dec 31, 2016; forecast period: Feb 1-28, 2017.
Round 3 due date: Jan 15, 2017; forecast period: Feb 1-28, 2017.
Round 4 due date: Jan 31, 2017; forecast period: Mar 1-31, 2017.
Round 5 due date: Feb 14, 2017; forecast period: Mar 1-31, 2017.
Round 6 due date: Feb 28, 2017; forecast period: Apr 1-30, 2017.
Report and code due date: Mar 10, 2017.

The deadline for each round is 11:59pm EST of the corresponding due date.

Submission

The submissions will be through email. Within two weeks of registration, the team leader should receive a confirmation email with the track name and team name in the email subject line. If the team registered both tracks, the team leader should receive two separate emails, one for each track.

The team lead should submit the forecast on behalf of the team by replying to the confirmation email.

The submission must be received before the deadline (based on the receipt time of the email system) to be counted in the leaderboard.

Template

The submissions should strictly follow the requirements below:

1. The file format should be *.xls;
2. The file name should be "TrackInitialRoundNumber-TeamName". For instance, Team "An Awesome Win" in the defined data track's round 3 should name the file as "D3-An Awesome Win".
3. The file should include 10 worksheets, named as CT, ME, NEMASSBOST, NH, RI, SEMASS, VT, WCMASS, MASS, TOTAL. Please arrange the worksheets in the same order as listed above. 
4. In each worksheet, the first two columns should be date and hour, respectively, in chronological order.
5. The 3rdto the 11th columns should be Q10, Q20, ... to Q90. 

The template is HERE. The contestants should replace the date column to reflect the forecast period in each round.

Evaluation

In round i, for a forecast submitted by team j for zone k, the average Pinball Loss of the 9 quantiles will be used as the quantile score of the probabilistic forecast Sijk. A benchmark method will be used to forecast each of the 10 zones. We denote the quantile score of the benchmark method in round i for zone k as Bik.

In round i, we will calculate the relative improvement (1 - Sijk/Bik) for each zone. The average improvement over all zones team j accomplishes will be the rating for team j, denoted as Rij. The rank of team j in round i is RANKij.

The weighted average of the rankings from all 6 rounds will be used to rank the teams in the qualifying match leaderboard. The first 5 rounds will be weighted equally, while the weight for the 6th round is doubled.

A team completing four or more rounds is eligible to for the prizes. The ratings for the missing rounds will be imputed before calculating the weighted average of the ratings.

Prizes

Institute Prize (up to 3 universities): $1000
1st place in each track: $2000
2nd place in each track: $1000
3rd place in each track: $500
1st place in each round of each track: $200

For more information about GEFCom2017, please visit www.gefcom.org.

GEFCom2017: Hierarchical Probabilistic Load Forecasting

IEEE Working Group on Energy Forecasting invites you to join the Global Energy Forecasting Competition 2017 (GEFCom2017): Hierarchical Probabilistic Load Forecasting.

Background

Emerging technologies, such as microgrids, electric vehicles, rooftop solar panels and intelligent batteries, are challenging the traditional operational practices of the power industry. While uncertainties on the demand side are pushing the operational excellence toward the edge of the grid, probabilistic load forecasting at various levels of the power system hierarchy is becoming increasingly important.

GEFCom2017 will bring together state-of-the-art techniques and methodologies for hierarchical probabilistic energy forecasting. The competition features a bi-level setup: a three-month qualifying match that includes two tracks, and a one-month final match on a large-scale problem.

Qualifying match

The qualifying match means to attract and educate a large number of contestants with diverse background, and to prepare them for the final match. The qualifying match includes two tracks, both on forecasting the zonal and total loads of ISO New England (the "DEMAND" column) for the next month in real-time on rolling basis.

The defined-data track (GEFCom2017-D) restricts the data used by the contestants. The data cannot go beyond the calendar data, load (the "DEMAND" column) and temperature data (the "DryBulb" and "DewPnt" columns) provided by ISO New England via the zonal information page of the energy, load and demand reports,  plus the US Federal Holidays as published via US Office of Personnel Management. The contestants may infer day of week and Federal Holidays based on the aforementioned data.

The open-data track (GEFCom2017-O)encourages the contestants to explore various public and private data sources and bring the necessary data into the load forecasting process. The data may include, but is not limited to the data published by ISO New England, the weather forecast data from any weather service providers, the local economy information, the penetration of solar PV published by US government websites.

Final match

The final match (GEFCom2017-F) will be open to the top entries from the qualifying match, tackling a more challenging, larger scale problem than the qualifying match problems. The final match includes one-track only, forecasting the load of a few hundred delivery points of a U.S. utility. The data is from the real world, so the contestants should expect many data issues, such as load transfers and anomalies. Details of the final match will be released on March 15, 2017.

Submission method

To save competition platform costs and implement more sophisticated evaluation methods, the submission will be via email. Within two weeks of the registration, the contestants will receive an email with the instructions about how to submit the forecasts.

Evaluation 

The "DEMAND" column published by ISO New England will be used to evaluate the skills of the probabilistic forecasts. Note that the "DEMAND" data may be revised during the settlement process. The version at the time of evaluation will be used to score the forecasts.

The evaluation metric is quantile score. For each forecasted period, the quantile score of a submitted forecast will be compared with the quantile score of the benchmark. The relative improvement over the benchmark will be used to rate and rank the teams.

World Energy Forecaster Rankings (WEFR)

Many contestants who joined GEFCom2012 also participated in GEFCom2014. To encourage the continuous investments in energy forecasting and recognize those who excel in these competitions, we will start building the World Energy Forecaster Rankings.

The contestants of GEFCom2017 will be eligible to participate in WEFR. We hope the rankings can help reward the participants with career opportunities and tickets to future competitions. In addition, editors of relevant journals can also leverage WEFR to enhance the peer review process.

Prize

IEEE Power and Energy Society budgeted 20,000 for this competition. The prize pool is $18,000, to be shared among the winning teams and institutions from qualifying match and final match.

Publication

Winning teams will be invited to submit papers to a special issue of the International Journal of Forecasting. 

Registration

The maximum team size is three. The team leader should register on behalf of the team. The registration period is from Oct 14, 2016 to Jan 14, 2017. Please register via THIS LINK if you want to join the competition.

Competition timeline

• Competition Problems Release  --  Oct 14, 2016
• Qualifying Match Starts  --  Dec 1, 2016
• Qualifying Match Ends  --  Feb 28, 2017
• Final Match Data Release  --  Mar 15, 2017
• Final Match Submission Due  --  May 15, 2017

Additional rules

• One email account per person. 
• Team mergers are not allowed. 
• Each team leader should have a profile on LinkedIn and join the LinkedIn Group on Global Energy Forecasting Competition. 
• Private sharing data or code outside teams is not allowed, though it is OK to publicly share information with other teams through the LinkedIn Group on Global Energy Forecasting Competition. 
• A winning team should present the methodology at the International Symposium on Energy Analytics (ISEA2017) to claim the full prize money. 
• The GEFCom2017 host has the right to publicly disseminate any entries or disqualify any teams.
• The GEFCom2017 host reserves the right to publicly change these terms and conditions at any time.

For any questions or comments, please put them in the comment field below. Please link your name to your LinkedIn profile. 

Datasets for Energy Forecasting

Reproducible research is a key to advancing knowledge. In energy forecasting, it is necessary and crucial that researchers compare their models and methods using the same datasets. Five years ago when we founded the IEEE Working Group on Energy Forecasting, "lack of benchmark data pool" was one of the issues we identified. Fortunately, things have been changing toward the right direction over the past few years. More and more datasets are being made available to and recognized by the energy forecasting community.

This post will serve as the starting point of a blog series on datasets. In each post, I will feature a dataset and discuss how to use it. I will also host the datasets on Dropbox and provide the links in these posts. Meanwhile, I would like to take a crowd-sourcing approach to making a comprehensive and widely accessible data pool:

• If you can host the datasets through other channels, please contact me. 
• If you know of some public datasets that are not on my list, please contact me. 
• If you have some private datasets that can be made available to the energy forecasting community, please contact me. 

Here is a list of 9 posts with the publicly available data that I have used in my papers. I will update the list with links and additional data sources, so check this page from time to time to see if there is something you need.

Electric load forecasting

1. GEFCom2012
2. GEFCom2014
3. ISO New England
4. RWE npower forecasting challenge 2015

Gas load forecasting

1. RWE npower forecasting challenge 2015

Electricity price forecasting

1. GEFCom2014

Wind power forecasting

1. GEFCom2012
2. GEFCom2014

Solar power forecasting

1. GEFCom2014

Stay tuned...

Mark Your 2016 Calendar: Tao's Recommended Conferences for Energy Forecasters

Last year I made a list of recommended conferences for energy forecasters. I think it is time to refresh the list for my 2016 calendar. Hopefully you will find the list useful as well.

1. Tao's courses

The next two SAS courses on load forecasting have been scheduled in Charlotte, March 9-11.

• Electric Load Forecasting: Fundamentals and Best Practices
• Electric Load Forecasting: Advanced Topics and Case Studies

I'm also going to teach three courses through EUCI:

• Fundamentals of Utility Analytics: Techniques, Applications and Case Studies
• Introduction to Energy Forecasting
• Long Term Load Forecasting

Stay tuned with the training page of Hong Analytics for the recent updates of all training courses.

2. 2016 PES T&D Conference & Exposition (T&D2016, Dallas, TX, May 2-5, 2016)

This is a bi-annual conference gathering "more than 700 companies and thousands of professionals from more than 80 countries around the world". Although the conference does not have much to do with forecasting, the exposition is indeed eye-opening. It is important for an energy forecaster to understand how the power grid is operated. This conference is that one-stop shop to see the new products and service offerings from the major industry players. I've been attending this conference since 2010. This year I'm going to present a paper "from high-resolution data to high-resolution probabilistic load forecasts".

3. 13th International Conference on the European Energy Market (EEM16, Porto, Portugal, June 6-9, 2016)

The conference organizing committee is launching an electricity price forecasting competition. It will be interesting to hear the methodologies used by the winners.

4. 36th International Symposium on Forecasting (ISF2016, Santander, Spain, June 19-22, 2016)

I'm organizing a session "load forecasting: research progress and challenges" at ISF2016. The session includes four talks:
What’s new in load forecasting since 2010? 
    Tao Hong | University of North Carolina at Charlotte, USA
Load forecasting using Lasso based time series methods
    Florian Ziel | European University Viadrina, Germany
Analysis of ex-ante probabilistic load forecasts at the low voltage substation level
    Stephen Haben, Siddharth Arora, Georgios Giasemidis, Tamsin Lee | University of Oxford, United Kingdom
Aggregate consistent forecasting algorithms for hierarchical electricity demand data
    Souhaib Ben Taieb | Monash University, Australia.
    James W. Taylor | University of Oxford, United Kingdom.
    Rob J. Hyndman | Monash University, Australia.

5. IEEE Power and Energy Society General Meeting 2016 (PESGM2016, Boston, MA, July 17-21, 2016)

We will have the 4th offering of our tutorial "energy forecasting in the smart grid era".

6. AEIC/WLRA annual conference (Chicago, IL, September 18-21, 2016)

AEIC Load Research Committee and Western Load Research Association will again have their joint meeting in Chicago.

7. 2016 International Conference on Probabilistic Methods Applied to Power Systems (PMAPS2016, Beijing, China, October 16-20, 2016)

This is also a bi-annual conference. The conference plays a big emphasis on dealing with uncertainties in power systems,  which is highly related to probabilistic forecasting. The full paper is due on March 31.

Happy Valentine's Day, and happy traveling!

2015 Greetings from IEEE Working Group on Energy Forecasting

The holiday season is coming again. As always, I'm pleased to list the accomplishments made by our working group in 2015:

JREF: Journal Rankings in Energy Forecasting (2015)

Publication is a very important activity for researchers. It is often tied to graduation, tenure, promotion, funding, and so forth. As far as I know, there is not yet a journal for energy forecasting.

As an author, I often asked myself,

Which journal shall I send my paper to?

As a reader, I had a similar question,

Which journals shall I read papers from?

Since I want my papers to show up with the other best papers, the interaction between the two questions above becomes

Where are the best energy forecasting papers?

Call for Nominations: IEEE PSPI Prize Paper Award

IEEE Power Systems Planning and Implementation Committee (PSPI) is calling for nominations for various awards. I am serving as the coordinator for the IEEE Working Group on Energy Forecasting to turn in the nominations. Here I would like to encourage you to nominate some energy forecasting papers for the PSPI Prize Paper Award.

Tao Hong: Be Honest

Below is my recent interview with T&D World Magazine. The original version is HERE.

Tao Hong: Be Honest

Tao Hong always sticks to his integrity, especially when it comes to energy forecasting. As graduate program director and EPIC assistant professor at the Systems Engineering and Engineering Management Department at the University of North Carolina at Charlotte, he said the best advice he has ever received is to always be honest.

Sometimes we are pressured to make the forecasts following someone else' personal agenda. Rather than modeling other people's mind and making fraudulent forecasts, we should always stick to our integrity.

Hong will be presenting Energy Forecasting in the Smart Grid Era (blog post) at the 2015 IEEE PES General Meeting, being held July 26-30, in Denver, Colorado. The full-day tutorial covers how wide-range deployment of smart grid technologies enables utilities to monitor the power systems and gather data on a much more granular level than ever before. While the utilities can potentially better understand the customers, design the demand response programs, forecast and control the loads, integrate renewable energy and plan the systems, etc., they are facing analytic issues with making sense and taking advantage of the "big data".

What's New in Energy Forecasting - Jun 2015

It's time for the mid-year report and summary of the exciting events in the energy forecasting community:

1. Global Energy Forecasting Competition 2014

The competition was launched in August 2014 and ended in December 2014. We are now in the stage of post-competition activities, such as organizing conference presentations and paper publications. Many finalists of the competition will gather at the IEEE PES General Meeting 2015. The papers are expected to be published in the IJF special issue on probabilistic energy forecasting in early 2016. To follow the update of this competition and future events, please join this LinkedIn group.

2. Activities at IEEE PES General Meeting

Energy Forecasting Activities at PESGM2015

IEEE Power & Energy Society just released the technical program agenda for PESGM2015. I'm pleased to highlight the two days on energy forecasting organized by our working group.

Please note that the Wednesday afternoon's panel session is in Plaza 3. We also added a reception on Wednesday evening.

Saturday, July 26, 2015

Energy Forecasting in the Smart Grid Era (full-day tutorial)
8:00 AM - 5:00 PM
Instructors: Tao Hong, Shu Fan, Hamidreza Zareipour, and Pierre Pinson

Wednesday, July 29, 2015

Mark Your 2015 Calendar: Tao's Recommended Conferences for Energy Forecasters

Update (3/1/2015): Due to high demand of my load forecasting courses, we added an offering at the New York City in May. 

Update (2/18/2015): I just confirmed two more speakers for ISF2015. Now we have six presentations in two sessions. 

Recently I received many inquiries about recommended energy forecasting conferences in 2015. First of all, I have never attended a conference that is perfectly designed and organized for energy forecasters, which motivates me to organize the ultimate energy forecasting conference. If you have to wait for this ultimate one, close this page and stay tuned for another two years. Otherwise, keep reading. I will provide a list of 6 venues for you to consider, in the chronological order.

1. Tao's load forecasting courses (May 27-29, 2015, New York, NY)

I have taught the fundamental course 15 times. More than 150 energy forecasters have attended the course. (See some statistics based on the first 10 offerings) Recently I have developed a one-day advanced level course for those who want some in depth coverage of the subject and hands on experience of SAS. The links to the courses are listed below:

• Electric Load Forecasting: Fundamentals and Best Practices
• Electric Load Forecasting: Advanced Topics and Case Studies

2. 35th International Symposium on Forecasting (ISF2015, June 21-24, 2015, Riverside, CA)

ISF2015 is a great conference if you want to learn the frontiers of forecasting. I'm organizing an energy forecasting session at ISF2015. The two sessions includes six talks with a balanced mix of state-of-the-art research and practice.

Session Title: Frontiers in Electricity Demand Forecasting I: The State of The Practice
Chair: Tao Hong (University of North Carolina at Charlotte, USA)

• SAS Energy Forecasting: Hourly load forecasting for all horizons
• Bradley Lawson (SAS, USA)
• Combining sister load forecasts
• Tao Hong (University of North Carolina at Charlotte, USA)
• Bidong Liu (University of North Carolina at Charlotte, USA)
• MEFM: An R package for long-term probabilistic forecasting of electricity demand
• Rob J. Hyndman (Monash University, Australia)

Session Title: Frontiers in Electricity Demand Forecasting II: Probabilistic Electric Load Forecasting
Chair: Tao Hong (University of North Carolina at Charlotte, USA)

• Quantile regression algorithms for forecasting uncertainty in electricity smart meters data
• Souhaib Ben Taieb, King Abdullah University of Science and Technology, Saudi Arabia.
• Rob J. Hyndman, Monash University, Australia
• Marc G. Genton, King Abdullah University of Science and Technology, Saudi Arabia.
• Electricity demand interval forecasting with Quantile Regression Averaging
• Jakub Notowarski (Wrocław University of Technology, Poland)
• The myths of residual simulation for probabilistic load forecasting 
• Jingrui Xie (University of North Carolina at Charlotte, USA)

3. 3rd International Conference Energy & Meteorology (ICEM2015, June 22-26, 2015, Denver, CO)

Weather drives electricity demand and wind/solar power generation. The conference has a unique focus on the interdisciplinary field of energy and meteorology. For more information, please read Make a Difference in the Energy & Meteorology World written by the guest blogger and conference chair Alberto Troccoli.

4. Modern Electric Power Systems Conference 2015 (MEPS2015, July 6-9, 2015, Wroclaw, Poland)

If you are in the area of load and price forecasting, you must be familiar with Rafal Weron and his book "Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach" and the recent IJF open access paper "Electricity Price Forecasting: A Review of the State-of-the-art with a Look into the Future". Rafal is based in Wroclaw, Poland. He is also a key player on the organizing committee of MEPS2015. I'm sure he will present some nice forecasting work at the conference.

5. IEEE Power and Energy Society General Meeting 2015 (PESGM2015, July 26-30, 2015, Denver, CO)

We have organized two days of agenda on energy forecasting. A full-day tutorial on "Energy Forecasting in the Smart Grid Era", and a full-day session of GEFCom2014 finalist presentations. I will write another blog post when the schedule is finalized.

6. AEIC/WLRA annual conference (time & location TBD)

This is a joint conference organized by AEIC Load Research Committee and Western Load Research Association. The two groups used to organize conferences separately. Last year was the first time they held a joint conference. I hope they will continue the joint conference this fall.

Mark your calendar and enjoy the trips!

2014 Greetings from IEEE Working Group on Energy Forecasting

Before I even realize it, the IEEE Working Group on Energy Forecasting is over three years old. This is the third annual greeting message since we started the working group in July 2011. Here I'm listing a few major accomplishments and activities in 2014:

• The size of IEEE Working Group on Energy Forecasting on LinkedIn has exceeded 1200. This is now the largest energy forecasting group and one of the largest forecasting groups on LinkedIn. 
• We launched the Global Energy Forecasting Competition 2014, a rolling energy forecasting competition lasting 16 weeks long. The competition attracted hundreds of participants worldwide joining the four tracks, probabilistic load/price/wind/solar forecasting. IEEE PES, the financial sponsor of the competition, made a press release, which was broadcasted by many major media outlets. We have also created a LinkedIn group to facilitate the discussions around the competition. 
• In 2014, we organized two panel sessions (Integrating Energy Forecasts into Utility Planning and Operations) in IEEE T&D conference and one panel session (Load Forecasting: the State of the Practice) in IEEE PES General meeting, The three sessions combined hosted 16 talks from 13 speakers. 
• The IEEE tutorial Energy Forecasting in the Smart Grid Era was offered at IEEE PES General Meeting 2014, and was again very well attended and highly rated. 
• The winning entries of GEFCom2012 have published their work on International Journal of Forecasting (special section on Energy Forecasting) and IEEE Transactions on Smart Grid (special section on Analytics for Energy Forecasting with Applications to Smart Grid). For more information, please read this post. 

The group has already planned its activities in 2015:

• We have scheduled a two half-day sessions on IEEE PES General Meeting 2015 to host presentations of the finalists of Global Energy Forecasting Competition 2014.
• We have been editing a special issue on Probabilistic Energy Forecasting for International Journal of Forecasting. This special issue will collect high quality papers from regular submissions and from winning teams of Global Energy Forecasting Competition 2014. 
• We will be teaching the IEEE tutorial Energy Forecasting in the Smart Grid Era at IEEE PES General Meeting 2015. 

Look forward to seeing you next year! Stay tuned with working group activities by joining our group on LinkedIn.

Happy holidays and happy forecasting!

A Survey on Software Packages for Load Forecasting

I'm preparing a report on load forecasting for National Association of Regulatory Utility Commissioners (NARUC), which will be published by the U.S. Department of Energy as a white paper in 2015. In one of the sections, I'd like to include a wide range of tools and solutions that are being used in practice. Although the case study is on long term load forecasting, I would be happy to include some contents about tools and solutions for short term forecasting as well.

Now I need your help to make this section as valuable as possible. If you are interested in contributing to this white paper, please send me your response by Oct 31, 2014.

Vendors in load forecasting space, please send me a 1-2 page non-commercial description of your product offerings, which should include, but is not limited to:

• A general overview of the company and your load forecasting products
• Pros and cons of the products and information about future enhancements
• Your customer base, i.e., number of customers and major customers
• References to your technical resources, such as journal/conference papers and white papers, etc.

Utility load forecasters, please kindly let me know by either sending me an email (hongtao01 AT gmail DOT com) or replying to this post:

• Your name and affiliation 
• Your main job duties relevant to load forecasting
• Tools and/or solutions you are using and your opinions about them (SAS, EViews, Itron, Matlab, MS Excel, R, or else?).

If you are tired of writing long emails, I can call you to take notes during the interview. I will acknowledge the ones who have provided the materials that are used in the final report. If you don't want your name to appear in a public document, please let me know as well.

What's New in Energy Forecasting - Jun 2014

There are four exciting events going on in the energy forecasting community:

1. Global Energy Forecasting Competition 2014
We will launch GEFCom2014 in August. This competition will feature four tracks: electric load forecasting, electricity price forecasting, wind power forecasting and solar power forecasting. All tracks will be under the theme of probabilistic energy forecasting. The competition will last for 15 weeks with incremental data update every week for rolling forecast. The competition forum is on LinkedIn. For more information, please visit www.gefcom.org.

2. Activities at 2014 IEEE PES General Meeting
IEEE Working Group on Energy Forecasting is hosting a one-day tutorial and a panel session at PESGM2014:

• Energy Forecasting in the Smart Grid Era (tutorial) Sunday, 27 July, 2014 8:00 AM-5:00 PM Chesapeake G
• Load Forecasting: the State of the Practice (panel session) Monday, 28 July, 2014 2:00 PM-5:00 PM National Harbor 4

3. Special Issue on Probabilistic Energy Forecasting | International Journal of Forecasting
Although the abstract submission deadline has passed (refer to Call For Papers), we welcome submissions of high quality work on this topic. In addition, the winners of GEFCom2014 will be invited to submit their methods to this special issue.

4. International Symposium on Forecasting
International Institute of Forecasters is hosting the 34th International Symposium on Forecasting in Rotterdam, The Netherlands, 6/29-7/2, 2014. There are four sessions on energy forecasting: Probabilistic Energy Forecasting; Electricity Demand I; Electricity Demand II; and Oil Prices. The abstracts of the talks are available in the program book. The full conference proceeding with presentations and papers will be available on the conference website. Stay tuned.