After one full year of planning and implementation, I'm pleased to announce that the Global Energy Forecasting Competition 2014 is ON. Please visit CrowdANALYTIX.com to look for the four tracks.
For now, I will defer all the thanks to the end of the competition. (BTW, it will have to be a long thank-you letter, because so many people have devoted so many days and nights to set up this competition.) Instead, I would spend my last sleepless night before the competition to provide a few tips and instructions to the GEFCom2014 contestants:
1. THREE registrations
To participate in this competition, you have to register as a solver (user) on CrowdANALYTIX.com. You will also need to register as a LinkedIn user to join the Global Energy Forecasting Competition LinkedIn Group.
Lastly and most importantly, you have to register through this Google Form so that we have the official record of you as a contestant. And make sure you are using your real name. If you win, IEEE PES will cut the check to the name you put there. The registration deadline is 10/10/2014. If you did not do this official registration by the deadline, we cannot offer you any prize or recognition regardless how high you rank.
2. Forming the TEAM.
You can join as an individual, or as part of a team. Please understand that once you join a track as an individual, you cannot be part of a team for the same track. And you have to make this decision before you join a track. If you decide to be part of a team, let the team leader join the competition first, and invite you to his/her team.
3. It's ROLLING!
This is not a data mining exercise as what we did in GEFCom2012. The competition will be run for 16 weeks as 15 tasks (the first task will be open for 16 days). In every task we are releasing incremental data for the contestants to forecast the next period. The first 3 tasks are designed for contestants to get used to the competition environment, of which the scores are not accounted in the final score. The last 12 tasks/weeks are called "evaluation" period, of which the scores will be accounted in the final score. So try to stay alert every week, especially the last 12.
5. Do NOT cheat.
We have provided all the data you need. Since all these data are real-world data, there is a good chance that someone can guess where they are from and take advantage of that. Keep in mind that we organizers will be examining the code and reports. It is not easy to pass our smoke test.
6. Stay COOL.
When you see yourself too far away from the top 1 entry, stay cool and keep working. You never know whether that top 1 will be disqualified or not at the end. In the load forecasting track of GEFCom2012, we disqualified the top 2 entries. Because the #3 entry did not submit the report or code, we did not recognize that as a winner either. The official #1 was actually #4 on the original leaderboard.
When you see yourself having a good leading position, stay cool and keep working. You never know who else will bypass you in the next day or so. Remember ALL forecasts are wrong, and all forecasts can be improved. You should always try to improve your forecasts in this competition.
7. Do NOT try to play with the rules.
The rule settings are not perfect, and we know that. So we own the final right to interpret the rules and to decide who are the winners. For instance, during the evaluation period, you are allowed to submit 1 entry per day. This means that you can submit up to 7 entries per week/task. This rule was designed to offer additional chances for those who mistakenly submit some bad files. However, Someone may try to take advantage of this rule and do as many submissions as possible to try to get a low score. When evaluating all the teams at the end, we will discount the achievement of those who have too many submissions during the evaluation period.
8. Document your methodology EVERYDAY.
The evaluation period ends on 12/6/2014. The reports and codes are due on 12/15/2014. Do NOT try to put all the documentation work at the end. You won't be able to write a nice report if there had been no log in the past 16 weeks. This is a lesson learned by many contestants in GEFCom2012. So my advice is to document your methodology EVERYDAY. Then your last week's work is simply to put these logs together and send them over to us. Documentation is an important part of the forecasting process in the real world. A good forecaster has to be good at documentation.
Enjoy the game!
For now, I will defer all the thanks to the end of the competition. (BTW, it will have to be a long thank-you letter, because so many people have devoted so many days and nights to set up this competition.) Instead, I would spend my last sleepless night before the competition to provide a few tips and instructions to the GEFCom2014 contestants:
1. THREE registrations
To participate in this competition, you have to register as a solver (user) on CrowdANALYTIX.com. You will also need to register as a LinkedIn user to join the Global Energy Forecasting Competition LinkedIn Group.
Lastly and most importantly, you have to register through this Google Form so that we have the official record of you as a contestant. And make sure you are using your real name. If you win, IEEE PES will cut the check to the name you put there. The registration deadline is 10/10/2014. If you did not do this official registration by the deadline, we cannot offer you any prize or recognition regardless how high you rank.
2. Forming the TEAM.
You can join as an individual, or as part of a team. Please understand that once you join a track as an individual, you cannot be part of a team for the same track. And you have to make this decision before you join a track. If you decide to be part of a team, let the team leader join the competition first, and invite you to his/her team.
3. It's ROLLING!
This is not a data mining exercise as what we did in GEFCom2012. The competition will be run for 16 weeks as 15 tasks (the first task will be open for 16 days). In every task we are releasing incremental data for the contestants to forecast the next period. The first 3 tasks are designed for contestants to get used to the competition environment, of which the scores are not accounted in the final score. The last 12 tasks/weeks are called "evaluation" period, of which the scores will be accounted in the final score. So try to stay alert every week, especially the last 12.
5. Do NOT cheat.
We have provided all the data you need. Since all these data are real-world data, there is a good chance that someone can guess where they are from and take advantage of that. Keep in mind that we organizers will be examining the code and reports. It is not easy to pass our smoke test.
6. Stay COOL.
When you see yourself too far away from the top 1 entry, stay cool and keep working. You never know whether that top 1 will be disqualified or not at the end. In the load forecasting track of GEFCom2012, we disqualified the top 2 entries. Because the #3 entry did not submit the report or code, we did not recognize that as a winner either. The official #1 was actually #4 on the original leaderboard.
When you see yourself having a good leading position, stay cool and keep working. You never know who else will bypass you in the next day or so. Remember ALL forecasts are wrong, and all forecasts can be improved. You should always try to improve your forecasts in this competition.
7. Do NOT try to play with the rules.
The rule settings are not perfect, and we know that. So we own the final right to interpret the rules and to decide who are the winners. For instance, during the evaluation period, you are allowed to submit 1 entry per day. This means that you can submit up to 7 entries per week/task. This rule was designed to offer additional chances for those who mistakenly submit some bad files. However, Someone may try to take advantage of this rule and do as many submissions as possible to try to get a low score. When evaluating all the teams at the end, we will discount the achievement of those who have too many submissions during the evaluation period.
8. Document your methodology EVERYDAY.
The evaluation period ends on 12/6/2014. The reports and codes are due on 12/15/2014. Do NOT try to put all the documentation work at the end. You won't be able to write a nice report if there had been no log in the past 16 weeks. This is a lesson learned by many contestants in GEFCom2012. So my advice is to document your methodology EVERYDAY. Then your last week's work is simply to put these logs together and send them over to us. Documentation is an important part of the forecasting process in the real world. A good forecaster has to be good at documentation.
Enjoy the game!
No comments:
Post a Comment
Note that you may link to your LinkedIn profile if you choose Name/URL option.