Thursday, July 7, 2016

GEFCom2012 Load Forecasting Data

The load forecasting track of GEFCom2012 was about hierarchical load forecasting. We asked the contestants to forecast and backcast (check out THIS POST for the definitions of forecasting and backcasting) the electricity demand for 21 zones, of which the Zone 21 was the sum of the other 20 zones.

Where to download the data?

You can also download an incomplete dataset from Kaggle, which does not have the solution data. The complete data was published as the appendix of our GEFCom2012 paper. If you don't have access to Science Direct, you can downloaded from my Dropbox link HERE. Regardless where you get the data, you should cite this paper to acknowledge the source:
  • Tao Hong, Pierre Pinson and Shu Fan, "Global energy forecasting competition 2012", International Journal of Forecasting, vol.30, no.2, pp 357-363, April-June, 2014. 

What's in the package?

Unzip the file, and navigate to "GEFCOM2012_Data\Load\" folder, you will see 6 files:
  • load_history
  • temperature_history
  • holiday_list
  • load_benchmark
  • load_solution
  • temperature_solution
Our GEFCom2012 paper has introduced the first five datasets but not the last one. The "temperature_solution" dataset includes the temperature data from 2008/6/30 7:00 to 2008/7/7 24:00, while the "load_solution" dataset does not include the load data from 2008/6/30 7:00 to 2008/6/30 24:00.

What's not working?

Before using the data, please understand that
there is no way to restore the exact Kaggle setup for you to make direct comparison on the error score. 
The main reason is that Kaggle pick a random subset of the solution data to calculate the scores for public leaderboard, and the rest for private leaderboard. We do not know which data was used for which leaderboard.

Nevertheless, it was never our intention to let you make comparisons in a Kaggle way. It is because the GEFCom2012 was set up more like a data mining competition than a forecasting competition. The contestants can submit their forecasts many times, while Kaggle was picking the best score. This is not a realistic forecasting process.

How to use the data?

Instead, we encourage you to use these 4.5 years of hourly data without considering the Kaggle setup. You can even keep 4 full calendar years and get rid of the last half a year in your case studies. With four years of data, you can perform one-year ahead ex post forecasting (see my weather station selection paper). You can also perform short term ex post forecasting on rolling basis (see my recency effect paper).

Then the question is whether the accuracy is "good enough". According to Table 3 of our GEFCom2012 paper, the winning teams improved the benchmark by about 30% - see the "test" column, which is the private leaderboard of Kaggle. In other words, if your model is getting about 30% error reduction comparing to the Vanilla benchmark on this dataset, it is a decent model.

Please also understand that this 30% is gained from a forecasting system with many bells and whistles, such as detailed modeling of temperature, and special treatment of holidays. If your research is focus on one components, the error reduction may be much smaller than 30%. You can find a more detailed arguments in my response to the second review comment in THIS POST.

It's been over two years since we publish the GEFCom2012 data. Many researchers have already used it to test their models. You can also replicate the experiment setup in the recently published papers that used this GEFCom2012 data, and compare your results with the results on those papers.

Saturday, July 2, 2016

Datasets for Energy Forecasting

Reproducible research is a key to advancing knowledge. In energy forecasting, it is necessary and crucial that researchers compare their models and methods using the same datasets. Five years ago when we founded the IEEE Working Group on Energy Forecasting, "lack of benchmark data pool" was one of the issues we identified. Fortunately, things have been changing toward the right direction over the past few years. More and more datasets are being made available to and recognized by the energy forecasting community.

This post will serve as the starting point of a blog series on datasets. In each post, I will feature a dataset and discuss how to use it. I will also host the datasets on Dropbox and provide the links in these posts. Meanwhile, I would like to take a crowd-sourcing approach to making a comprehensive and widely accessible data pool:
  • If you can host the datasets through other channels, please contact me. 
  • If you know of some public datasets that are not on my list, please contact me. 
  • If you have some private datasets that can be made available to the energy forecasting community, please contact me. 
Here is a list of 9 posts with the publicly available data that I have used in my papers. I will update the list with links and additional data sources, so check this page from time to time to see if there is something you need.

Electric load forecasting
  1. GEFCom2012
  2. GEFCom2014
  3. ISO New England
  4. RWE npower forecasting challenge 2015
Gas load forecasting
  1. RWE npower forecasting challenge 2015
Electricity price forecasting
  1. GEFCom2014
Wind power forecasting
  1. GEFCom2012
  2. GEFCom2014
Solar power forecasting
  1. GEFCom2014
Stay tuned...

Saturday, May 21, 2016

Call For Papers: 2017 International Symposium on Energy Analytics (ISEA2017)

2017 International Symposium on Energy Analytics
(ISEA2017)
Cairns, Australia, June 22-23, 2017
Predictive Energy Analytics in the Big Data World


Modern information and communication technologies have brought big data to virtually every segment of the energy and utility industries. While predictive analytics is an important and necessary step in the data-driven decision-making process, how to generate better forecasts in the big data world is an emerging issue and challenge to both industry and academia.

This symposium aims at bringing forecasting experts and practitioners together to share experiences and best practices on a wide range of important business problems in the energy industry. Here the energy industry broadly covers utilities, oil, gas and mining industries. The subjects to be forecasted range from supply, demand and price, to asset/system condition and customer count.

The topics of interest include but are not limited to:
Probabilistic energy forecasting
Hierarchical energy forecasting
High-dimensional energy forecasting
High-frequency and high-resolution energy forecasting
Equipment failure prediction
Power systems fault prediction
Automatic outlier detection
Load profiling
Customer segmentation
Customer churn prediction

Selected papers will be published in a special section of the International Journal of Forecasting (IJF). If you are interested in contributing a presentation to this symposium, please submit a one-page extended abstract to both guest editors via email with the subject line “ISEA2017 Abstract Submission”. Authors of selected abstracts will be invited to submit full papers to the IJF.

Important dates
Abstract submission open - November 15, 2016
Abstract submission due - January 15, 2017
Abstract acceptance - February 15, 2017
Paper submission for consideration of journal publication - May 31, 2017
ISEA2017 - June 22-23, 2017
ISF2017 - June 25-28, 2017
First round review completion - August 31, 2017
IJF special section publication - 2018

Guest Editors:
Tao Hong, University of North Carolina at Charlotte, USA (hong@uncc.edu)
Pierre Pinson, Technical University of Denmark, Denmark (ppin@elektro.dtu.dk)

Editor-in-Chief
Rob J Hyndman, Monash University, Australia
International Journal of Forecasting

Thursday, May 19, 2016

Job Openings for Energy Analysts and Forecasters

During the past two years or so, the largest tag in this blog has been "jobs". From February 2013 to February 2015, I posted 56 jobs. Due to the increased demand for energy forecasters, I can no longer respond to the job posting requests in time. Therefore, I have decided to take a new approach to job posting.

You are invited to post your job openings using the Name/URL option in the comments field. Please be brief about your job postings. Rather than pasting the whole job description, I would recommend you just listing the job title, company and a link to the job description or application site. In addition, you can also provide your contact information for the readers to contact you if you are the hiring manager or recruiter. I will moderate the comment field.

I'm posting the BigDEAL recruiting message first as an example.

Happy recruiting and job hunting!

Wednesday, May 18, 2016

My Path to Forecasting

The International Institute of Forecasters posted my profile this week.

How did you become a forecaster?

My path to forecasting was more like a maze than a straight line.

In 2005, I joined North Carolina State University’s Electrical Engineering doctoral program. Halfway through my PhD study, in January 2008, I started working in a consulting firm as an electrical engineer, providing services to the energy and utility industries. My first project was on long term spatial load forecasting – forecasting the 30-year ahead annual peak demand for each 50-acre small area of a US utility. In today’s terminology, it is a hierarchical forecasting problem. Knowing almost nothing about forecasting at that time, I formulated the problem as an optimization problem: minimizing the errors in the historical fit and the discrepancies between the sum of lower level forecasts and the upper level forecast, subject to some constraints on saturation load level and load growth rate, etc. I wrote thousands of lines of code in VBA to solve it. I also developed a user interface in MS Excel for power system planners to override the parameters estimated by the computer and see the results on a map. Finally, the solution was very well received by the customer and then sold to many other customers. I also packaged the work into a thesis when I got my master’s degree in operations research and industrial engineering.

At the end of 2008, I was tasked with forecasting hourly electricity demand for another US utility. It was a competition – if my forecast won, the customer would give us a big contract. While the spatial load forecasting project did not require a rigorous evaluation based on the forecast accuracy, this one did. I knew I couldn’t win without getting some statistical forecasting skills. With the help from my wife, a forecaster working at SAS, we developed a linear model that eventually led to a big consulting contract to develop a short term load forecasting solution. In 2009, I joined the Operations Research PhD program of NC State while developing and delivering that short term load forecasting solution at work. I took some time series forecasting courses from David Dickey, who later joined my doctoral committee. In 2010, I completed the dissertation “Short Term Electric Load Forecasting” and received my PhD in operations research and electrical engineering. That’s when I first considered myself a forecaster.

What did you do after getting your PhD?

I continued working in that consulting firm for another few months. In 2011, I got an offer from SAS to work on some forecasting projects for large retailers. The problem was very challenging and interesting to me – how to forecast millions of products on weekly basis? At that time, smart meters were just being deployed in the US. The data would not be ready for analysis for a year or two. I thought it would be nice to take some time off from the utility industry and learn from other industries that had been dealing with hierarchical time series data for decades. I took the offer and became an analytical consultant at SAS for their retail business unit. In January 2012, the General Manager of SAS’ newly formed utilities business unit recruited me to build the energy forecasting vertical. Then I lead a team to commercialize my doctoral research into the SAS Energy Forecasting solution. After the solution was successfully launched, I headed to the next challenge – the workforce crisis in the energy industry. In August 2013, I came back to academia to become a professor, with the mission of educating the next generation of analysts.

What areas of forecasting interest you?

I’m most interested in energy forecasting, more specifically electricity demand forecasting, an area I’ve been working on since the beginning of my forecasting career. Electricity demand typically comes in with high resolution, long history, strong correlation with weather, and sometimes a hierarchy. We can use the load forecasting problem to demonstrate many forecasting techniques and methodologies. Moreover, the problem is so important because it’s tied to the life quality of billions of people on this planet. In addition to energy forecasting, I also have experience and strong interest in retail forecasting and sports forecasting. Recently, I started working on forecasting problems in the healthcare industry, another fascinating field.

Are you working with companies to improve their forecasting practices?

Yes. I maintain active consulting practices through Hong Analytics. Every year I teach 5 to 10 training courses internationally, and work on a few consulting projects to tackle some problems that are challenging in nature. These consulting projects and interactions with clients have inspired many novel research ideas. We turn these ideas into scholarly papers and teaching materials. Many other companies use our papers to improve their forecasts and forecasting practices.

What’s your proudest accomplishment in forecasting?

I have several accomplishments to be proud of, such as commercializing both my master thesis and doctoral dissertation research into software solutions, founding the IEEE Working Group on Energy Forecasting, and authoring a blog on energy forecasting, Nevertheless, my favorite one is the Global Energy Forecasting Competition. It was a team effort. Thanks to a group of enthusiastic scholars and the sponsorships from IEEE Power and Energy Society and the IIF, we have organized two competitions so far: GEFCom2012 and GEFCom2014. Both competitions attracted hundreds of participants worldwide. In addition to highlighting the winning methodologies, these competitions have made data publicly available, to encourage and enable reproducible research in the energy forecasting community. We are currently planning for the next competition. Stay tuned :)

What do you do in your free time?

Other than the family time and work time, I love blogging the most. I started my blog Energy Forecasting in 2013 after seeing Rob Hyndman’s blog Hyndsight. In 2015, the blog attracted 12,119 users from 2,146 cities across 134 countries. In my normal life as a professor living in the peer review system, I had to constantly fight with anonymous reviewers. Blogging is also an escape for me – nobody can reject my post other than myself!

Sunday, May 1, 2016

Hong Analytics One Year Anniversary: A 60-hour Energy Analytics Curriculum

One year ago, I incorporated Hong Analytics LLC to house my consulting practices. At this anniversary, I would love to review a major milestone that was recently accomplished:
A 60-hour energy analytics curriculum. 
One of the frequently asked questions I have been getting from my clients is
Tao, can you recommend some training courses I should take?
If a SAS user asked me this question, my answer would be easy:
Check out the list of my recommended SAS courses.
While the list was put together two years ago, it can no longer address all the needs from my clients. For instance, some clients want to know more about the applications of analytics in the utility industry; some do not have access to advanced analytics software; some need to develop wind and solar forecasts rather than load forecasts; some are interested in the state-of-the-art load forecasting methodologies.

To bridge the gap, I have developed a 60-hour (or 7.5 days) energy analytics curriculum. The curriculum is made of 5 courses as illustrated below:

A 60-hour Energy Analytics Curriculum
  1. T101/ Fundamentals of Utility Analytics: Techniques, Applications and Case Studies
  2. T201/ Introduction to Energy Forecasting
  3. T301/ Electric Load Forecasting I: Fundamentals and Best Practices
  4. T302/ Long Term Load Forecasting
  5. T401/ Electric Load Forecasting II: Advanced Topics and Case Studies
If you are new to the industry, analytics, or both, you can start with T101. If you are new to energy forecasting, T201 would be a good start. T301 is the flagship course that has accommodated a wide range of audience. If you are a long term load forecaster using MS Excel, you may take T302. If you are looking for the secret sauce, T401 is the level you should reach.

Did I forget to develop a master level course? No. Nobody can become a master in 60 hours. One may be able to talk like an expert after completing this 60-hour curriculum. To reach the master level, one has to spend 10,000 hours on the subject. Of course, the BigDEAL would be the #1 choice for energy forecasters!

The next offering of Fundamentals of Utility Analytics has been scheduled in Chicago, IL, August 10-11, 2016. Look forward to seeing some of you over there!

Friday, April 22, 2016

BigDEAL Students Receiving Promotions

As a professor, I find nothing better than hearing the success stories of my students. Currently I have two PhD students, Jingrui (Rain) Xie and Jon Black. Both of them are also working full time in the industry. This is the season of promotion announcements in many companies. Rain was promoted from Sr. Associate Research Statistician Developer to Research Statistician Developer, while Jon was promoted from Lead Engineer to Manager. Here I'm very pleased to feature their short biographies with the new business titles. For more details about their profiles, please check out the BigDEAL current students page.

Congratulations, Rain and Jon, for the well-deserved promotions!


Jingrui Xie
Jingrui (Rain) Xie, Research Statistician Developer, Forecasting R&D, SAS Institute Inc.
Jingrui (Rain) is pursuing her Ph.D. degree at UNC Charlotte where her research focuses on probabilistic load forecasting. Meanwhile, she also works full-time as a Research Statistician Developer at SAS Forecasting R&D. At SAS, she works on the development of SAS forecasting components and solutions, and leads the energy forecasting research. Prior to joining SAS Forecasting R&D, Rain was an analytical consultant at SAS with expertise in statistical analysis and forecasting especially on energy forecasting. She was the lead statistician developer for SAS Energy Forecasting solution and delivered consulting services to several utilities on load forecasting for their system operations, planning and energy trading.
Rain has extensive experience in energy forecasting including exploratory data analysis, selection of weather stations, outlier detection and data cleansing, hierarchical load forecasting, model evaluation and selection, forecast combination, weather normalization and probabilistic load forecasting. She also has extensive knowledge and working experience with a broad set of SAS products.

Jonathan D. Black
Jonathan D. Black, Manager of Load Forecasting, System Planning, ISO New England Inc.
Jon is currently Manager of Load Forecasting at ISO New England, where he provides technical direction for energy analytics and both short-term and long-term forecasting of load, distributed photovoltaic (PV) resources, and energy efficiency. For the past three years he has led ISO-NE’s long-term PV forecasting for the six New England states based on a variety of state policy support mechanisms, and provided technical guidance for the modeling of PV in system planning studies. Jon is directing ISO-NE’s efforts to develop enhanced short-term load forecast tools that incorporate the effects of behind-the-meter distributed PV, and has developed methods of estimating distributed PV fleet production profiles using limited historical data, as well as simulating high penetration PV scenarios to identify future net load characteristics. Jon participates in industry-leading research on forecasting and integrating large-scale renewable energy resources, and has served as a Technical Review Committee member on several multi-year Department of Energy studies. Upon joining ISO-NE in 2010, Jon assisted with the New England Wind Integration Study and the design of wind plant data requirements for centralized wind power forecasting.
Mr. Black is currently a PhD student researching advanced forecasting techniques within the Infrastructure and Environmental Systems program at the University of North Carolina at Charlotte. He received his MS degree in Mechanical Engineering from the University of Massachusetts at Amherst, where his research at the UMass Wind Energy Center explored the effects of varying weather on regional electricity demand and renewable resource availability. He is an active member of both the Institute of Electrical and Electronics Engineers (IEEE) and the Utility Variable Generation Integration Group (UVIG).

Tuesday, April 19, 2016

Improving Gas Load Forecasts with Big Data

This is my first gas load forecasting paper. We introduce the methodology, models and lessons learned from the 2015 RWE npower gas load forecasting competition, where the BigDEAL team ranked Top 3. The core idea is to leverage comprehensive weather information to improve gas load forecasting accuracy.

Citation
Jingrui Xie and Tao Hong, "Improving gas load forecasts with big data". Natural Gas & Electricity, vol. 32, no. 10, pp 25–30, 2016. doi:10.1002/gas.21905 (working paper available HERE)

Improving Gas Load Forecasts with Big Data

Jingrui Xie and Tao Hong

Abstract

The recent advancement in computing, networking, and sensor technologies has brought a massive amount of data to the business world. Many industries are taking advantage of the big data along with the modern information technologies to make informed decisions, such as managing smart cities, predicting crime activities, optimizing medicine based on genetic defects, detecting financial frauds, and personalizing marketing campaigns. According to Google Trends, the public interest in big data now is 10 times higher than it was five years ago (Exhibit 1). In this article, we will discuss gas load forecasting in the big data world. The 2015 RWE npower gas load forecasting challenge will be used as the case study to introduce how to leverage comprehensive weather information for daily gas load forecasting. We will also extend the discussion by articulating several other big data approaches to forecast accuracy improvement. Finally, we will discuss a crowdsourcing, competition-based approach to generating new ideas and methodologies for gas load forecasting.

Thursday, April 14, 2016

IJF Special Section on Probabilistic Energy Forecasting: GEFCom2014 Papers and More

As of this week, 21 of the 22 papers for the IJF Special Section on Probabilistic Energy Forecasting are on ScienceDirect (link to the CFP). Many thanks to the GEFCom2014 organizers, participants, and the expert reviewers, whose time and effort warranted an exceptionally high quality collection of energy forecasting papers. Although these papers are not yet pagerized, I can't wait to compile and post this list.

Editorial and GEFCom2014 Introduction Article

Review Article

Research Articles (Non-GEFCom2014)

Research Articles (GEFCom2014)
Enjoy reading and stay tuned for the next GEFCom!

Wednesday, April 13, 2016

Announcing BFCom2016s Winners

The Spring 2016 BigDEAL Forecasting Competition (BFCom2016s) just ended last week. I received 49 registrations from 15 countries, of which 18 teams from 6 countries completed all four rounds of the competition. I want to give my special appreciation to Prof. Chongqing Kang and his teaching assistant Mr. Yi Wang. They  organized 8 teams formulated by students from Tsinghua University, an institute prize winner of GEFCom2014. Two of the Tsinghua Teams were finally ranked among the Top 6.

The topic of BFCom2016s is ex ante short term load forecasting. I provided 4 years of historical load and temperature data, asking the contestants to forecast the next three months given historical day-ahead temperature forecasts. Three months of incremental data was released in each round.

The benchmark is made by the Vanilla model, the same as the one used in GEFCom2012. This time among the top 6 teams, five were able to beat the benchmark on average ranking, while four beat the benchmark on average MAPE. The detailed rankings and MAPEs of all teams are listed HERE.

I invited each of the top 6 teams to send me a piece of guest blog to describe their methodology. Their contributions (with my minor editorial changes) are listed below, together with the Vanilla Benchmark, which ranked No. 7.

No.1: Jingrui Xie (avg. ranking: 1.25; avg. MAPE: 5.38%)
Team member: Jingrui Xie
Affiliation: University of North Carolina at Charlotte, USA
The same model selection process was used in all four rounds. The implementation was in SAS. The model selection process follows the point forecasting model selection process implemented in Xie and Hong, IJF-2016. In this competition, the forecasting problem was dissected into three sub-problems with each of them having slightly different candidate models being evaluated.
The first sub-problem was a very-short term load forecasting problem, which considered forecasting the first day of the forecast period. The model selection process started with the "Vanilla model plus the lagged load of the previous 24th hour". It then considered the recency effect, the weekend effect, the holiday effect, the two-stage model, and the combination of forecasts as introduced in Hong, 2010 and Xie and Hong, IJF-2016.
The second sub-problem was a short term load forecasting problem, which considered forecasting the second to the seventh day of the month. The model selection process was the same to that for the very-short term load forecasting problem except that the starting benchmark model is the Vanilla model.
The third sub-problem can be categorized as a middle term load forecasting problem in which the rest of the forecast period were forecasted. The model selection process also started with the Vanilla model, but it only considered the recency effect, the weekend effect, and the holiday effect.

No.2: SMHC (avg. ranking: 3.75; avg. MAPE: 5.90%)
Team members: Zejing Wang; Qi Zeng; Weiqian Cai
Affiliation: Tsinghua University, China
We tried the support vector machine (SVM) and artificial neural networks (ANN) models in the model selection stage. We found that the ANN model had a better performance than SVM. When considering the cumulative effect, we introduced the aggregated temperatures of several hours as augmented variables, while and the number of hours was also determined in the model selection process.
In the first round, we used all the provided data for training but didn't consider the influence of holidays. Then in the next three rounds, we divided the provided data into two seasons, “summer” and “winter”. We separately forecasted the load of normal days and special holidays. These so-called seasons are not the traditional ones but were roughly defined by the plot of the average load of the given four years. Then we used the data from each seasons for training to forecast the corresponding season in 2014. This ultimately achieved a higher accuracy. All the aforementioned results and algorithms were implemented by using the MATLAB and C language.

No. 3: eps (avg. ranking: 5.25; avg. MAPE: 6.08%)
Team member: Ilias Dimoulkas
Affiliation: KTH Royal Institute of Technology, Sweden
I used the Matlab’s Neural Network toolbox for the modeling. The evolution of my model during the four rounds was as follows.
1st round: I used the “Fiiting app” which is suitable for function approximation. The training vector was IN =  [Hour Temperature] and the target vector OUT = [Load]
2nd round: I used the “Time series app” which is suitable for time series and dynamical systems. I used the Nonlinear Input-Output model instead of the Nonlinear Autoregressive with External Input model because it performs better for long term forecasting. The training vector was still IN =  [Hour Temperature] and the target vector OUT = [Load]. The number of the delays I found it works better is 5 (= 5 hourly lags).
3rd round. I used the same model but I changed the training vector to IN = [Month Weekday Hour Temperature AverageDailyTemperature MaxDailyTemperature] where AverageDailyTemperature is the average temperature and MaxDailyTemperature is the maximum temperature of the day that the specific hour belongs to.
4th round: I used two similar models with different training vectors. The final output was the average of the two models. The training vectors where IN1 = [Month Weekday Hour Temperature MovingAverageTemperature24 MovingMaxTemperature24] and IN2 = [Month Weekday Hour Temperature AverageTemperaturePreAfter4Hours MovingAverageTemperature24 MovingAverageTemperature5 MovingMaxTemperature24] where MovingAverageTemperature24 is the average temperature of the last 24 hours, MovingAverageTemperature5 is the average temperature of the last 5 hours, MovingMaxTemperature24 is the maximum temperature of the last 24 hours and AverageTemperaturePreAfter4Hours is the average temperature of the hours ranging from 4 hours before till 4 hours after the specific hour.

No. 4: Fortune Teller (avg. ranking: 6.25; avg. MAPE: 6.45%)
Member: Guangzheng Xing; Zetian Zheng; Liangzhou Wang
Affiliation: Tsinghua University, China
Round 1. Variables:Hour, Weekday, T_act, TH(the highest temperature in a day), TM(the mean temperature), TL(the lowest temperature). First of all, we used the MLR, fitting the mean load by TM, TM^2, TM^3. This method didn’t work well, the MAPE could reach about 14%. Then we used neural network, the data set contains the six variables above, and the target value is the Load_MW. The result is better, but because of improper parameters, the model was kind of overfitted, and we didn’t do the cross-validation. The result was not so good.
Round 2. We changed the parameter, and used the max value/min value/ mean value of the previous 24 hours rather than those of the day. The result was much better.
Round 3. We tried to use SVM to classify the two kinds of day curve, and then used the nnet separately. But this method did not seem to be effective. Then we used the SVM to do regression, the data set is same in nnet. Using the test set, the results of SVM and nnet were similar, so we submitted the mean value of both methods’ result.
Round 4: The MAPE of both methods reach over 7% during model selection, the result of SVM was worse, so we only submitted the result of nnet.

No. 5: Keith Bishop (avg. ranking: 6.50; avg. MAPE: 6.47%)
Team member: Keith Bishop
Affiliation: University of North Carolina-Charlotte, USA; Hepta Control Systems, USA
For my forecast, I utilized SkyFoundry’s SkySpark analytics software.  SkySpark is designed for modelling complex building systems and working with the time-series data on a wide range of levels. To support my model, I extended the inherent functionality of this software to support polynomial regression.  My model itself went through several iterations.  The first of these was fairly similar to Dr. Hong’s Vanilla Model with the exception that instead of clustering by month, I clustered based on whether the date was a heating or cooling date.  The heating or cooling determination was made by fitting a third-degree polynomial curve to each, hourly clustered, load-temperature scatter plot, solving for the minimums and then calculating the change-over point by averaging these hourly values.  If the average temperature for a day was above this point, it was a cooling day and vice-versa.  As my model progressed, I incorporated monthly clustering and the recency effect discussed in Electric load forecasting with recency effect: A big data approach.  With the recency effect, I optimized the number of lag hours for each monthly cluster by creating models for each of the past 24-hours and selecting the one with the lowest error.  In the end, I was able to reduce the MAPE of the forecast against the known data from 8.51% down to 5.01%.

No. 6: DUFEGO (avg. ranking: 7.25; avg. MAPE: 6.39%)
Team members: Lei Yang; Lanjiao Gong; Yating Su
Affiliation: Dongbei University of Finance and Economics, China
During the 4-round competition,we selected MATLAB as our tool. We use multiple linear regression models (MLR), each of which has 291 variables including trend, polynominal terms,interaction terms and recency effect. We just used all past historical data without cleansing the data. Considering the forecasting task is to improve predicting accuracy rather than the goodness of fit, we seperated the data into training set and validation set. We used cross validation and out of sample test method to select variables to give our model more generalizaton ability.
In Round 1, we trained one MLR model using the entire historical data. In Round 2, we roughly grouped the historical data by season (such as January - March and April - June,) and trained four MLR models, which improved the results significantly. We also found the distinct relationship between temperature and load in different temporal dimensions.We did some work about selecting the best MLR model in different temporal dimensions and found seasonal separate better. We made a mistake in Round 3 that resulted in a very high MAPE.

No. 7: Vanilla Benchmark (avg. ranking: 7.25; avg. MAPE: 6.42%)
The model is the same as the one used in GEFCom2012. See Hong, Pinson and Fan, IJF2014 for more details. All available historical data in each round was used to estimate the model.

Finally, congratulations to these top 6 teams of BFCom2016s, and many thanks to all of you who participated and are interested in BFCom2016s!