Lara Kretschmer: Captain

Today (April 14, 2025), Lara Kretschmer defended her master thesis Value-added Analysis of Electricity Price Forecasts via Battery-based Energy Arbitrage.

Lara received her bachelor's degree in mathematics and a graduate certificate in data science and business analytics from UNC Charlotte in May 2024. Right after that, she came to our department to pursue the master of science degree in engineering management. 

Lara Kretschmer's master thesis defense

From left to right: Dr. Tao Hong, Lara Kretschmer, Dr. Churlzu Lim. (on zoom: Dr. Lin Ma)

She set a record by finishing our MSEM program in just one year, with a thesis. 

She set this record while playing NCAA Division I volleyball!

Lara is a student athlete, born and raised in Germany. In 2020, she was recruited to UNC Charlotte on the athletics scholarship to play volleyball. In the 2024-25 season, also her 5th and last season at Charlotte, she served as a co-captain of our volleyball team. She was also named to the College Sports Communicators Academic All-District Team.

From her weekly reports, I can see many weeks of 70+ hours. Every week, she spent countless hours in the gym, for workouts, practices, and games.  Sometimes she had to travel to other states to play, adding another 10 to 20 hours on the road. On the academic side, I told all my BigDEAL students to take the most challenging courses on campus. Lara is of no exception. 

Lara is a straight-A student. 

I must be the luckiest professor in the world. I don't know what else I should dream of. 

Well, I wish she can continue pursuing her PhD, but Lara wanted to play volleyball. She plans to join a semi-pro volleyball team in Germany, while taking a day-time job in the industry.

Update 10/3/2025:

After graduation, Lara went back to Germany. She is a forecaster and demand planner at Delta Dore Rademacher, while she continues playing her favorite sports, volleyball, at a local semi-pro team. 

Lara visited me today with a hard-copy of her thesis. 

Congratulations, Dr. Li!

Today (Oct 6, 2022) Yike Li defended his doctoral dissertation on short-term ex ante load forecasting. 

Yike joined our MSEM program in Fall 2018. He completed his master thesis under my supervision in Spring 2020. After that, he continued working with me to pursue his PhD.

The COVID-19 pandemic did not slow him down at all. Instead, the lockdown days may have speeded up his research. Towards the end of his PhD journey, I was frequently surprised by the amount of work he has done within a short timeframe between our meetings. 

From January 2021 to May 2022, Yike worked at Duke Energy as a Data Scientist intern, where he built predictive models to understand consumer electricity usage. After that, he spent a summer with Cruise. While Yike received many offers during the past few months, he decided to bring his talent back to Cruise after graduation. 

While the load forecasting literature is dominated by ex post forecasting studies, many practical issues with ex ante forecasting have never been thoroughly studied. Yike's dissertation tackled some of the most difficult problems in ex ante load forecasting. I'm glad to see that he is applying these skills in the transportation sector. 

Thanks to Yike, I can tell my department chair: "My student makes more than your Dean!"

Congratulations, Dr. Yike Li!

Jordan McCorey - MVP

Last Thursday (April 15th, 2021), Jordan McCorey defended his master thesis Forecasting Most Valuable Players of the National Basketball Association.

Jordan McCorey received his B.S. in Mechanical Engineering from NC A&T University in 2017. Right after graduation, he started his full time job at Boeing in North Charleston as a process engineer. He joined our Master of Science in Engineering Management as a part-time student in Fall 2018. 

Jordan McCorey's master thesis defense
From left to right: Dr. Tao Hong, Dr. Linquan Bai, Jordan McCorey, and Dr. Pu Wang

MVP, a.k.a. Most Valuable Player, is the highest individual award for the most performing player in the entire league. If I were asked to name the MVP among all the graduate students in our program during this pandemic year, Jordan McCorey would be the one.

I have always been interested in sports, basketball in particular. After ISF2019, I sent an email to our graduate students list with a few project topics. One topic was sports analytics - NBA forecasting. Jordan responded to my email with a passionate cover letter expressing his strong interest.

We quickly set up a phone call to discuss a plan to move forward. During the phone call, I was very pleasant to know that Jordan was a varsity basketball player in high school. His understanding of the game was definitely a big plus for this topic. On the other hand, I also got to know that he has limited experience in programming, statistics, and forecasting.

I explained the challenge to him. Apparently he didn't back off. Then I asked him to take my forecasting course, which is known as one of the most demanding courses on campus. The COVID-19 pandemic hit us right in the middle of Spring 2020 semester. Many students took the easy route by taking a passing grade. Jordan, however, worked extra hard to earn a solid A while working on his full-time job at Boeing. 

Due to the quarantine, I was never able to meet Jordan in person. Instead, we had many phone calls to discuss his plan of study, research progress, and of course, our shared passion about the game of basketball. 

A few weeks before his defense date, I got a call from Jordan telling me that he just had Achilles injury. That's the same injury that led to Kobe's retirement, and the same injury that took down Kevin Durant during Game 5 of 2019 final. I asked him if he wanted to postpone the defense. He said no.

Then it came the defense date. 

A fabulous presentation Jordan delivered. 

I was super impressed, so were the other two committee members. 

Jordan McCorey, MVP of the 2020-2021 academic year. 

Deeksha Dharmapal - Well-rounded

Today (4/7/2020), Deeksha Dharmapal defended her master thesis on Gross Domestic Product for short-term load forecasting.

Deeksha received her B.S. in Civil Engineering from Bangalore Institute of Technology, India, in 2012. Prior to joining UNC Charlotte in Fall 2018, she had several years of industry experience in India, with Amazon and EMC2 (now DellEMC). During her graduate study at our program, she also completed a summer internship at Bosch Rexroth, Charlotte.

During the past few years, I gave a departmental seminar to new graduate students every year. Deeksha is definitely the one that left the best impression on me. Most students at the seminar stayed quiet, while Deeksha was one of the few asking questions. Her questions sounded genuine and intelligent. Nevertheless, I didn't think she would eventually join my group, because she appeared to have a bright future on the managerial track.

Among the students in our program, the self-motivated ones start looking for their faculty mentor rather early. Deeksha took an unique approach. She showed up at one of my MS student, Shreyashi Shukla's thesis defense. Later she signed up and completed my forecasting course, which was a surprise to me.

When she told me that she wanted to join my group, I gave her the same task, passing two SAS programmer certification exams. She completed the base one, but failed the advanced one in her first attempt. The outcome didn't surprise me, because I knew her programming background was weak at that time. Most students at this point would just give up and look for other professors. She didn't. Finally she got the SAS Advanced Programmer Certification. Since she didn't pass it within the time limit I assigned, I gave her an extra task, which she completed in time. I brought her to BigDEAL as my MS thesis student in the summer of 2019.

The thesis topic I gave her was non-trivial. Economic indicators are typically used for long term load forecasting but not short term load forecasting. I asked her to investigate what are the situations that we should consider economy, GDP to be specific, in short term load forecasting. This research involves a lot of programming skills as well as knowledge in statistics.  She picked up those things along the way, and had the thesis beautifully done.

As a forecaster, I love to investigate the things that I failed to predict. I was wondering how a little girl Deeksha surprised me multiple times by pushing herself out of her comfort zone and fighting such a tough uphill battle. Through some casual conversations, I learned that she was a student athlete. She ran track for the most part of her student life - short distance sprints and relay. She was on the basketball and volleyball teams. Post marriage, she has been playing competitive badminton.  I guess the sports experience must have built her a strong heart!

Congratulations, Deeksha!

Yike Li - Eager to Learn

Last Friday (2/14/2020), Yike Li defended his MS thesis, Optimal Weather Station Selection for Electric Load Forecasting.

Yike Li's MS thesis defense
From left to right: Dr. Tao Hong, Yike Li, Dr. Pu Wang and Dr. Linquan Bai

Yike received his B.S. degree in Applied Physics from Tianjin University, China, in 2010, and his M.S. degree in Electrical Engineering from North Carolina State University in 2012. He joined our MSEM program in Fall 2018. Meanwhile, he also enrolled in our INES PhD program. From 2012 to 2019, he had a progressive career in the power industry. His was promoted to a consulting manager at Accenture last year, and then decided to come back to school to pursue his PhD.

I got to know Yike since his days at NC State University. I was giving lectures on load forecasting and demand response, when he was one of the students in the class. At that time, he was definitely the student showing most interest in the subject. He was eager to learn, and asking me many questions about the software, models, and applications. Since then, we have been keeping in touch. Occasionally, he sent me greeting messages and updates about his progress in the industry. 

Couple years ago, Yike asked me about pursuing a PhD degree under my supervision. Since he didn't have thesis writing experience, I asked him to complete a master thesis first. Although I've known him for years, I still had him going through the BigDEAL interview process including the screening tests. He passed them without a surprise. His thesis topic is a continuation of my IJF paper on weather station selection. The task was to propose a method beating the one in my IJF paper. It was not an easy task, but he nailed it. He was able to complete the thesis research while working full time. 

Now he can focus on his dissertation research!

Congratulations, Yike!

Zehan Xu - Pursuing Perfection

Yesterday (April 1, 2019), Zehan Xu defended his MS thesis Customer Attrition Modeling and Forecasting.

Zehan Xu's MS thesis defense
From left to right: Dr. Linquan Bai, Zehan Xu, Dr. Tao Hong, and Dr. Shaoyu Li

Zehan received his B.S. degree in Industrial and Systems Engineering from Virginia Tech in 2016. He joined our MSEM program in Fall 2017.

During his first semester, I gave a seminary talk about the research opportunities at BigDEAL. He approached me after that, passed the tests I gave him, and officially joined my research group in February 2018.

Knowing his solid math background, I asked him to work on forecasting customer count using survival analysis. The topic was an extension of Jingrui Xie's MS thesis and TSG paper. Since Zehan did not have much background in statistics, he had to teach himself about survival analysis. He quickly figured out that the tools working well on those textbook examples are not optimal for the real-world datasets I gave him. During the past year, he has been refining his work and finally came up with an effective methodology.

Our original plan was to have him graduate at the end of 2018, when I considered the quality of his work exceed a MS thesis level. Nevertheless, he was never satisfied until very recently.

I stopped by my office last Sunday, and saw one of my student Saurabh Sangamwar in the conference room presenting something. Since Saurabh already defended his thesis a month ago, I was a little curious. I went in and found him and another BigDEAL student Yike Li working with Zehan on Zehan's defense rehearsal.

I thought Zehan's defense preparation was done, but apparently he was pursuing that perfection.

His defense was very well done. I was impressed!

While advising him for the thesis research, I found Zehan a great candidate for doctoral research.  He also realized the need and value of advanced education, so he decided to continue pursuing his doctoral degree here at BigDEAL.

Congratulations, Zehan!

Saurabh Sangamwar - Nothing is Impossible

Yesterday (February 12, 2019),  Saurabh Sangamwar defended his MS thesis Grouping Calendar Variables for Electric Load Forecasting.

Saurabh Sangamwar's MS thesis defense
From left to right: Dr. Liquan Bai, Saurabh Sangamwar, Dr. Pu Wang, Dr. Tao Hong

Saurabh received his B.Eng. in Mechanical Engineering from K J Somaiya College of Engineering, Mumbai, in 2015. After working in India for two years, he joined our MSEM program in Fall 2017. 

I still remember the scene of our first conversation a year ago, when he expressed his interest in joining BigDEAL.

"learn SAS and get the SAS Base Programmer Certification." I told him the same as what I said to the other students.

"I did." Saurabh said. 

"Then go ahead and get the SAS Advance Programmer Certification." I responded. 

"I've done that too." He said. 

Apparently, he came to me so well prepared, and he was the first student I met this well prepared. 

I admitted him without a blink. 

The topic I gave him is about grouping calendar variables. It took him a while to get the preliminary results. Then I asked him to change a few parameters in his algorithms, and refresh the results. I took him another long while to get the second batch done. I saw him working hard everyday, so I was wondering why it took so long to get the results. During the conversation, I realized that his code is not fully automated. In other words, he had to do a lot of manual work to get the results. I also understood that he did have any programming background until last semester, when he was preparing for the SAS certification exams. 

I'm a professor who likes to pull the students out of their comfort zone. Knowing his weakness, I increased the programming requirements in his master thesis research, so that he can sharpen his programming skills. 

Saurabh did not disappoint me. Over the following few months, he automated his code, picked up parallel computing techniques, and even learned additional languages such as Python and R. Moreover, he is one of the few students took two tough courses from me and got a 4.0 GPA. 

To Saurabh, nothing is impossible. 

Congratulations!

Shreyashi Shukla - Determined to Excel

Today (October 23, 2018), Shreyashi Shukla defended her MS thesis Daily Load Forecasting Using Hourly Temperatures.

Shreyashi Shukla's MS thesis defense
From left to right: Dr. Tao Hong, Shreyashi Shukla, Dr. Simon Hsiang, Dr. Churlzu Lim

Shreyashi received her B.Tech. with Honors in Production Engineering & Management from National Institute of Technology, Jamshedpur, in 2006. Before moving to the U.S. with her family, she had a 10-year progressive career in the energy sector in India. She joined our MSEM program in Fall 2017.

Every year I give a department seminar to share with the students about the research projects at BigDEAL. The purpose of these seminars is two-fold. On one hand, these seminars can broaden the students' view about systems engineering and engineering management. On the other hand, I would like to attract the most self-motivated and talented students from the program.

While most students were scared away after seeing how productive the BigDEAL students are, Shreyashi was one of the fearless students who contacted me after the seminar. During our first conversation in October 2017, I explained to her my expectation, and told her about the BigDEAL entrance tests. She took the challenge, passed the tests, and officially joined BigDEAL in Janurary 2018 to conduct her MS thesis research under my supervision.

The research problems BigDEAL students work on are never easy. In addition to tackling the research challenge, Shreyashi had the family duties too. Everyday she spends the morning on campus working on her coursework and research, and the rest of the day with her little daughter at home. She always comes to the lab on time, leaves on time, and works very efficiently. 9 months later, her research turned into a solid MS thesis, which made her the third "mom" student completing MS thesis research at BigDEAL (after Jingrui Xie and Ying Chen). She is also my first Indian student. Next semester, Shreyashi will continue working with me towards her PhD degree.

Congratulations, Shreyashi!

Masoud Sobhani - From Petroleum Engineer to Load Forecaster

Today (November 27, 2017), Masoud Sobhani just defended his MS thesis on data cleansing, the first BigDEAL thesis authored by a non-Chinese student.

Masoud was a petroleum engineer in Iran. He migrated to the U.S. several years ago. He first came to my office in 2015 with inquiries about our MS Engineering Management program, when I was the program director. At that time he could barely speak English. Nevertheless, I admitted him to the program mainly because of his solid academic background and industry experience in the energy sector.

He started the program in Spring 2016 to pursue a non-thesis master degree, planning to graduate in Summer 2017. Due to the challenging nature of my courses (see some student comments HERE), most of the non-thesis master students in our MSEM program try their best to avoid them. Masoud is certainly an exception. He managed to take all my courses during his tenure in the program. At Npower forecasting challenge 2016, Masoud took a top 3 place.

In Spring 2017, he came to me to discuss the possibilities of pursuing a PhD under my supervision. Recognizing him as the top student in the program, I agreed to take him as my doctoral student with the condition that he completes a MS thesis by the end of the year. He took the challenge. From May to November, he passed SAS Advanced Programmer certification exam, identified his thesis topic, designed and implemented a novel data cleansing algorithm, and finished his 10,000-word thesis. The defense was very well done.

Congratulations, Masoud, and best luck with your PhD journey!

Congratulations, Dr. Jingrui Xie!

Today (October 13, 2016), Jingrui (Rain) Xie defended her doctoral dissertation on probabilistic electric load forecasting, which made her the first BigDEAL PhD.

When coming back to academia three years ago, I had the mission of producing the next generation of finest analysts for the industry. As the first PhD from BigDEAL, Rain sets the standard for BigDEAL products and tells what the finest analyst looks like.

Rain joined UNC Charlotte in August, 2013, as my first master student. She received her M.S. degree in Engineering Management in May, 2015, and continued with her PhD in Infrastructure and Environmental Systems.

In just three years, she published 7 journal papers:

• Temperature scenario generation for probabilistic load forecasting (TSG, in press)
• Relative humidity for load forecasting models (TSG, in press)
• On normality assumption in residual simulation for probabilistic load forecasting (TSG, 2016)
• GEFCom2014 probabilistic electric load forecasting: an integrated solution with forecast combination and residual simulation (IJF, 2016)
• Improving gas load forecasts with big data (GAS, 2016)
• Long term retail energy forecasting with consideration of residential customer attrition (TSG, 2015)
• Long term probabilistic load forecasting and normalization with hourly information (TSG, 2014)

and 3 conference papers:

• Comparing two model selection frameworks for probabilistic load forecasting (PMAPS, 2016)
• From high-resolution data to high-resolution probabilistic load forecasts (T&D, 2016)
• Combining load forecasts from independent experts: experience at NPower forecasting challenge 2015 (NAPS, 2015)

She was among the top contestants in all of the forecasting competitions she participated:

• Top1 in BigDEAL Forecasting Competition 2016
• Top 3 in NPower Gas Demand Forecasting Challenge 2015
• Top 3 in NPower Electricity Demand Forecasting Challenge 2015
• Top 3 in Load Forecasting Track of Global Energy Forecasting Competition 2014

She has also received several prestigious awards:

• 2016 IEEE PES Technical Committee Prize Paper Award
• International Symposium on Forecasting 2016 Travel Award
• 2015 IEEE Transactions on Smart Grid Best Reviewer Award
• 2015 Foundation of the Association of Energy Engineers Scholarship
• International Symposium on Forecasting 2015 Travel Award
• 2015 UNCC College of Engineering Outstanding Graduate Research Assistant Award
• 2015 International Institute of Forecasters Student Forecasting Award

Rain has been full-time working at SAS during the past three years. In addition to the academic excellence, Rain received a promotion earlier this year for her outstanding performance at work. 

It took her 21 months to get the PhD - she enrolled in the PhD program in January, 2015, and defended the dissertation today. That said, she just proved the reproducibility of my 20-month PhD!

Lastly, but most importantly, she became a mother two years ago - her daughter is now two-year old. 

Again, congratulations, Dr. Jingrui Xie!

BigDEAL Students Receiving Promotions

As a professor, I find nothing better than hearing the success stories of my students. Currently I have two PhD students, Jingrui (Rain) Xie and Jon Black. Both of them are also working full time in the industry. This is the season of promotion announcements in many companies. Rain was promoted from Sr. Associate Research Statistician Developer to Research Statistician Developer, while Jon was promoted from Lead Engineer to Manager. Here I'm very pleased to feature their short biographies with the new business titles. For more details about their profiles, please check out the BigDEAL current students page.

Congratulations, Rain and Jon, for the well-deserved promotions!

Jingrui Xie

Jingrui (Rain) Xie, Research Statistician Developer, Forecasting R&D, SAS Institute Inc.

Jingrui (Rain) is pursuing her Ph.D. degree at UNC Charlotte where her research focuses on probabilistic load forecasting. Meanwhile, she also works full-time as a Research Statistician Developer at SAS Forecasting R&D. At SAS, she works on the development of SAS forecasting components and solutions, and leads the energy forecasting research. Prior to joining SAS Forecasting R&D, Rain was an analytical consultant at SAS with expertise in statistical analysis and forecasting especially on energy forecasting. She was the lead statistician developer for SAS Energy Forecasting solution and delivered consulting services to several utilities on load forecasting for their system operations, planning and energy trading.
Rain has extensive experience in energy forecasting including exploratory data analysis, selection of weather stations, outlier detection and data cleansing, hierarchical load forecasting, model evaluation and selection, forecast combination, weather normalization and probabilistic load forecasting. She also has extensive knowledge and working experience with a broad set of SAS products.

Jonathan D. Black

Jonathan D. Black, Manager of Load Forecasting, System Planning, ISO New England Inc.

Jon is currently Manager of Load Forecasting at ISO New England, where he provides technical direction for energy analytics and both short-term and long-term forecasting of load, distributed photovoltaic (PV) resources, and energy efficiency. For the past three years he has led ISO-NE’s long-term PV forecasting for the six New England states based on a variety of state policy support mechanisms, and provided technical guidance for the modeling of PV in system planning studies. Jon is directing ISO-NE’s efforts to develop enhanced short-term load forecast tools that incorporate the effects of behind-the-meter distributed PV, and has developed methods of estimating distributed PV fleet production profiles using limited historical data, as well as simulating high penetration PV scenarios to identify future net load characteristics. Jon participates in industry-leading research on forecasting and integrating large-scale renewable energy resources, and has served as a Technical Review Committee member on several multi-year Department of Energy studies. Upon joining ISO-NE in 2010, Jon assisted with the New England Wind Integration Study and the design of wind plant data requirements for centralized wind power forecasting.
Mr. Black is currently a PhD student researching advanced forecasting techniques within the Infrastructure and Environmental Systems program at the University of North Carolina at Charlotte. He received his MS degree in Mechanical Engineering from the University of Massachusetts at Amherst, where his research at the UMass Wind Energy Center explored the effects of varying weather on regional electricity demand and renewable resource availability. He is an active member of both the Institute of Electrical and Electronics Engineers (IEEE) and the Utility Variable Generation Integration Group (UVIG).

Announcing BFCom2016s Winners

The Spring 2016 BigDEAL Forecasting Competition (BFCom2016s) just ended last week. I received 49 registrations from 15 countries, of which 18 teams from 6 countries completed all four rounds of the competition. I want to give my special appreciation to Prof. Chongqing Kang and his teaching assistant Mr. Yi Wang. They  organized 8 teams formulated by students from Tsinghua University, an institute prize winner of GEFCom2014. Two of the Tsinghua Teams were finally ranked among the Top 6.

The topic of BFCom2016s is ex ante short term load forecasting. I provided 4 years of historical load and temperature data, asking the contestants to forecast the next three months given historical day-ahead temperature forecasts. Three months of incremental data was released in each round.

The benchmark is made by the Vanilla model, the same as the one used in GEFCom2012. This time among the top 6 teams, five were able to beat the benchmark on average ranking, while four beat the benchmark on average MAPE. The detailed rankings and MAPEs of all teams are listed HERE.

I invited each of the top 6 teams to send me a piece of guest blog to describe their methodology. Their contributions (with my minor editorial changes) are listed below, together with the Vanilla Benchmark, which ranked No. 7.

No.1: Jingrui Xie (avg. ranking: 1.25; avg. MAPE: 5.38%)
Team member: Jingrui Xie
Affiliation: University of North Carolina at Charlotte, USA

The same model selection process was used in all four rounds. The implementation was in SAS. The model selection process follows the point forecasting model selection process implemented in Xie and Hong, IJF-2016. In this competition, the forecasting problem was dissected into three sub-problems with each of them having slightly different candidate models being evaluated.
The first sub-problem was a very-short term load forecasting problem, which considered forecasting the first day of the forecast period. The model selection process started with the "Vanilla model plus the lagged load of the previous 24th hour". It then considered the recency effect, the weekend effect, the holiday effect, the two-stage model, and the combination of forecasts as introduced in Hong, 2010 and Xie and Hong, IJF-2016.
The second sub-problem was a short term load forecasting problem, which considered forecasting the second to the seventh day of the month. The model selection process was the same to that for the very-short term load forecasting problem except that the starting benchmark model is the Vanilla model.
The third sub-problem can be categorized as a middle term load forecasting problem in which the rest of the forecast period were forecasted. The model selection process also started with the Vanilla model, but it only considered the recency effect, the weekend effect, and the holiday effect.

No.2: SMHC (avg. ranking: 3.75; avg. MAPE: 5.90%)
Team members: Zejing Wang; Qi Zeng; Weiqian Cai

Affiliation: Tsinghua University, China

We tried the support vector machine (SVM) and artificial neural networks (ANN) models in the model selection stage. We found that the ANN model had a better performance than SVM. When considering the cumulative effect, we introduced the aggregated temperatures of several hours as augmented variables, while and the number of hours was also determined in the model selection process.
In the first round, we used all the provided data for training but didn't consider the influence of holidays. Then in the next three rounds, we divided the provided data into two seasons, “summer” and “winter”. We separately forecasted the load of normal days and special holidays. These so-called seasons are not the traditional ones but were roughly defined by the plot of the average load of the given four years. Then we used the data from each seasons for training to forecast the corresponding season in 2014. This ultimately achieved a higher accuracy. All the aforementioned results and algorithms were implemented by using the MATLAB and C language.

No. 3: eps (avg. ranking: 5.25; avg. MAPE: 6.08%)

Team member: Ilias Dimoulkas

Affiliation: KTH Royal Institute of Technology, Sweden

I used the Matlab’s Neural Network toolbox for the modeling. The evolution of my model during the four rounds was as follows.
1st round: I used the “Fiiting app” which is suitable for function approximation. The training vector was IN =  [Hour Temperature] and the target vector OUT = [Load]
2nd round: I used the “Time series app” which is suitable for time series and dynamical systems. I used the Nonlinear Input-Output model instead of the Nonlinear Autoregressive with External Input model because it performs better for long term forecasting. The training vector was still IN =  [Hour Temperature] and the target vector OUT = [Load]. The number of the delays I found it works better is 5 (= 5 hourly lags).
3rd round. I used the same model but I changed the training vector to IN = [Month Weekday Hour Temperature AverageDailyTemperature MaxDailyTemperature] where AverageDailyTemperature is the average temperature and MaxDailyTemperature is the maximum temperature of the day that the specific hour belongs to.
4th round: I used two similar models with different training vectors. The final output was the average of the two models. The training vectors where IN1 = [Month Weekday Hour Temperature MovingAverageTemperature24 MovingMaxTemperature24] and IN2 = [Month Weekday Hour Temperature AverageTemperaturePreAfter4Hours MovingAverageTemperature24 MovingAverageTemperature5 MovingMaxTemperature24] where MovingAverageTemperature24 is the average temperature of the last 24 hours, MovingAverageTemperature5 is the average temperature of the last 5 hours, MovingMaxTemperature24 is the maximum temperature of the last 24 hours and AverageTemperaturePreAfter4Hours is the average temperature of the hours ranging from 4 hours before till 4 hours after the specific hour.

No. 4: Fortune Teller (avg. ranking: 6.25; avg. MAPE: 6.45%)

Member: Guangzheng Xing; Zetian Zheng; Liangzhou Wang

Affiliation: Tsinghua University, China

Round 1. Variables：Hour, Weekday, T_act, TH(the highest temperature in a day), TM(the mean temperature), TL(the lowest temperature). First of all, we used the MLR, fitting the mean load by TM, TM^2, TM^3. This method didn’t work well, the MAPE could reach about 14%. Then we used neural network, the data set contains the six variables above, and the target value is the Load_MW. The result is better, but because of improper parameters, the model was kind of overfitted, and we didn’t do the cross-validation. The result was not so good.
Round 2. We changed the parameter, and used the max value/min value/ mean value of the previous 24 hours rather than those of the day. The result was much better.
Round 3. We tried to use SVM to classify the two kinds of day curve, and then used the nnet separately. But this method did not seem to be effective. Then we used the SVM to do regression, the data set is same in nnet. Using the test set, the results of SVM and nnet were similar, so we submitted the mean value of both methods’ result.
Round 4: The MAPE of both methods reach over 7% during model selection, the result of SVM was worse, so we only submitted the result of nnet.

No. 5: Keith Bishop (avg. ranking: 6.50; avg. MAPE: 6.47%)

Team member: Keith Bishop

Affiliation: University of North Carolina-Charlotte, USA; Hepta Control Systems, USA

For my forecast, I utilized SkyFoundry’s SkySpark analytics software.  SkySpark is designed for modelling complex building systems and working with the time-series data on a wide range of levels. To support my model, I extended the inherent functionality of this software to support polynomial regression.  My model itself went through several iterations.  The first of these was fairly similar to Dr. Hong’s Vanilla Model with the exception that instead of clustering by month, I clustered based on whether the date was a heating or cooling date.  The heating or cooling determination was made by fitting a third-degree polynomial curve to each, hourly clustered, load-temperature scatter plot, solving for the minimums and then calculating the change-over point by averaging these hourly values.  If the average temperature for a day was above this point, it was a cooling day and vice-versa.  As my model progressed, I incorporated monthly clustering and the recency effect discussed in Electric load forecasting with recency effect: A big data approach.  With the recency effect, I optimized the number of lag hours for each monthly cluster by creating models for each of the past 24-hours and selecting the one with the lowest error.  In the end, I was able to reduce the MAPE of the forecast against the known data from 8.51% down to 5.01%.

No. 6: DUFEGO (avg. ranking: 7.25; avg. MAPE: 6.39%)

Team members: Lei Yang; Lanjiao Gong; Yating Su, 

Affiliation: Dongbei University of Finance and Economics, China

During the 4-round competition,we selected MATLAB as our tool. We use multiple linear regression models (MLR), each of which has 291 variables including trend, polynominal terms,interaction terms and recency effect. We just used all past historical data without cleansing the data. Considering the forecasting task is to improve predicting accuracy rather than the goodness of fit, we seperated the data into training set and validation set. We used cross validation and out of sample test method to select variables to give our model more generalizaton ability.
In Round 1, we trained one MLR model using the entire historical data. In Round 2, we roughly grouped the historical data by season (such as January - March and April - June,) and trained four MLR models, which improved the results significantly. We also found the distinct relationship between temperature and load in different temporal dimensions.We did some work about selecting the best MLR model in different temporal dimensions and found seasonal separate better. We made a mistake in Round 3 that resulted in a very high MAPE.

No. 7: Vanilla Benchmark (avg. ranking: 7.25; avg. MAPE: 6.42%)

The model is the same as the one used in GEFCom2012. See Hong, Pinson and Fan, IJF2014 for more details. All available historical data in each round was used to estimate the model.

Finally, congratulations to these top 6 teams of BFCom2016s, and many thanks to all of you who participated and are interested in BFCom2016s!

BigDEAL Forecasting Competition - Spring 2016

[Update]: Announcing BFCom2016s winners; link to competition results.

I organized two in-class competitions last semester for my Energy Analytics course, one on short term load forecasting, and the other on probabilistic load forecasting. The competitions were very well received by my students and the external participants. The probabilistic forecasting competition generated a nice article for the International Journal of Forecasting.

I'm teaching another forecasting class, Technological Forecasting and Decision Making, this semester at UNC Charlotte. The course is at the same level as the Energy Analytics one. While the Energy Analytics course covers on various forecasting problems in the energy industry, this technological forecasting course covers various forecasting techniques without a specific focus on any industry. The course outline is available HERE.

I would like to open one of the exams to the external participants. I also plan to do so for my other forecasting-related courses going forward. Since I will leverage the help from BigDEAL members to run the show, I'm branding these activities as the BigDEAL Forecasting Competitions.

Here are the rules for this one:

• The competition will start on 3/24/2016, and end on 4/6/2016.
• The exam is individual effort. Each student will form a single-person team. No offline collaboration is allowed.
• External participants may form multi-person teams with the team members identified during the registration process.
• The competition topic will be on point forecasting. (At this stage, I haven't decided the exact problem to release yet.)
• Incremental data will be released during the competition.
• A report documenting how the models have been evolving is required to be eligible on the final leaderboard.

Interested? Register HERE by 3/22/2016. (If you don't have access to Google Form, you can email me directly to register.) 

Analytics, Smart Grid and Big Data: Are They Like Teenage Sex?

I can hardly find the original source for this quote about teenage sex:

Everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.

Over the past few decades, people have been inventing, abusing, reinventing and re-abusing various buzzwords. The title of this blog post is taken from a talk I gave last year.

In that talk, I was showing the audience how the public interest on these three terms has been evolving over time. For instance, "smart grid" on Google Trends look like this:

I also introduced my understanding of big data analytics using a series of research projects on load forecasting with NCEMC. At the end, I was making three points:

• Forget about the buzzwords
• Focus on what the industry needs
• Solve real-world problems

The original presentation is available HERE, in case you are interested in taking a look.

p.s., when naming my lab two years ago, I almost used all of these three terms, analytics, smart grid and big data. Because I didn't really understand what smart grid is, I put "energy" instead of "smart grid" in my lab's name, making it BigDEAL - Big Data Energy Analytics Laboratory. 

BigDEAL Backup and Archiving Process

One surprise I got over this winter holiday is the sudden death of my hard drive. The even bigger surprise is that I didn't have the backup. More precisely, I backed up the data every semester, but I accidentally deleted the backup during my last backup over the summer. That said, I have lost almost all the files created since I joined UNC Charlotte in August 2013.

This hard lesson reminded me to establish the BigDEAL backup and archival process below.

Storage options

Each BigDEAL member should have at least one external hard drive and one cloud drive for backup purpose. BigDEAL uses two external hard drives for backup purpose. One (M-drive) is managed by the lab manager (typically a senior PhD student on campus) for monthly backup. The other one (S-drive) managed by the BigDEAL director (Dr. Hong) for semester backup. BigDEAL has been using OneDrive as the cloud drive for archival purpose.

Backup process

BigDEAL backup process is executed on three levels:

• Weekly backup. Each BigDEAL member is expected to perform weekly backup on the personal hard drive and through the cloud storage service.
• Monthly backup. At the end of each month, each BigDEAL member should back up all the data on M-drive. 
• Semester backup. At the end of each semester, each BigDEAL member should back up all the data on S-drive. 

Archiving process

BigDEAL archives the following items on the cloud:

• Data
• Code
• Papers
• Thesis and dissertations
• Technical reports
• Presentations
• Administrative reports (weekly reports, semester summary, and semester plan, etc.)

In principle, once an item is acquired, updated or completed, the BigDEAL member in charge of the project or corresponding research subject should archive the item on the cloud and report the archival status to the BigDEAL director.

When archiving the revised version of the data, all changes and justifications should be documented and archived along with the new version. When archiving a paper, all the data, code, submitted versions, decision letters, response letters, and published version should be archived. 

BigDEAL Students Winning NPower Gas Demand Forecasting Challenge 2015

RWE npower hosted its second forecasting challenge in 2015 last month. The topic was ex post 6-month ahead daily gas demand forecasting with hourly weather information.

44 teams worldwide joined the competition, of which 22 were from U.K. Three of my students (Jingrui Xie, Bidong Liu and Ying Chen) formed the team BigDEAL@UNCC to join the competition. 

Ying Chen - From Meteorology to Energy Forecasting

Today (October 21, 2015), Ying Chen just defended her MS thesis "How Does Relative Humidity Affect Electricity Demand?"

Ying received her B.S. in Atmospheric Science from Nanjing University of Information Science and Technology in 2006, and her M.S. in Meteorology from University of Hawaii at Manoa in 2010. In August 2014, Ying joined the Master of Science in Engineering Management program of UNC Charlotte as a member of BigDEAL to conduct MS thesis research under my supervision.  She received her SAS Base Programmer and SAS Advanced Programmer certifications in fall 2014. She participated in the Global Energy Forecasting Competition 2014 with a top 9 place in the probabilistic solar power forecasting track. This past summer, Ying presented her research work at the 3rd International Conference on Energy & Meteorology.

Since October 2014, Ying has been working at North Carolina Electric Membership Corporation as a Load Forecasting Analyst Intern. This December, She will receive her MS in Engineering Management.

Congratulations, Ying!

BigDEAL Recruitment Process

I came back to academia to produce the finest data scientists for the energy industry. Over the past two years, I have established the factory, BigDEAL, offering golden opportunities for students to conduct cutting edge research through solving real world problems. As BigDEAL is getting increasingly recognized by the industry and academia, I'm receiving more and more requests from both sides too. While the employers are anxious to hire my students, many prospective students are eager to join BigDEAL.

There is no way for me to respond to every applicant. Here I'm sharing the BigDEAL recruitment process, so that the applicants can prepare accordingly.

1. When an application reaches my inbox, I take a brief look to see if the person has followed the instructions. If so, and if the background seems to be relevant, I would forward the email to the lab manager of BigDEAL, typically a senior PhD student. 
2. The lab manager will conduct the first round of interview. If s/he believes the person may be a good match, s/he will forward the application to several other lab members for the second round interview. After the second round interview, all interviewers will get together to give me their feedback. 
3. If the feedback is overall positive, I will conduct the last round interview. The applicants who pass the final round of interview will be offered a position at the BigDEAL.

With the strong support from the industry, I have never had a budget constraint when making hiring decisions. Nevertheless, I do recognize that the quality of the raw materials plays an important role of driving the quality of the end product. Therefore, I have decided to maintain the highly selective recruitment process. So far, we have been hiring no more than one student each semester.

If you are interested in joining BigDEAL, I would strongly suggest that you read the blog posts for prospective students before contacting me. Best luck with your application!

Combining Load Forecasts from Independent Experts: Experience at NPower Forecasting Challenge 2015

Forecast combination is regarded as one of the best practices of forecasting. I think it is a straightforward and practical approach to improving existing forecasts. This paper describes the method my students took in the NPower Forecasting Challenge 2015. We will present the paper at the 47th North America Power Symposium.

Citation
Jingrui Xie, Bidong Liu, Xiaoqian Lyu, Tao Hong, and David Basterfield, "Combining load forecasts from independent experts: experience at NPower forecasting challenge 2015", the 47th North American Power Symposium (NAPS2015), October 4 - 6, 2015

Combining Load Forecasts from Independent Experts

Experience at NPower Forecasting Challenge 2015

Jingrui Xie, Bidong Liu, Xiaoqian Lyu, Tao Hong, and David Basterfield

Abstract

The NPower Forecasting Challenge 2015 invited students and professionals worldwide to predict daily energy usage of a group of customers. The BigDEAL team from the Big Data Energy Analytics Laboratory landed a top 3 place in the final leaderboard. This paper presents a refined methodology based on the implementation during the competition. We first build the individual forecasts using several forecast techniques, such as Multiple Linear Regression (MLR), Autoregressive Integrated Moving Average (ARIMA), Artificial Neural Network (ANN) and Random Forests (RF). We then select a subset of the individual forecasts based on their performance on a validation period, a.k.a. post-sample. Finally we obtain the final forecast by averaging the selected individual forecasts. The forecast combination on average yields a better result than the forecast from a single technique.

BigDEAL Students Receiving Awards from Lee College of Engineering

Today (April 21, 2015), William States Lee College of Engineering hosted an award luncheon to celebrate student achievements. I'm very pleased to see two of our BigDEAL students being recognized in this event:

• Jingrui (Rain) Xie received the Outstanding Graduate Research Assistant Award. I recently highlighted Rain's achievement in THIS POST. 
• Bidong Liu received the Outstanding Graduate Student Award, mainly for his leadership to win the poster contest of Analytics2014 and NPower Forecasting Challenge 2015. 

Moreover, we had the first full house today with all BigDEAL members gathered on campus. Here is a selfie we took after the award luncheon. Thanks to Bidong's selfie stick :)

BigDEAL Selfie - Spring 2015

Congratulations, Rain and Bidong!