Friday, July 31, 2015

Reading for Writing

Writing skills are badly needed in the professional world, no matter in the industry or academia. I have seen many people (mostly international students, including myself) struggling with their writing skills. I don't believe there is any shortcut but constant and regular writing practice to improve writing skills. In addition, I think reading is a good compliment to writing practice. Here I'm putting together a list of recommended books.

The first two are on general writing (not necessarily scientific writing):
  • On Writing Well by William Zinsser
  • The Elements of Style by William Strunk and E. B. White

Then here are two books for scientific writing:
  • Scientific Writing and Communication: Papers, Proposals, and Presentations by  Angelika Hofmann
  • Writing Science: How to Write Papers That Get Cited and Proposals That Get Funded by Joshua Schimel 
I also got a list of classic novels from my colleague Jason Wilson (co-author of my TSG2014 paper):
  • Slaughterhouse Five by Kurt Vonnegut, Jr.
  • Gulliver’s Travels by Jonathan Swift
  • Ulysses by James Joyce
  • Hamlet by Shakespeare 
  • 1984 by George Orwell
  • Brave New World by Aldous Huxley
  • Animal Farm by George Orwell
  • The Grapes of Wrath by John Steinbeck
  • A Farewell to Arms by Ernest Hemingway
  • The Old Man and The Sea by Ernest Hemingway
  • Charlotte’s Web by E.B. White
  • War and Peace by Leo Tolstoy
  • Jurassic Park by Michael Crichton
Here I'm trying not to recommend a list of scientific papers in energy forecasting. I would rather suggest that we focus on the analytical and technical aspects of those papers.

Happy reading!

Saturday, July 25, 2015

Combining Load Forecasts from Independent Experts: Experience at NPower Forecasting Challenge 2015

Forecast combination is regarded as one of the best practices of forecasting. I think it is a straightforward and practical approach to improving existing forecasts. This paper describes the method my students took in the NPower Forecasting Challenge 2015. We will present the paper at the 47th North America Power Symposium.

Jingrui Xie, Bidong Liu, Xiaoqian Lyu, Tao Hong, and David Basterfield, "Combining load forecasts from independent experts: experience at NPower forecasting challenge 2015", the 47th North American Power Symposium (NAPS2015), October 4 - 6, 2015

Combining Load Forecasts from Independent Experts
Experience at NPower Forecasting Challenge 2015

Jingrui Xie, Bidong Liu, Xiaoqian Lyu, Tao Hong, and David Basterfield


The NPower Forecasting Challenge 2015 invited students and professionals worldwide to predict daily energy usage of a group of customers. The BigDEAL team from the Big Data Energy Analytics Laboratory landed a top 3 place in the final leaderboard. This paper presents a refined methodology based on the implementation during the competition. We first build the individual forecasts using several forecast techniques, such as Multiple Linear Regression (MLR), Autoregressive Integrated Moving Average (ARIMA), Artificial Neural Network (ANN) and Random Forests (RF). We then select a subset of the individual forecasts based on their performance on a validation period, a.k.a. post-sample. Finally we obtain the final forecast by averaging the selected individual forecasts. The forecast combination on average yields a better result than the forecast from a single technique.

Friday, July 24, 2015

Tao Hong: Be Honest

Below is my recent interview with T&D World Magazine. The original version is HERE.

Tao Hong: Be Honest

Tao Hong always sticks to his integrity, especially when it comes to energy forecasting. As graduate program director and EPIC assistant professor at the Systems Engineering and Engineering Management Department at the University of North Carolina at Charlotte, he said the best advice he has ever received is to always be honest.
Sometimes we are pressured to make the forecasts following someone else' personal agenda. Rather than modeling other people's mind and making fraudulent forecasts, we should always stick to our integrity.
Hong will be presenting Energy Forecasting in the Smart Grid Era (blog post) at the 2015 IEEE PES General Meeting, being held July 26-30, in Denver, Colorado. The full-day tutorial covers how wide-range deployment of smart grid technologies enables utilities to monitor the power systems and gather data on a much more granular level than ever before. While the utilities can potentially better understand the customers, design the demand response programs, forecast and control the loads, integrate renewable energy and plan the systems, etc., they are facing analytic issues with making sense and taking advantage of the "big data".

This tutorial developed by IEEE Working Group on Energy Forecasting offers a comprehensive overview of energy forecasting to utility forecasters, analysts, planners, operators and their managers. The participants will learn the fundamentals and the state-of-the-art of load, price and wind forecasting through real world examples and case studies. Topics include:
  • Fundamentals of energy forecasting
  • Short-term and long-term electricity demand forecasting
  • Price forecasting in competitive electricity markets
  • Wind power forecasting in theory and practice
Hong also writes a blog on energy forecasting. See the Q&A below for more information on Hong's varied experience and how he applies it to consulting and teaching:

Q: How does your current position help you in presenting your session on energy forecasting, and how does your past experience help you in this role?

In my current role as a professor at University of North Carolina at Charlotte, I conduct cutting-edge research in the area of energy analytics, help energy companies worldwide through consulting engagement, and teach undergraduate/graduate students and industry professionals in and outside my university. All of these activities allow me to grasp the emerging industry trends and establish best practices, so that I can share them with the folks in my class.

Prior to joining the academic world, I worked in the industry as a consultant, first in a utility consulting firm (Quanta Technology) and then in a large analytics software vendor (SAS). That experience helps me understand what the industry really need, today and tomorrow. With that in mind, I can tailor the education materials to produce useful workforce for the industry.

Q: When and why did you decide to go into your particular career field?

My father was a utility executive. My grandfather was the head of electrical engineering department at a Chinese university. I think the family has more or less influence on my career moves - now I'm a professor in the power and energy field.

In 2008, I started my career at Quanta Technology, a firm started by several IEEE Fellows. In my first few years, I had the fortune of working with folks such as Jim Burke, Lee Willis and Richard Brown. I knew I couldn't replicate any of them. Meanwhile, I saw more and more smart grid technologies are being deployed. I thought data analytics would be badly needed and my education background in operations research could be a perfect fit. That was the time I started building my career toward data analytics.

Q: Best thing about your job right now?

Freedom. The university offers great flexibility to the professors, so that they can do the things that they think is meaningful.

Q: What courses/sessions have you presented in the past, and what’s coming up?

I have taught many courses, such as power distribution losses, energy markets, and power systems planning. My signature course is Electric Load Forecasting: Fundamentals and Best Practices, which has been taught for 17 times, with about 200 attendees coming from more than 50 organizations across more than 10 countries. I'm heading to Australia to teach this course for its 18th offering. I'm also writing a textbook with the same title. The book is available online with free access.

See the blog post and statistics about the course.

Q: What's the most important thing you’ve learned in your past experience that you want to communicate to students or participants?

Be honest. We, as forecasters, are often making forecasts to assist million-dollar decisions. Sometimes we are pressured to make the forecasts following someone else' personal agenda. Rather than modeling other people's mind and making fraudulent forecasts, we should always stick to our integrity. Once I had hard time making a decision on a related matter. I asked my mentor Jim Burke for his advice. This is what he wrote to me:
Always be honest, and while the consequences may be negative for a while, when you get my age, you'll be happy with yourself.....Jim
Which I consider the most valuable advice I got from him.

See the blog post about honesty.

Q: Why do you think "energy forecasting in the smart grid era" is important to the industry? How will it help attendees?

Thanks to the smart grid technologies and desire of a green future, the power industry today is facing active electricity demand, intermittent renewable generation and unpredictable electricity price spikes, which has brought significant challenges to power systems planning and operations. In such an uncertain environment, we have to rely on data and analytics in addition to our experience to make informed decisions. Forecasting is a crucial and fundamental step in the analytics workflow. From this tutorial on energy forecasting, students will gain an overview of the field of energy forecasting, understand the utility applications of forecasting, and learn best practices validated from field implementations.

Q: What do you like to do in your spare time?

If spare time is defined as the off-work hours minus family time, I would say that blogging is my favorite. I run a blog on energy forecasting, which is surprisingly popular given that this is not a big field. Last year, my audience were from more than 1400 cities across more than 120 countries. Apparently it is an extension of my career. I also see it as an escape, because I don't get anonymous reviewers to stop me from publishing a blog post.

See the blog post Energy Forecasting @2014.

Q: Anything else you would like to add about your teaching philosophy?

IEEE PES sponsored the Global Energy Forecasting Competition 2014. As the General Chair of the competition, I would like to invite the tutorial attendees to join the full-day program on Wednesday (7/29) including two panel sessions and a reception in the evening.

Monday, July 13, 2015

Job Opening at TEP: Customer Analytics & Data Forecasting Analyst

Posting a position for a friend at TEP. Click THIS LINK to see the details of the position

Customer Analytics & Data Forecasting Analyst

Company: Tucson Electric Power
Location: Tucson, AZ
Job Category: Rates and Revenue Requirements
Position Type: Unclassified

Friday, June 26, 2015

What's New in Energy Forecasting - Jun 2015

It's time for the mid-year report and summary of the exciting events in the energy forecasting community:

1. Global Energy Forecasting Competition 2014

The competition was launched in August 2014 and ended in December 2014. We are now in the stage of post-competition activities, such as organizing conference presentations and paper publications. Many finalists of the competition will gather at the IEEE PES General Meeting 2015. The papers are expected to be published in the IJF special issue on probabilistic energy forecasting in early 2016. To follow the update of this competition and future events, please join this LinkedIn group.

2. Activities at IEEE PES General Meeting

Wednesday, June 17, 2015

On Normality Assumption in Residual Simulation for Probabilistic Load Forecasting

I have been so fortunate to work with many talents in both industry and academia. Their involvement has added significant values to most, if not all, of my research projects. This paper is the result of one of those great examples.

The motivation of this project was simple. We were interested in improving the forecasts we produced for NCEMC a few years ago. (You can check our TSG2014 paper for the methodology we used to produce the previous forecasts. The same methodology was used in my recent EISPC/NARUC report.)

The initial idea was simple, too. Since all forecasts have errors, we would like to see if modeling and simulating the residuals can help improve the probabilistic forecasts. We expect an YES answer, because so many papers in the literature have reported a very similar approach. We also had a little doubt, because none of those papers really verified the approach via any formal error measures for probabilistic forecasting.

Monday, June 15, 2015

Energy Forecasting Talks at ISF2015

It's only one week toward the International Symposium on Forecasting 2015. I took a brief look at the program and found 5 sessions including 14 talks on energy forecasting. The at-a-glance program schedule is available HERE.

Wednesday, May 20, 2015

Energy Forecasting Activities at PESGM2015

IEEE Power & Energy Society just released the technical program agenda for PESGM2015. I'm pleased to highlight the two days on energy forecasting organized by our working group.

Please note that the Wednesday afternoon's panel session is in Plaza 3. We also added a reception on Wednesday evening.

Saturday, July 26, 2015

Energy Forecasting in the Smart Grid Era (full-day tutorial)
8:00 AM - 5:00 PM
Instructors: Tao Hong, Shu Fan, Hamidreza Zareipour, and Pierre Pinson

Wednesday, July 29, 2015

Thursday, May 14, 2015

Probabilistic Load Forecasting via Quantile Regression Averaging on Sister Forecasts

Although the probabilistic load forecasting literature can be traced back to 1970s, the importance of the subject was not well recognized until recent years. There are several approaches to producing probabilistic load forecasts, such as generating weather scenarios to feed to point forecasting models, applying probabilistic modeling and forecasting techniques, and identifying the density function of residuals. This paper starts a whole new category for probabilistic load forecasting methods - combining point load forecasts.

Tuesday, May 5, 2015

HONG Analytics LLC

I started my career as a consultant when I was pursuing my first MS degree. Then I developed my MS thesis and PhD dissertation, each from a consulting project. Both of them were later commercialized and now being used by many utilities worldwide. Most of my research ideas were inspired by the consulting projects through working closely with the utility analysts, managers and executives. After testing and validating these research ideas through several field implementations, I converted them to teaching materials for my courses. These courses fully packed with solid fundamental knowledge, best industry practices, advanced topics and a wide range of case studies have been very well received by the industry (see what the clients are saying), generating even more consulting business.

As mentioned in why I left a great place to work - from industry to academia, majority of my academic job is made of consulting, teaching and research. Going through such a cycle has been quite rewarding for both my clients and myself. Nevertheless, some of my clients are still complaining about the lengthy and verbose contracting process. To make the cycle even more enjoyable and productive, I decided to incorporate HONG Analytics LLC to formally house my consulting practices outside the university.

HONG Analytics provides premium training and consulting services in the area of energy and retail analytics. While being a full-time professor, I can allocate up to one day per week of my time to HONG Analytics. I would like to take on those projects that are relatively small in scale (i.e., less than $100k) and relatively short in performance period (i.e., less than 6 months). While I'm happy to help with conventional short and long term load forecasting projects, I would give preference to the projects that are challenging and requiring heavy-duty analytics. Since revenue is not the a KPI of my business, I am also willing to provide the services free of charge for those projects with high value to the industry.

For more information, please visit