Statistical Learning and Machine Learning

Machine learning is a field that studies how to make computers (machines) learn from the data to make predictions or help with data-driven decisions. The term "machine learning" started to become popular in 1990s, and then was somewhat surpassed, if not replaced, by "analytics". In the recent few years, following the buzzword wave of big data and deep learning,  the field of machine learning is gaining some momentum again. Since 1980s, machine learning techniques have been on most papers in the load forecasting literature.

Statistical learning, on the other hand, is not a familiar term to many people. I first got to know this term during my graduate school days, when I was reading the book The Elements of Statistical Learning. Since many techniques and methodologies introduced in this book can be applied to forecasting, I'm using this book as a reference book for my forecasting course this semester.

As of today, I still don't quite understand the difference between the two terms. It seems to me that statisticians like to use the term "statistical learning", while computer scientists and engineers are used to the term "machine learning". Maybe statistical learning has more statistical rigor, while machine learning emphasizes more on the algorithmic aspect. Another discussion about the difference between the two can be found on Wikipedia.

Early this year, I registered two online courses from Stanford University:

• Introduction to statistical learning by Trevor Hastie and Rob Tibshirani
• Machine learning by Andrew Ng

I thought it would be a good investment of my time for three reasons:

• As a researcher in energy forecasting, I want to refresh my memory on machine/statistical learning;
• As an instructor, I want to leverage the materials from other relevant courses when preparing for my forecasting course;
• As a Graduate Program Director for a top-ranked online MS program, I want to see how Stanford University sets up its online courses.

I started both courses almost at the same time, though I was quickly addicted to the statistical learning one and gave up the other course. While both courses are taught by world-class professors and equipped with state-of-the-art online teaching technologies, the one by Trevor and Rob fits my taste much better:

• The instruction style is very much grounded. These two statistics professors are very good at explaining the theory intuitively without involving much math. 
• The subtitle is awesome.
• Most of the quiz questions are testing the understanding of the concept, so I don't have to pull my computers to run R code. 
• The guest speakers bring valuable perspectives and insights about the subject. 
• The design of the progress bar is very simple and effective. Due to my busy schedule for other commitments, I almost quit the course several times. That progress bar helped me stay focus and on track. 

There are other nice features this course offers but I never had time to try, such as free textbook and an online discussion forum. I'm sure they are useful to the ones who can devote more time than me. In January and early February, I was watching video and answering the quiz questions late night before sleep every day. After getting about 35% overall progress, I was distracted by other commitments. Then in early April, I went through a few more lectures to pass the 50% bar. 

I will most likely take this course again, or at least watch the 15-hour video lectures. I would like to recommend this course to the energy forecasting community as well :)

Back to Load Forecasting Terminology.

Energy Efficiency, Demand Response and Demand Side Management

Among many challenges human beings are facing, two are related to energy resources and infrastructure, such as:

• Do we have enough coal (or fuel or natural gas) to burn over the next few centuries? 
• How to meet the peak demand with limited budget for infrastructure?

Load Forecasting and Load Research

Load forecasting is simply the activity to forecast load. (refer to Load, Demand, Energy and Power). According to AEIC Load Research and Analytics Committee, load research is the activity embracing the measurement and study of the characteristics of electric loads to provide a thorough & reliable knowledge of trends, and general behavior of the load characteristics of the customers serviced by the electric power industry.

Forecasting and Data Mining

The main difference between forecasting and data mining is on the goal of the task. The goal of forecasting is to make statements about the future, while the goal of data mining is to extract patterns from large datasets. (The term "data mining" was a buzzword 15 years ago to broadly refer to working on the data, which is a misuse.) Many techniques can be applied to both forecasting and data mining, such as artificial neural networks, regression analysis, and clustering analysis, and so forth.

Statistical Models and Econometric Models

Statistical models are the models built with statistical techniques. The counterpart can be artificial intelligence models, which are based on artificial intelligence. (Note that statistics and artificial intelligence are not mutually exclusive.) If a statistical model includes economic variables, no matter on the left or right side of the equation, the model can be called an econometric model.

Most forecasting analysts graduate from either economic department or statistics department. Depending upon their education background, they may call the same forecasting models differently. People with economics major usually call their models econometric models, while people with statistics major often call their models statistical models.

Calendar Month and Billing Month

Calendar month is the period of duration from the same date of one month to the same date of the next month, which can be 28, 29 (February during a leap year), 30 or 31. Most countries in the world are using solar calendar.

Prior to the smart grid era, the utilities were sending meter readers to read meters every month. Apparently they were not able to read all the meters at the same time. Since the meter readers work during work days, utilities put the customers into twenty plus groups, one for each workday, called a billing group. On each day, they read the meters from the corresponding billing group. Billing month is just the period between the two adjacent billing days for a billing group. The electricity consumption on a monthly bill was the difference between the two monthly meter readings, which can be quite different from the consumption over a calendar month.

For load forecasting purposes, we would like to map the energy on monthly bills to the energy on calendar months. This often involves a convoluted process, which creates a lot of troubles and conflicts between the accounting and planning departments in a utility. A famous and challenging problem, unbilled energy, was born due to the mismatch between calendar month and billing month. One of the benefits of smart meter deployment is to resolve the unbilled energy problem.

Back to Load Forecasting Terminology.

Load Factor, Coincidence Factor, Diversity Factor and Responsibility Factor

Load factor is average load of a system divided by its peak load. The higher the load factor is, the smoother the load profile is, and the more the infrastructure is being utilized. The highest possible load factor is 1, which indicates a flat load profile.

In the old days, load factor is often used for long term peak demand forecasting. The forecasters first develop a energy forecast. They then calculate the average hourly load. Finally by dividing the forecasted average load by a predefined load factor, they can obtain the forecasted peak. However, I would avoid using this method for long term load forecasting in today's world where high resolution data is available for load forecasting.

Standard Time, Daylight Saving Time and Local Time

The earth is round like a ball. When it's night in the US, it's morning in China. To do business beyond a local region, people need a common reference to communicate time. Standard time is the synchronization of clocks in different geographical locations within a time zone to a common time standard, usually based on the meridian at the center of the time zone.

Prediction Interval and Confidence Interval

This is a pair of terms very difficult to distinguish, because statisticians and economists don't follow the same standard. Since load forecasting falls under the umbrella of forecasting, I'm following the terminology developed by the forecasting community. Special thanks to Rob Hyndman, who answered many questions from me during my preparation of this post. I highly recommend you his two blog posts The difference between prediction intervals and confidence intervals and Prediction intervals too narrow.

In short, there is a simple rule that tells where to use confidence or prediction interval:

A confidence interval is associated with a parameter, while a prediction interval is associated with a prediction. 

Below I'm using three examples to illustrate how to apply these two terms in load forecasting.

Weather, Climate and Temperature

Weather is the condition of the atmosphere, such as temperature, humidity and rainfall, at a particular place over a short period of time, i.e., a few days. For instance, a weather forecast usually goes a few days ahead. Climate refers to the weather pattern of a place over a long period, i.e., a few decades or more. A well-known term is "climate change".

In load forecasting, the most frequently used weather variable is temperature. A temperature station is often called weather station, though a weather station may measure many variables beyond temperature.

Many utilities also include other weather variables as predictors in short term load forecasting models, such as humidity, wind speed and cloud cover. In reality, because it is difficult to forecast these predictors with good accuracy, there is a trade-off between the information gained by adding these additional variables and the noise introduced by their forecast errors. I always go with the principle of parsimony. Unless rigorous tests have been conducted showing the benefits of adding additional variables, I would try to keep the model as lean as possible.

Back to Load Forecasting Terminology. 

Resolution (for Hierarchical Load Forecasting) and Resolution (for Probabilistic Load Forecasting)

During the past several decades, utilities have been developing long term load forecasts mostly using monthly data aggregated up to revenue class level or higher. Deployment of smart grid technologies allows utilities to collect data with hourly or sub-hourly interval at household level. Using these "high resolution" data, we can develop load forecasts at various levels in the system, which is called hierarchical load forecasting. There are two aspects of resolution in hierarchical load forecasting:

• Spatial resolution

Spatial resolution means how many points are being measured in a piece of land. In my master thesis on spatial load forecasting, I divided the service territory of a medium sized utility into 3460 small areas, about 50 acres each. The data was from transformer load management system. In today's world, a "small area" can be 0.2 acre (the size of a typical single family home) or smaller.  While short term load forecasts have been mostly developed based on hourly or half-hourly data, having load information at low levels can help enhance the forecasting accuracy (See One Size No Longer Fits All: Electric Load Forecasting with a Geographic Hierarchy).

• Temporal resolution

Temporal resolution means the sampling frequency of the meters. In my 2014 TSG paper, a major contribution was to demonstrate the additional forecasting accuracy gained by using high resolution data.

In probabilistic load forecasting, resolution refers to how the size of prediction interval varies at different time periods. A high-resolution probabilistic forecast can properly quantify the uncertainties at different time periods by providing the prediction interval with variable size. For instance, in the figure below, the prediction interval of summer months is much narrower than that of winter months, which tells that load is much more uncertain in winter than in summer.

Back to Load Forecasting Terminology.

Quantile, Quartile and Percentile

Suppose we have a set of data sorted in ascending order,  by dividing the data into q equal-sized pieces, we can get q-quantiles. The quantiles are the values marking the boundaries between two adjacent subsets.

Reliability (for Planning) and Reliability (for Forecasting)

My first job interview in the US was with Richard Brown. At that time, I knew virtually nothing about the electric power industry. To prepare for the interview, I checked Richard's profile, and found out that he is a Fellow of IEEE for his contribution in power systems reliability. I also browsed through his book Electric Power Distribution Reliability, a must-read book in power systems reliability. Although none of the things I prepared was actually used in the interview, I got to know about reliability before I even heard of load forecasting.

Probability Forecasting and Probabilistic Forecasting

Probabilistic energy forecasting is an emerging branch of energy forecasting. I think it's very important to clarify some concepts in the early stage, so that we don't have to run into troubles arguing what the terms mean 10 years later. This post is about probability forecasting and probabilistic forecasting.

Weather Normalization and Load Normalization

In the electric power industry, there are two variables often associated with "weather normalization": reliability and load. If you are interested in reliability, you may refer to the activities of IEEE Working Group on Distribution Reliability. In this post, I'm going to focus on load.

Planning - the business driver of weather normalization

Training, Validation and Test

When developing models for forecasting or data mining (I will write a post about these two terms), we usually slice the data into three pieces, training, validation and test:

• Training data is used to estimate the parameters. 
• Validation data is used to select models. 
• Test data is used to confirm the model performance. 

Here let me use two representative techniques, regression analysis and Artificial Neural Networks (ANN) to illustrate how the process works.

Linear Models and Linear Relationship

Update 3/20/2015: sharing a blog post from Freakonometrics On Some Alternatives to Regression Models.
In many papers, we can find the statements similar to the one below:

Because linear models can hardly capture the nonlinear relationship between load and temperature, we use Artificial Neural Networks (or other black-box models) in this paper. 

The major conceptual error of the above statement is due to a common misunderstanding that linear models cannot capture nonlinear relationship.

Model, Variable, Function and Parameter

My first job was an engineer at an expert-based consulting firm. One of my first tasks was to develop models for a large Investor Owned Utility. I was quite exited when being assigned to this project - I thought it was a good opportunity to show off my modeling skills. During the project kick-off meeting, I realized that I misunderstood the scope of work. The "models" I was asked to develop are circuit models. "Modeling" was simply to draw the lines, fuses, switches and transformers on a distribution engineering software platform based on their physical specifications, which did not involve any math or statistics at all.

The predictive models in energy forecasting are different from the physical models mentioned above. A regression-based load forecasting model, for example, describes the relationship between load and the factors that drive the load. There are three components in such a model:

Very Short, Short, Medium and Long Term Load Forecasting

Load forecasting is so fundamental that it is being used across all sectors in the electric power industry for various business applications. Because of the wide spread of its applications, there are many ways to classify the various load forecasts:

• based on forecast horizon: very short, short, medium and long term load forecasts;
• based on resolution of the data or updating frequency (these two concepts are different!): hourly, daily, monthly, seasonal and annual load forecasts;
• based on business needs: operational, planning and retail load forecasts;

Forecasting, Forecast and Forecaster

The story starts with my graduate school days. After my PhD defense, Dr. David Dickey, who was one of the members on my doctoral committee, handed me a printout of my draft dissertation. Based on the edits, I can tell that he read it in detail and revised it carefully. One of the many things he suggested was to change "short term load forecaster" to "short term load forecasting system" to represent the solution, or to "short term load forecasting model" to represent the mathematical formula that captures the relationship between load and other explanatory variables.

It was probably the only time I didn't follow his advice. My reason was that some papers in the load forecasting literature used the term "load forecaster" to represent a load forecasting system. For instance, ANNSTLF, a system developed 15 years ago by EPRI, is the acronym of "Artificial Neural Network Short Term Load Forecaster".