I recently wrote this article for Intelligent Utility. They used the title "13 lucky tips to juggle the analytics of forecasting". I have been using the INFORMS definition of analytics, which includes descriptive analytics (or summary statistics), predictive analytics (or forecasting) and prescriptive analytics (or optimization). Since forecasting is part of analytics, I'm using "13 lucky tips for energy forecasting" here to be consistent with the definition from INFORMS.
Published in Intelligent Utility Magazine July/Aug 2014
Citation
Tao Hong, "How to Juggle the Analytics of Forecasting: 13 Lucky Tips", Intelligent Utility, pp. 11-13, July/August, 2014
Energy forecasting is one of those areas of great importance to electric grid that gets little attention—even from power industry insiders. But you need to know how to make the best of your forecasting process. Here are 13 tips to get you started.
1. Understand business needs
Forecasting is a necessary and crucial process across virtually all segments of the utility industry: generation, transmission and distribution companies, retailers, regulatory commissions, trading firms and financial institutes. Applications of energy forecasting include (but are not limited to) power systems planning and operations, financial planning and risk management. Different business needs may require different forecasting horizons, frequencies and even techniques. For instance, although black-box approaches are often favored by energy traders on day-ahead load forecasting, they are rarely acceptable in long-term load forecasting for rate case filing purposes
2. Know your audience
Energy forecasts by themselves do not generate any value unless understood and used properly. It is very important to keep the business users in mind when developing forecasts. Most models and methodologies in academic literature are meaningless in the real world because nobody can really understand and reproduce them. Very often an interpretable (but less accurate) model can offer more value than its academic counterpart
3. Find the right tools
There are many tools in the market that may be used for energy forecasting purposes, such as MS Excel, SAS and R. They all have different learning curves, levels of technical support, depth of forecasting procedures and price tags. There is not one single tool dominating all metrics. For instance, MS Excel is very easy to use but does not offer many advanced forecasting techniques. When considering which tools to use, you have to evaluate many factors, such as direct costs (i.e., license and service fees) and indirect costs (salaries and training costs for the users) of the software package, potential value-add and implementation time.
4. Set a realistic target
All forecasts are wrong. Expecting perfect forecasts is unrealistic and one of the worst practices in forecasting. A best practice would be to set a realistic target with the understanding that accuracy can be affected by many factors: magnitude of the load, customer segmentation, timing and predictability of those dependent variables. A bad model may have some lucky moments (and vice versa), but a good forecaster should always be able to analyze the situation with a cool mind regardless.
5. Keep it simple, stupid
Simple is not trivial. A simple model or methodology can be quite powerful. In fact, in the energy forecasting arena, most models that offer significant practical value to the industry are eveloped from simple ideas. Always start with those simple models. Avoid adding new elements (i.e., techniques and variables), unless the additional complexity can be justified by significant performance improvement.
6. Fix data quality issues analytically, not just technically
There are many quality issues with real-world data. The most common data quality issue is missing value, which may be due to a temporary shut down of the system—i.e., meter, SCADA or weather station. These missing observations can be technically "fixed" by imputing the missing values based on some linear extrapolation or regression splines. However, depending upon where these observations are, the technical fixes may create severe problems in the forecasting process. For instance, if the loads during daily peak hours were missing, those technical fixes most likely would result in a lower peak than actual. These faked historical values often lead to underestimation of the future peaks. Instead of having your IT department do the data cleansing, you should look into the data quality issue analytically through a modeling approach—developing a model based on available historical data to fill in the blanks.
7. Look beyond accuracy
For many decades, researchers and practitioners have been working hard to reduce forecasting errors. As a result, various techniques have been tried for energy forecasting. However, a forecast is just the output of the entire forecasting process. And accuracy is just one measure of the quality of the forecast. When evaluating the forecasting process (including the forecast itself), there are many other things we should consider, such as the output format, computational complexity, interpretability, reproducibility, traceability, defensibility and so forth.
8. Gather a second opinion (combining forecasts)
All models are wrong. If only one model is being used, you will experience "bad" forecasts from time to time. If multiple models are available, the situation can be completely different. Have confidence when the models agree with each other. Focus on the periods when these models disagree with each other significantly. Empirically, combining forecasting techniques usually does a better job than each individual by offering more robust and accurate forecasts.
9. Conduct ex post forecasting analysis
Forecasters enjoy predicting the future more than looking back to the past. Ex post forecasting analysis refers to after-the-event forecasting, or forecasting with actual information of the independent variables in the forecasted period. The ex post forecasting error can be treated as the modeling error. Very often the modeling error is way higher than the errors from the predicted independent variables, such as temperature and macroeconomic indicators. Such analysis can help improve the forecasting process.
10. Take an interdisciplinary approach
In today's world, developing the load forecasts (at high voltage level) without using temperature forecasts does not offer much practical value. Energy forecasting is an interdisciplinary field. To further advance our knowledge, we have to take an interdisciplinary approach by involving various communities, such as statistical forecasting, artificial intelligence, meteorological science and power engineering.
11. Build a forecasting center of excellence
Virtually all types of energy forecasts are connected. A short-term load forecasting model can be augmented to a long-term model by adding macroeconomic indicators. Electricity prices are no longer driven by load only. The volatile renewable generation from wind and solar farms also affects prices significantly. Price signals trigger demand response programs, which in turn affect loads. So, build an in-house analytics center of excellence where statisticians, data miners, meteorologists, business liaisons, IT analysts and software developers can work together to tackle the emerging challenges of energy forecasting.
12. Keep learning and sharing
The forecasting community is advancing methodologies and techniques every day. Many of these new findings are applicable to energy forecasting. For instance, the hierarchal time series forecasting techniques used in retail and the consumer packaged goods industry can be adopted to household-level load forecasting. Probabilistic forecasting techniques used in meteorological forecasting have been applied to wind power forecasting. To keep the forecasts competitive in the market, energy forecasters have to follow the recent findings in the field. Getting involved in professional groups, such as the IEEE Working Group on Energy Forecasting, is an effective way to learn ideas from and share experience with other energy-forecasting experts.
13. Lastly, but most importantly, be honest.
Honesty is the foundation of everything. As a forecaster, you may get external pressure to "adjust" the forecasts to meet someone's agenda. The CFO may want to see an increasing demand so that the predicted revenue will be high. The regulators may expect a mild load growth to reflect the past investment on the energy efficiency programs. Nevertheless, you should never manipulate the forecasts to fit other people's agendas. Once you start modeling other people's minds instead of real data, your forecast is useless and a disservice to your utility and the customers it serves.
Happy forecasting and best of luck.
Published in Intelligent Utility Magazine July/Aug 2014
Citation
Tao Hong, "How to Juggle the Analytics of Forecasting: 13 Lucky Tips", Intelligent Utility, pp. 11-13, July/August, 2014
How to Juggle the Analytics of Forecasting: 13 Lucky Tips
Tao Hong
1. Understand business needs
Forecasting is a necessary and crucial process across virtually all segments of the utility industry: generation, transmission and distribution companies, retailers, regulatory commissions, trading firms and financial institutes. Applications of energy forecasting include (but are not limited to) power systems planning and operations, financial planning and risk management. Different business needs may require different forecasting horizons, frequencies and even techniques. For instance, although black-box approaches are often favored by energy traders on day-ahead load forecasting, they are rarely acceptable in long-term load forecasting for rate case filing purposes
2. Know your audience
Energy forecasts by themselves do not generate any value unless understood and used properly. It is very important to keep the business users in mind when developing forecasts. Most models and methodologies in academic literature are meaningless in the real world because nobody can really understand and reproduce them. Very often an interpretable (but less accurate) model can offer more value than its academic counterpart
3. Find the right tools
There are many tools in the market that may be used for energy forecasting purposes, such as MS Excel, SAS and R. They all have different learning curves, levels of technical support, depth of forecasting procedures and price tags. There is not one single tool dominating all metrics. For instance, MS Excel is very easy to use but does not offer many advanced forecasting techniques. When considering which tools to use, you have to evaluate many factors, such as direct costs (i.e., license and service fees) and indirect costs (salaries and training costs for the users) of the software package, potential value-add and implementation time.
4. Set a realistic target
All forecasts are wrong. Expecting perfect forecasts is unrealistic and one of the worst practices in forecasting. A best practice would be to set a realistic target with the understanding that accuracy can be affected by many factors: magnitude of the load, customer segmentation, timing and predictability of those dependent variables. A bad model may have some lucky moments (and vice versa), but a good forecaster should always be able to analyze the situation with a cool mind regardless.
5. Keep it simple, stupid
Simple is not trivial. A simple model or methodology can be quite powerful. In fact, in the energy forecasting arena, most models that offer significant practical value to the industry are eveloped from simple ideas. Always start with those simple models. Avoid adding new elements (i.e., techniques and variables), unless the additional complexity can be justified by significant performance improvement.
6. Fix data quality issues analytically, not just technically
There are many quality issues with real-world data. The most common data quality issue is missing value, which may be due to a temporary shut down of the system—i.e., meter, SCADA or weather station. These missing observations can be technically "fixed" by imputing the missing values based on some linear extrapolation or regression splines. However, depending upon where these observations are, the technical fixes may create severe problems in the forecasting process. For instance, if the loads during daily peak hours were missing, those technical fixes most likely would result in a lower peak than actual. These faked historical values often lead to underestimation of the future peaks. Instead of having your IT department do the data cleansing, you should look into the data quality issue analytically through a modeling approach—developing a model based on available historical data to fill in the blanks.
7. Look beyond accuracy
For many decades, researchers and practitioners have been working hard to reduce forecasting errors. As a result, various techniques have been tried for energy forecasting. However, a forecast is just the output of the entire forecasting process. And accuracy is just one measure of the quality of the forecast. When evaluating the forecasting process (including the forecast itself), there are many other things we should consider, such as the output format, computational complexity, interpretability, reproducibility, traceability, defensibility and so forth.
8. Gather a second opinion (combining forecasts)
All models are wrong. If only one model is being used, you will experience "bad" forecasts from time to time. If multiple models are available, the situation can be completely different. Have confidence when the models agree with each other. Focus on the periods when these models disagree with each other significantly. Empirically, combining forecasting techniques usually does a better job than each individual by offering more robust and accurate forecasts.
9. Conduct ex post forecasting analysis
Forecasters enjoy predicting the future more than looking back to the past. Ex post forecasting analysis refers to after-the-event forecasting, or forecasting with actual information of the independent variables in the forecasted period. The ex post forecasting error can be treated as the modeling error. Very often the modeling error is way higher than the errors from the predicted independent variables, such as temperature and macroeconomic indicators. Such analysis can help improve the forecasting process.
10. Take an interdisciplinary approach
In today's world, developing the load forecasts (at high voltage level) without using temperature forecasts does not offer much practical value. Energy forecasting is an interdisciplinary field. To further advance our knowledge, we have to take an interdisciplinary approach by involving various communities, such as statistical forecasting, artificial intelligence, meteorological science and power engineering.
11. Build a forecasting center of excellence
Virtually all types of energy forecasts are connected. A short-term load forecasting model can be augmented to a long-term model by adding macroeconomic indicators. Electricity prices are no longer driven by load only. The volatile renewable generation from wind and solar farms also affects prices significantly. Price signals trigger demand response programs, which in turn affect loads. So, build an in-house analytics center of excellence where statisticians, data miners, meteorologists, business liaisons, IT analysts and software developers can work together to tackle the emerging challenges of energy forecasting.
12. Keep learning and sharing
The forecasting community is advancing methodologies and techniques every day. Many of these new findings are applicable to energy forecasting. For instance, the hierarchal time series forecasting techniques used in retail and the consumer packaged goods industry can be adopted to household-level load forecasting. Probabilistic forecasting techniques used in meteorological forecasting have been applied to wind power forecasting. To keep the forecasts competitive in the market, energy forecasters have to follow the recent findings in the field. Getting involved in professional groups, such as the IEEE Working Group on Energy Forecasting, is an effective way to learn ideas from and share experience with other energy-forecasting experts.
13. Lastly, but most importantly, be honest.
Honesty is the foundation of everything. As a forecaster, you may get external pressure to "adjust" the forecasts to meet someone's agenda. The CFO may want to see an increasing demand so that the predicted revenue will be high. The regulators may expect a mild load growth to reflect the past investment on the energy efficiency programs. Nevertheless, you should never manipulate the forecasts to fit other people's agendas. Once you start modeling other people's minds instead of real data, your forecast is useless and a disservice to your utility and the customers it serves.
Happy forecasting and best of luck.
gained after reading, temperature matters.........
ReplyDelete