Most of my professional experience has been on energy forecasting. I have also worked on retail forecasting in the CPG (consumer package goods) industry for large retailers who have thousands of stores and millions of SKUs (store keeping units). This paper is a perfect combination of the two - energy forecasting for electricity retailers. In this paper, we tackle the challenge of customer attrition. A significant contribution of this paper is that it sets a role model of putting business needs in front of designing and developing load forecasting models.
Unfortunately, this paper went through 4 revisions before it was finally accepted. It could have been accepted in the second round if I had tried to please the three reviewers. One of them was excellent, but the other two were nonsense. To maintain my publication quality and keep the dignity of the paper, I decided to argue with the nonsense ones. Although it wasted me a lot of time, I didn't change more than three sentences in the last three rounds of revisions.
The paper was accepted by IEEE Transactions on Smart Grid last week. The working paper is available HERE. I will update the citation once the paper is on IEEE Xplore.
Citation
Jingrui Xie, Tao Hong and Josh Stroud, "Long term retail energy forecasting with consideration of residential customer attrition", IEEE Transactions on Smart Grid, vol.6, no.5, pp. 2245-2252, September, 2015. working paper available from http://www.drhongtao.com/articles.
Abstract
Deregulation of the electric power industry has created both wholesale markets and retail markets. Most load forecasting studies in the literature are on the wholesale side. Minimal research efforts have been devoted to tackling the challenges on the retail side, such as limited data history and high customer attrition rate. This paper proposes a comprehensive methodology to long term retail energy forecasting in order to feed the forecasts to a conservative trading strategy. We dissect the problem into two sub-problems: load per customer forecasting and tenured customer forecasting. Regression analysis and survival analysis are applied to each sub-problem respectively. The proposed methodology has been implemented at a fast growing retailer in the US showing superior performance, in terms of Mean Absolute Percentage Error (MAPE) of hourly demand, and daily and monthly energy, over a commonly used method that assumes constant customer attrition rate.
Unfortunately, this paper went through 4 revisions before it was finally accepted. It could have been accepted in the second round if I had tried to please the three reviewers. One of them was excellent, but the other two were nonsense. To maintain my publication quality and keep the dignity of the paper, I decided to argue with the nonsense ones. Although it wasted me a lot of time, I didn't change more than three sentences in the last three rounds of revisions.
The paper was accepted by IEEE Transactions on Smart Grid last week. The working paper is available HERE. I will update the citation once the paper is on IEEE Xplore.
Citation
Jingrui Xie, Tao Hong and Josh Stroud, "Long term retail energy forecasting with consideration of residential customer attrition", IEEE Transactions on Smart Grid, vol.6, no.5, pp. 2245-2252, September, 2015. working paper available from http://www.drhongtao.com/articles.
Long Term Retail Energy Forecasting with Consideration of Residential Customer Attrition
Jingrui Xie, Tao Hong and Josh Stroud
Abstract
Deregulation of the electric power industry has created both wholesale markets and retail markets. Most load forecasting studies in the literature are on the wholesale side. Minimal research efforts have been devoted to tackling the challenges on the retail side, such as limited data history and high customer attrition rate. This paper proposes a comprehensive methodology to long term retail energy forecasting in order to feed the forecasts to a conservative trading strategy. We dissect the problem into two sub-problems: load per customer forecasting and tenured customer forecasting. Regression analysis and survival analysis are applied to each sub-problem respectively. The proposed methodology has been implemented at a fast growing retailer in the US showing superior performance, in terms of Mean Absolute Percentage Error (MAPE) of hourly demand, and daily and monthly energy, over a commonly used method that assumes constant customer attrition rate.
Hi Tao,
ReplyDeleteI really enjoyed your recent article on REP forecasting. It discussed a number of issues that I have dealt with, but have not seen addressed anywhere in the literature. I have a few questions if you have the time.
I am surprised that you didn't use any customer information, besides customer count, when forecasting customer usage. Plotting Usage per Customer versus Temperature, it is not uncommon to see dramatically different patterns for different years. Have you made any attempts to incorporate meter reads or service class information?
How did you validate the forecast of tenured customers? The per customer usage on a future date is a combination of the current tenured customer (minus attrition) and new customers.
Thanks,
Andrew
Hi Andrew,
ReplyDeleteExcellent questions.
On customer information - this paper is all about residential load. If industrial and small commercial customers were in the mix and have a significant share of the total load, I would try to forecast them separately. We didn't have detailed and reliable customer information about these residential customers, so we did not try to breakdown the residential customers by size of household or zipcode, etc. The only thing we did was to break them down to different contract size as discussed in the paper.
On validation of tenured customer forecast - we did have start and end date of each account (both currently active ones and the previously closed ones). If we take any two snapshots of the historical data, we know there are X customers at the first snapshots. And we know which of these X customers are still tenured at the second snapshot based on the start and end date information. In addition, the trading strategy we illustrated in the paper helps mitigate the problems with newly acquired customers.
Hopefully this helps.