Seven Lessons Learned from Two Plagiarism Cases

Recently, I was presented two plagiarism cases in one month, which triggered this blog post.

Case #1

Research group A used an idea proposed by research group B a few years ago to publish two papers in a flagship journal. 

In the first paper, A applied B's method to an application where B applied the same method, but A did not cite B's original paper at all. In the second paper, A applied B's method to a different application. A cited the original paper where B proposed that method, but did not clearly specify that the method was first proposed by B. Instead, A only cited how B commented on the previous work, which was the motivation of B's original idea. In other words, by reading A's second paper, a reader would consider that the method was originally proposed by A. 

Since the two papers were published almost at the same time. It's clear that A was aware of B's work, but did not give proper credit to the original paper. The improper citation misled the editors and reviewers. 

I have no intention to defend the editors and reviewers who handled those two papers in the peer review process. Their irresponsibility was part of the problem! In fact, both papers are good papers if the authors had properly cited the literature. At least the second one deserved to be published by that flagship journal.

My recommendation to A was to retract both papers, and to apologize to B. 

Case #2

Last year, I worked on a proposal with a few collaborators, including the Lead PI (A), PI (B), myself, and a few others. In our proposal, we used B's idea, which he published in a paper for a different application. The proposal was rejected by the funding agency. A decided to complete the research and publish a paper with us, so she sent the proposal to her student (C) to continue the work.

A few weeks ago, A sent the manuscript to us coauthors, and told me that C completed the research, and the results looked very promising. I quickly glanced through the manuscript and found the list of references very disorganized, so I asked A to work with C to re-do the literature review. In addition, I found the results a bit fishy. The proposed method dominated its counterparts with a landslide win. I thought it's too good to be true (see THIS POST about my smoke tests). I asked A to revise the paper and check the results.

After some investigation, A told me that C manipulated the computational results to make the benchmark models look bad. She has asked C to present the full picture.

Last week, I received the revised manuscript. I briefly read through the manuscript, but still didn't like how references are being cited. In addition, I felt some sentences read familiar. I asked A to work with C to further enhance the reference list, and to validate that this manuscript did not copy sentences from other papers.

Yesterday, A told me that she found the method proposed in this draft identical to the method in B's paper. She was pissed off, because C told her that the idea was original. Moreover, the manuscript never specified that the same method was used in a different application. Again, the improper citation misled A. With such frustration, A wanted to kill the manuscript.

My recommendation to A was to properly cite B's original paper, and submit the manuscript to a first-tier journal.

Lessons learned

1. Plagiarism is defined as "the practice of taking someone else's work or ideas and passing them off as one's own." 
2. Always give proper credit to the prior research by citing the papers in the right places!
3. Every co-author should understand and be able to defend every piece of the paper. 
4. Every reviewer and editor in the peer review process should carefully review the assigned paper.
5. Do damage control ASAP. Don't wait!
6. Don't over-react to plagiarism. 
7. Make preventive actions to avoid plagiarism in the future. 

In the next blog post, I will further explain what "novelty" really means in the academic literature.

Benchmarking Robustness of Load Forecasting Models under Data Integrity Attacks

GIGO - garbage in, garbage out. In forecasting, GIGO means that if the model is fed with garbage (bad) data, the forecast would be bad too. In the power industry, bad load forecasts often result in waste of energy resources, financial losses, brownouts or even blackouts.

Anomaly detection and data cleansing procedures may help alleviate some of the bad data from the input side. However, what if the bad data was created by hackers? Can the existing models "survive" or stay accurate under data attacks? This paper offers some benchmark results.

This paper sets a few "first":

1. This is the first paper formally addressing the cybersecurity issues in the load forecasting literature. I believe that the data attacks should be of a great concern to the forecasting community. This paper sets a solid ground for future research. 
2. This is my first journal paper co-authored with a professor in my doctoral committee, Dr. Shu-Cherng Fang. Many years ago, I picked up the topic of my dissertation from one of my consulting projects. I then invited a team of world class professors from different areas to form the committee. None of them were really into load forecasting, though I had the opportunities learning from different perspectives. 
3. This is my first journal paper that went through one year of peer review cycle, the longest peer review I've experienced. It's definitely worth the effort. The IJF editors and reviewers certainly spent a significant amount of time reading the paper and offered so many constructive comments. I wish I could know their names and identify them in the acknowledgement section.

Citation

Jian Luo, Tao Hong and Shu-Cherng Fang, "Benchmarking robustness of load forecasting models under data integrity attacks", International Journal of Forecasting, accepted. (working paper)

Benchmarking robustness of load forecasting models under data integrity attacks

Jian Luo, Tao Hong and Shu-Cherng Fang

Abstract

As the internet continues to expand its footprint, cybersecurity has become a major concern for the governments and private sectors. One of the cybersecurity issues is on data integrity attacks. In this paper, we focus on the power industry, where the forecasting processes heavily rely on the quality of data. The data integrity attacks are expected to harm the performance of forecasting systems, which greatly impact the financial bottom line of power companies and the resilience of power grids. Here we reveal how data integrity attacks can affect the accuracy of four representative load forecasting models (i.e., multiple linear regression, support vector regression, artificial neural networks, and fuzzy interaction regression). We first simulate some data integrity attacks by randomly injecting some multipliers that follow a normal or uniform distribution to the load series. Then the aforementioned four load forecasting models are applied to generate one-year ahead ex post point forecasts for comparisons of their forecast errors. The results show that the support vector regression model, trailed closely by the multiple linear regression model, is most robust, while the fuzzy interaction regression model is least robust among the four. Nevertheless, all of the four models fail to provide satisfying forecasts when the scale of data integrity attacks becomes large. This presents a serious challenge to the load forecasters and the broader forecasting community: How to generate accurate forecasts under data integrity attacks? We use the publicly-available data from Global Energy Forecasting Competition 2012 to construct the case study. At the end, we also offer an outlook of potential research topics for future studies.

Breakthrough or Too Good To Be True: Several Smoke Tests

When sharing my Four Steps to Review an Energy Forecasting Paper, I spent about a third of the blog post elaborating what "contribution" means. This post is triggered by several review comments to my recent TSG paper variable selection methods for probabilistic load forecasting. Here I would like to elaborate what "contribution" means from a different angle.

7 Reasons to Attend ISEA2017

The International Symposium on Energy Analytics (ISEA2017) is coming in 7 weeks. If you are still wondering whether you should join the event or not, here are 7 reasons for you to attend ISEA2017:

1. Grow your international network

ISEA2017 is truly international. The early registrations came from 16 countries. As a conference attendee, you will hear 20+ presentations describing methodologies and insights gained from various places in the world. You will also share your experience and expertise with this diverse audience and get their critique and compliment.

2. Check out the winning methods of GEFCom2017

Selected GEFCom2017 teams will be presenting their methodologies at ISEA2017. You will witness the recognition of GEFCom2017 winners and have the face-to-face discussion with them. Rather than reading thousands of energy forecasting papers published every year and wondering which ones work well, you can grasp the secret sauce of the most effective methods during ISEA2017.

3. Experience a novel peer review process 

Whether we like today's peer review system or not, we have to live with it, at least for the next few years before a better one is in place. We have tied ISEA2017 to an IJF special section on energy forecasting, where we try to implement a new peer review process. The ISEA2017 attendees will have the opportunity to experience this new process and help improve it.

4. Peek and shape the future of energy analytics 

If you are struggling with the topic for your next paper, ISEA2017 is a must-attend conference for you. We will discuss the emerging topics as well as the research agenda for the future. Rather than guessing where the future goes, you can contribute to the plan!

5. Attend International Symposium on Forecasting

The 37th International Symposium on Forecasting (ISF) will be held two days after ISEA2017, right at the same location. ISF is the only major scientific forecasting conference I know of. I find it very rewarding to attend ISF, where I hear forecasting topics from various industries, as well as the methodological breakthroughs in general. Many of them could be applied to the energy forecasting problems. Extending the trip to include ISF in your travel plan would be a wise choice.

6. Two world heritage sites in one place

The World Heritage Centre has a list of about 1000 world heritage sites around the globe. Two of them (Great Barrier Reef and Daintree Rainforest) are in Cairns, Australia, making Cairns the only place in this planet with two world heritage sites side by side. ISF organizers have planned the social program including numerous social events and tour opportunities for delegates, their friends and family.

7. Low registration fees

Our sponsors, the International Institute of Forecasters, Tangent Works and Journal of Modern Power Systems and Clean Energy, have generously contributed to the organization of ISEA2017, helping significantly subsidize the registration fees. If you attend both ISEA and ISF, there is an additional discount. To register both ISEA and ISF, click HERE. To register ISEA only, click HERE.

ISEA2017 will be held in Cairns, Australia, June 22-23, 2017. Look forward to seeing you there!

7 Reasons to Send Your Best Papers to IJF

Last week, I was surfing the Web of Science to gather some papers to read during the holidays. Yes, some poor professors like myself work 24x7, including holidays. Suddenly I found that FIVE of my papers are listed by the Essential Science Index (ESI) as Highly Cited Papers. (Check them out HERE!) What a good surprise for Thanksgiving :)

What's even more surprising is that all of these five papers were published by the International Journal of Forecasting! As an editorial board member of two very prestigious and highly ranked journals, IEEE Transactions on Smart Grid (TSG) and International Joirnal of Forecasting (IJF), I send my best papers to these two journals every year, with an even split. So far, I've had six papers in TSG (not counting two editorials) and six in IJF. How come only my IJF papers were recognized by ESI?

The curiosity ate most of my Thanksgiving time. I was doing some research to answer this question, which eventually led to this blog post. In short,

you should send your best energy forecasting papers to IJF first!

Here is why:

1. No page limit. IJF does not charge authors for extra pages. You can take as many pages as you like to elaborate your idea. The longest IJF paper I've read is Rafal Weron's 52-page review paper on price forecasting. My IJF review on probabilistic load forecasting is 25 pages long. Both reviews are now ESI Highly Cited Papers. 
2. Short review time. A manuscript first reaches EIC, editor and then Associate Editor. It may be rejected by any of these three people. In other words, if it is a clear rejection, the decision would be coming to you rather quickly. If the manuscript is assigned to the reviewers, the first decision typically comes back within three to four months. 
3. Very professional comments. I have seen many IJF review reports by far, as an author, reviewer and editor. Most of them are very professional. Eventually these review comments help the authors improve their work. I haven't seen any nonsense reviewer in the IJF peer-review system, which is quite remarkable! I guess the editors have done their job well by filtering out the nonsense reviewers before passing the comments to the authors. 
4. High quality copy-editing service free of charge. Once the manuscript is accepted, it will be forwarded to a professional copy editor to polish the English for free, so you don't need to spend too much time with wordsmith. You don't need to worry about formatting either, because there is another copy editor doing that before the publisher sends you the proof. 
5. Bi-annual awards. Every other year, IJF awards a prize for the best paper published in a two-year period. The prize is $1000 plus an engraved plaque. Details of the most recent one can be found HERE. Making some money and getting recognized for your paper, isn't it nice? 
6. Publicity. Six years ago when I was pursuing my PhD, I was frustrated about the many useless papers in the literature. I brought my frustration to David Dickey. He made a comment that shocked me for a while. Instead of encouraging me to publish, he said that he had lost interest in publishing papers, because "the excellent papers are often buried by so many bad ones". Having been a professor for about three years, I have to agree with him. I believe in the era of "publish or perish", we have to "publish and publicize" to make our papers highly cited. Publishing your energy forecasting papers with IJF means that you get the opportunity of leveraging various channels, such as Hyndsight, Energy Forecasting, and the social media accounts of Elsevier and those renowned IJF editors. 
7. "Business and economics" category in ESI. This is probably the most important distinction between IEEE Transactions and IJF. Many IEEE Transactions papers (including the ones in TSG) are grouped into engineering, while IJF papers are in the category of business and economics. The business and economics papers get much fewer citations on average than the engineering ones, which makes the ESI thresholds of business and economics lower than those of engineering. For instance, my TSG2014 paper is not an ESI paper, but it would have been if it were published by IJF. 

Unfortunately, IJF's acceptance rate is very low. To increase the chance to have the paper accepted, you should understand how reviewers evaluate the manuscript.

Look forward to your next submission!

Temperature Scenario Generation for Probabilistic Load Forecasting

When using weather scenarios to generate probabilistic load forecasts, a frequently asked question is

How many years of weather history do we need? 

This paper gives an answer based on an empirical study.

Most of my papers were accepted after two or more revisions. This time it only took one revision to have this paper accepted. In the first round of review, We received 40 comments from 6 reviewers. Our first revision was accepted after 4 of the reviewers recommended acceptance. In this blog post, I'm attaching the submitted version of the revision including our response letter. Some of our responses were rebuttals to one of the reviewers who made a personal attack on me.

Citation

Jingrui Xie and Tao Hong, "Temperature scenario generation for probabilistic load forecasting", Transactions on Smart Grid, accepted.

The working paper is available HERE.

Temperature Scenario Generation for Probabilistic Load Forecasting

Jingrui Xie and Tao Hong

Abstract

In today’s dynamic and competitive business environment, probabilistic load forecasting (PLF) is becoming increasingly important to utilities for quantifying the uncertainties in the future. Among the various approaches to generating probabilistic load forecasts, feeding simulated weather scenarios to a point load forecasting model is being commonly accepted by the industry for its simplicity and interpretability. There are three practical and widely used methods for temperature scenario generation, namely fixed-date, shifted-date, and bootstrap methods. Nevertheless, these methods have been used mainly on ad hoc basis without being formally compared or quantitatively evaluated. For instance, it has never been clear to the industry how many years of weather history is sufficient to adopt these methods. This is the first study to quantitatively evaluate these three temperature scenario generation methods based on the quantile score, a comprehensive error measure for probabilistic forecasts. Through a series of empirical studies on both linear and nonlinear models with three different levels of predictive power, we find that 1) the quantile score of each method shows diminishing improvement as the length of available temperature history increases; 2) while shifting dates can compensate short weather history, the quantile score improvement gained from the shifted-date method diminishes and eventually becomes negative as the number of shifted days increases; and 3) comparing with the fixed-date method, the bootstrap method offers the capability of generating more comprehensive scenarios but does not improve the quantile score. At the end, an empirical formula for selecting and applying the temperature scenario generation methods is proposed together with a practical guideline.  

Relative Humidity for Load Forecasting Models

The ultimate driver of using big data for predictive modeling and forecasting, in my opinion, is customization. Obviously, such customization can be reflected by providing special treatments to individual regions in a territory and individual hours of a day, as discussed in my recent IJF paper Electric Load Forecasting with Recency Effect: A Big Data Approach. In this paper, we are taking another big data approach to load forecasting by breaking a composite variable Heat Index. We show that the NWS' formula for Heat Index is not really designed for load forecasting.

The paper went through three rounds of reviews with 5 reviewers. Most reviewers were very good at providing helpful comments. Only one reviewer raised a few interesting but naive comments. I didn't bother to please him/her by revising my paper. Nevertheless, I would love to list them here so that other authors can use my argument to respond to similar comments.

1. "Twenty-five papers and previous works are cited in this paper, with exactly 54 references along the paper. On these 54 times, the authors are citing themselves at least 31 times" [...]"This behavior tends to give the reader a strange feeling and do not give a lot of credit to your work: it is hard to value it since you mostly compare yourself to.... yourself. You need to justify it properly before claiming these kind of affirmations."

Dear reviewer,
Unfortunately you are having a "strange" feeling. I believe that this is mostly due to the fact that you are not very much aware of the recent academic literature and field practice on load forecasting. Maybe you should go to google "load forecasting", "electric load forecasting" or "energy forecasting", and see how my work shows up among the top three entries on Google's first page. THIS POST discusses my way of citing references. The papers on my reference list are carefully picked based on relevance and quality.The quality is mainly determined by whether the work is being used by the industry or not.  So far, my readers have been very pleased with the useful references I've listed on my papers. Oh, you might be pissed off by my not citing (enough) of your papers. In order for me to cite more of your papers, you should write more high quality papers to show how your work is being valued by the industry.

In my first submission, there were 25 references, of which 9 were my own papers. We carefully considered the reviewer's comments, and increased the number of references to 34 in the final submission, of which 12 papers were mine.

2. "All the models in the paper are derivatives of the Vanilla's Tao Benchmark. It has been shown that this model can be outperformed by a significant margin by state of the art models (see GEFCOM2012 results). Is it useful to use such models (and to finally improve it by max 9%) while GEFCOM2012 results show that some models can improve its forecast by almost 40%."

Dear reviewer,
Take another look at Table I please. That 5.21% were from the Vanilla model. The MAPE of model B4 without humidity is already down to 3.79%. Our proposed model is at 3.62%. This is more than 30% improvement over the Vanilla model. Here we did not even add holiday effect to the models. On the other hand, please take a look at this paper Weather Station Selection for Electric Load Forecasting. Are you wondering why the entire paper is based on the Vanilla model? Why didn't I even add recency effect? It is because the proposed methodology can also be applied to more complicated models. To avoid verbose presentation and distraction from the main theme of a paper, we can show the results on a benchmark model.

3. "It is well known that multicolinearity is very bad in MLR models and can lead to instability of parameters and false results. It would be nice to have an idea of estimated parameters for the different variables and models, and to exhibit significance results, tests, etc."

Dear reviewer,
Please read some papers in the load forecasting literature, and see how often people are using lagged variables (both load and temperatures). Maybe my recent IJF paper on recency effect can totally piss you off. Are you wondering why those papers are using these highly correlated variables? It is because we have so many observations in load forecasting. BTW, please do not show those significant tests, such as p-value, in your papers. They are useless in load forecasting. Again, we have so many observations that the residuals are rarely normally distributed. Moreover, those p-values are from in-sample fit, which tells nothing about the predictive power of your models. Furthermore, there are hundreds of parameters being estimating in a load forecasting model, how do you plan to show the significant tests of those variables? You may want to read Scott Armstrong's paper on Illusions in Regression Analysis to re-examine your understandings in regression analysis.

Citation

Jingrui Xie, Ying Chen, Tao Hong and Thomas D. Laing, "Relative humidity for load forecasting models", IEEE Transactions on Smart Grid, in press.

The working paper is available HERE.

Relative Humidity for Load Forecasting Models

Jingrui Xie, Ying Chen, Tao Hong and Thomas D. Laing

Abstract

Weather is a key driving factor of electricity demand. During the past five decades, temperature is the most commonly used weather variable in load forecasting models. Although humidity has been discussed in the load forecasting literature, it has not been studied as formally as temperature. Humidity is usually embedded in the form of Heat Index (HI) or Temperature-Humidity Index (THI). In this paper, we investigate how Relative Humidity (RH) affects electricity demand. From a real-world case study at a utility in North Carolina, we find that RH plays a vital role in driving electricity demand during the warm months (June to September). We then propose a systematic approach to including RH variables in a regression analysis framework, resulting in the recommendation of a group of RH variables. The proposed models with the recommended addition of RH variables improve the forecast accuracy of Tao’s Vanilla Benchmark Model and its three derivatives in one-day (24-hour) ahead, one-week ahead, one-month ahead and one-year ahead ex post forecasting settings, with the relative reduction in Mean Absolute Percentage Error (MAPE) ranging from 4% to 9% in this case study. It also outperforms two HI based models under the same settings.  Moreover, an extended test case also demonstrates the effectiveness of these RH variables on improving the Artificial Neural Network models.

Improving Short Term Load Forecast Accuracy via Combining Sister Forecasts

Over the past three years, I have had 14 papers published/accepted by reputable journals. These papers are from about 40 original and revision submissions, about two revisions per finally accepted paper. Among the 40 some decision letters I received in the recent three years, four were rejections. Three of the four rejections were on this paper. I'm summarizing the journey of this paper in this post, to shed some lights on the peer review process for the other authors like myself living in this broken system.

The original work was from the thesis of one of my MS students. I first submitted a two-page letter to PES Letter in February 4, 2015, and got a rejection after four months, mostly due to lack of details. The working paper is HERE. Although the reviews were not much helpful, I think the rejection was fair.

After the PESL submission, I worked with Rafal Weron and two students to expand the two-page letter to a full paper. I then submitted it to IEEE Transactions on Power Systems, and got a rejection after about 10 weeks. We received the comments from three reviewers:

• Reviewer #1 wrote 6 sentences. One sentence was summarizing the work presented in our paper. One sentence was "The submitted manuscript is relatively well written and the references list is good.". Three sentences were repeating themselves "This manuscript does not present any novelty." The other sentence was "Load forecast is easy. Price forecast is very difficult!". I had no idea why the review made such an irrelevant comment...
• Reviewer #2 confirmed the value of our paper, and asked a list of clarification questions. Most of the comments were quite helpful. 
• Reviewer #3 wrote two pages of comments, or 1106 words, trying his/her best to kill our paper. What amazed me was that after reading the comments multiple times, I wasn't sure if the comments were to our paper. The review also used the term "garbage writings" to criticize our paper, which reminded me one of the 10 reviewers who rejected my papers 5 years ago. That reviewer also liked to use the terms "garbage writings" and "garbage tables" in the review report. 

Being speechless about the decision letter, we revised the paper a bit and resubmitted it to Applied Energy. Applied Energy was not my choice, because the journal has an awful reputation of manipulating the impact factor through self-citation. I did not want to associate my papers with a fishy journal. Since Rafal took the role of corresponding author this time, I respected his decision. The paper was rejected within a week by the editor-in-chief for the reason "out-of-scope". Recalling that I just reviewed some load forecasting papers, and even probabilistic load forecasting papers for the journal, I have to say that "out of scope" is equivalent to "not citing enough papers from Applied Energy". It looks like the journal is still playing the same game to keep up its impact factor.

After getting three rejections, Rafal decided to send the paper to Energy. The first decision came back in three months asking for a major revision. There comments were from three reviewers. One reviewer confirmed the contribution and pointed out a typo. Another reviewer suggested us including a machine learning method for forecast combination. The other reviewer copied exactly the same comments from TPWRS' Reviewer #3! While the TPWRS Reviewer #3's comments were not much relevant to our TPWRS submission, they are even more irrelevant to our EGY version.

In our revision, we rebutted the comments from that familiar reviewer, and revised the paper according to the comments from the other two reviewers. The second decision letter came back in about six weeks asking for another major revision. This time the other two reviewers were pleased. That familiar reviewer was making some more nonsense comments, and said "I will give authors one more time to revise their paper..."

We again rebutted all of the comments from this last reviewer, and submitted the paper on December 28, 2015. The acceptance letter came back on New Year's Eve. I guess that nonsense reviewer was finally kicked out by the editor.

Citation

Jakub Nowotarski, Bidong Liu, Rafał Weron, Tao Hong, Improving short term load forecast accuracy via combining sister forecasts, Energy, Volume 98, 1 March 2016, Pages 40-49,

You can access to this paper for free by March 24, 2016 through THIS LINK.

Improving Short Term Load Forecast Accuracy via Combining Sister Forecasts

Jakub Nowotarski, Bidong Liu, Rafał Weron, Tao Hong

Abstract

Although combining forecasts is well-known to be an effective approach to improving forecast accuracy, the literature and case studies on combining electric load forecasts are relatively limited. In this paper, we investigate the performance of combining so-called sister load forecasts, i.e. predictions generated from a family of models which share similar model structure but are built based on different variable selection processes. We consider 11 combination algorithms (three variants of arithmetic averaging, four regression based, one performance based method and three forecasting techniques used in the machine learning literature) and two selection schemes. Through comprehensive analysis of two case studies developed from public data (Global Energy Forecasting Competition 2014 and ISO New England), we demonstrate that combing sister forecasts outperforms the benchmark methods significantly in terms of forecasting accuracy measured by Mean Absolute Percentage Error. With the power to improve accuracy of individual forecasts and the advantage of easy generation, combining sister load forecasts has a high academic and practical value for researchers and practitioners alike. 

Where is My Dissertation?

I got my PhD with zero journal paper published or accepted. Since then, I have been wondering how many journal papers can eventually be published out of my dissertation. I think today, the end of the first five-year period, is a good time to check it out.

The list below maps the original chapters and sections to my papers:

• Ch1 Introduction  >>  (Hong, 2014)
• Ch2 Literature review  >>  (Hong and Fan, 2016)
• Sec4.1 Benchmark  >>  (Hong, Pinson and Fan, 2014)  >> (Hong, Wang and White, 2015)
• Sec4.2.2 MTLF/LTLF  >>  (Hong, Wilson and Xie, 2014)  >> (Xie, Hong and Stroud, 2015) & (Xie, Hong, Laing and Kang, 2015)
• Sec4.3.1 Recency effect  >>  (Wang, Liu and Hong, 2015)  >> (Liu, Nowotarski, Hong and Weron, 2015)
• Ch5 Possibilistic load forecasting  >>  (Wang and Hong, 2014)

The dissertation itself has been cited 41 times according to GoogleScholar. 

As I mentioned in the blog post (The Gap between Academic Research and Industry Needs) two years ago, the first two papers written from my dissertation materials were rejected by a journal in 2010. I got negative comments from 10 reviewers, of which most are nonsense. One of the reviewers loved to use the word "garbage" in the review comments. 

A couple of months ago, I got another rejection from the same journal. The similar garbage-style review comments showed up again. Shame on me - I should have stopped sending papers to this journal. Instead of yelling back to the editor-in-chief, I stayed calm this time and moved on, with the hope that the journal can clean up the garbage reviewers.

At least, the editor and editor-in-chief should do their homework: clean up the garbage comments before sending them out to the authors. 

Language for Rejecting Energy Forecasting Papers

As a volunteer in the peer review system, I'm handling (as an editor & editor-in-chief) and reviewing (as a reviewer) at least 10 papers per month. While I try to perform thorough review for each paper following these four steps, the consequence is that I spend more time helping others improve and publish papers than working on my own papers. As the editorial workload keeps growing, I have been trying to figure out a more efficient way of performing editorial services. One phenomenon I have observed is that most papers (about 80% or more) get rejected at the end. Most of my editorial services are for top journals such as TSG and IJF, of which the rejection rate is even higher than 80%. In fact many papers share similar reasons for rejection. I guess one way is to standardize the rejection language, so that I don't have to type the same reasons each time. To save time for other editors and reviewers in the same peer review system, I would be more than happy to share the following rejection language.

Manuscript Review Time

If you ever submit a paper to a journal, most likely your contact information is listed in the pool of reviewers. If your paper gets published, most likely you will be invited to review some other papers in the similar area. If you have published several good papers in the same area or you have submitted top quality review reports, the likelihood of your getting review requests can be very high. In my Four Steps to Review an Energy Forecasting Paper, I indicated that it takes me on average 4 to 8 hours to review a manuscript for a journal. This blog post tries to answer another question:

How much elapsed time should it take a reviewer to prepare the review report?

Six Tips on Citing References in a Research Paper

I thought my first winter holiday on the academic job would have been very relaxing. Somehow I started the holiday by doing review and editorial work for at least 10 manuscripts. After reviewing half of them, I had a strong desire to write a blog post about how to cite references in a research paper. Below are six tips from me written on a Christmas Eve. Hopefully they are helpful to the students and scholars who are struggling with the list of references.

Four Steps to Review an Energy Forecasting Paper

Despite criticism, the peer review system is still widely used by most scholarly journals. Having been through the process as an author, reviewer and editor, I would love to share my four steps to review a paper, particularly in the area of energy forecasting.

1. Shall I review it?
I don't review every paper assigned to me. Under the following circumstance, I will turn down a review request after browsing through its abstract or the entire paper:

Three Years after my PhD Defense: the Gap between Academic Research and Industry Needs

It has been three full years since I defended my dissertation. There has been a lot going on during these three years. I would like to put some of my random thoughts into a few blog posts starting from this one:

The Gap between Academic Research and Industry Needs.