As I stated in my last post, I wanted to write something about how my last astronomical paper went through the publication process. As promised, here is the second part of that post.
Part II
A question often asked by a scientist's friends and family is 'what do you actually do?' A pretty fair question really, especially as many of us only get to be career scientists through the support of the government and thus the tax payer. The truth is that, most often, scientists are writing papers. A (largely) complete description of an experiment, the results and some analysis/summary based on those results. Scientists are always writing at least one paper because this is what we are really judged on, really, to a degree that would probably astonish you.
Getting these papers published means submitting them to peer review, something pretty much any scientist can tell you is a fairly painful process. We have all at some time created a work of art, a piece of writing, a meal or anything at all that is then subjected to some level of criticism. For some people this is a more personal and profound experience than for others. It is easy to imagine that a painter or playwright may feel quite affected by a series of poor reviews of their work. It may be somewhat harder to imagine that scientists often go through a similar experience.
The process of peer review has grown from the earliest ideas that any judgement may be best done by a collection of your peers. In the 18th century this idea began to be applied to the publication of scientific results. As early as 1731 scientific results were being distributed amongst individuals deemed by the editor to be worthy to evaluate them. The idea being not so much that peer review could police fraudulent results or the like but simply that a filter was in place to help guide the editors towards relevant and interesting results. It is almost certainly a coincidence but only one year earlier a tightening of control over judicial peer review was also put into place.
The utopian model of peer review is one in which the journal you want your paper published in is run by a 'benevolent dictator' of an editor. This editor would be all-knowing, all-wise, fair, ethical and, above all has the time and energy to read and evaluate my paper along with the thousands of others submitted every year. This actually wasn't far from the case when Albert Einstein himself published his 'Annus Mirablis' papers in 1905. Each of Einstein's 'Miracle Year' papers was read and reviewed by Max Planck himself, the associate editor of Annalen der Physik at the time. Max Planck was the genius physicist on whose work Einstein had based his first paper and who would later win the Nobel Prize himself.
These days papers are doled out by the editors (or sub-editors) to referees whom they deem to be experts in the field. I have received papers myself to review, although that is another story for another time, and I don't intend to criticise this process. After all, Churchill's quote on democracy is particularly apt here in that peer review may arguably be a horrible process but it does have the advantage of being better than any others that have been tried.
Once an article has been placed with a referee, that referee evaluates it and submits a report, along with one of three recommendations.
As I alluded to in my last post, my most recent paper took a great deal of time to publish. The reasons for that are many, some of which are my own fault, some are just happenstance and some are the fault of the journal (in my opinion). I submitted the paper just before Christmas 2012 with high hopes that this paper would draw a little attention, enough perhaps that I might be able to score a job interview or two off the back of it. I had certainly invested enough time and effort into it and, at over 40 pages at the time it was hefty enough to be recognised as a solid body of work.
The paper came back with a substantial referee report. We took this report seriously and worked through the suggested revisions, resubmitting the paper in early April. At this point I was still hopeful that the paper would get out quickly and be useful in my ongoing job-hunting. A further referee's report came back, again fairly substantial, although this time the report asked only for some moderate revisions. The editor told us at this point that the referee was happy with the paper assuming that we took care of the second set of revisions and that they didn't want to review it a third time. We received assurance from the editor that, so long as we were conscientious about our revisions, the paper would be accepted and published with our next submission.
So, thinking we were on the home straight, I took care of the changes and resubmitted the paper. At this point the editor, without consulting us, forwarded the manuscript to another referee altogether. This was unexpected, especially considering that the editor had specifically told us this wouldn't happen. We were a bit taken aback by this turn of events but not particularly worried, given that we had already been through two revisions of the paper.
This is where the random and capricious nature of the peer review process struck. The second (anonymous) referee rejected our paper outright with a fairly cursory explanation as to why. Given that the paper now represented over two years of work on my part I was understandably upset. Challenging this resulted in a rather hostile statement that the paper did not present results worthy of publication. This seemed a rather odd thing to say after the paper had already been under review for eight months and had been essentially accepted, even praised by the first referee. We were able to win an agreement to send the paper to a third, deciding referee. This referee was sympathetic to both previous reports and suggested re-writes of their own to make the paper somewhat more concise. At this point the paper had been 'in the works' for nearly eight months and I was in the middle of a move (both in career and location) and my wife's pregnancy. All of the impetus of the paper had been taken away and so it took a good deal of time to make yet another set of revisions and resubmit the paper. This was done 13 months after its first submission, when it was finally accepted.
I hope that this kind of experience is rare but I do believe that the journals and editors need to look carefully at their processes. In a field in which publication is so vital to the careers of the submitting authors it seems irresponsible to take less than the utmost care and diligence in publishing their works. In the scientific realm scientists are supposed to be above reproach when presenting their results. Falsifying data is one of the most heinous academic crimes possible and, along with plagarism, about the only things one can do to have their PhD revoked. I think that editors should be held to similar standards and that misleading authors and/or delaying the publication of results (intentionally or not) are arguably just as serious.
I'm not sure of the point of blogging this story, maybe it's just to complain about the peer review process. Perhaps I just want to get a dig in at the editors that held up my paper when my attention should have been elsewhere. In the end I'm just trying to give an impression of some of the pain that a scientist goes through when trying to get their work published. It is not an easy process and, akin to writing or other artistic processes is rife with criticisms and other obstacles. It seems that the public's perception of science is that it has a clear answer, things are right or they are wrong. If this were the case then publishing scientific results would be a much easier process, a paper would be right or wrong and could be published or rejected. As it is, it often comes down to a friendly or hostile editor combined with a friendly or hostile referee (or a majority of such!) to decide whether or not you will be published and how quickly. When your career can depend on these decisions it might be just as well to let the criticisms go and, like any endeavour, have faith in your work. You will know the effort that you have poured into it and you will know what that is worth, even if others don't always recognise it.
Part II
A question often asked by a scientist's friends and family is 'what do you actually do?' A pretty fair question really, especially as many of us only get to be career scientists through the support of the government and thus the tax payer. The truth is that, most often, scientists are writing papers. A (largely) complete description of an experiment, the results and some analysis/summary based on those results. Scientists are always writing at least one paper because this is what we are really judged on, really, to a degree that would probably astonish you.
Getting these papers published means submitting them to peer review, something pretty much any scientist can tell you is a fairly painful process. We have all at some time created a work of art, a piece of writing, a meal or anything at all that is then subjected to some level of criticism. For some people this is a more personal and profound experience than for others. It is easy to imagine that a painter or playwright may feel quite affected by a series of poor reviews of their work. It may be somewhat harder to imagine that scientists often go through a similar experience.
The process of peer review has grown from the earliest ideas that any judgement may be best done by a collection of your peers. In the 18th century this idea began to be applied to the publication of scientific results. As early as 1731 scientific results were being distributed amongst individuals deemed by the editor to be worthy to evaluate them. The idea being not so much that peer review could police fraudulent results or the like but simply that a filter was in place to help guide the editors towards relevant and interesting results. It is almost certainly a coincidence but only one year earlier a tightening of control over judicial peer review was also put into place.
The utopian model of peer review is one in which the journal you want your paper published in is run by a 'benevolent dictator' of an editor. This editor would be all-knowing, all-wise, fair, ethical and, above all has the time and energy to read and evaluate my paper along with the thousands of others submitted every year. This actually wasn't far from the case when Albert Einstein himself published his 'Annus Mirablis' papers in 1905. Each of Einstein's 'Miracle Year' papers was read and reviewed by Max Planck himself, the associate editor of Annalen der Physik at the time. Max Planck was the genius physicist on whose work Einstein had based his first paper and who would later win the Nobel Prize himself.
These days papers are doled out by the editors (or sub-editors) to referees whom they deem to be experts in the field. I have received papers myself to review, although that is another story for another time, and I don't intend to criticise this process. After all, Churchill's quote on democracy is particularly apt here in that peer review may arguably be a horrible process but it does have the advantage of being better than any others that have been tried.
Once an article has been placed with a referee, that referee evaluates it and submits a report, along with one of three recommendations.
- Firstly, that the journal should accept the article 'as is'; this does actually happen, although extremely rarely.
- Secondly, that the article should be accepted, though some revisions should be made first. This is the most common result in my experience, particularly when this recommendation is subdivided further into decisions ranging from 'major revisions with subsequent review' to 'minor revisions with no subsequent review'.
- Finally, an article may be outright rejected if it is considered to contribute nothing of any publishable value and/or be so poorly written that a complete rewrite is necessary.
As I alluded to in my last post, my most recent paper took a great deal of time to publish. The reasons for that are many, some of which are my own fault, some are just happenstance and some are the fault of the journal (in my opinion). I submitted the paper just before Christmas 2012 with high hopes that this paper would draw a little attention, enough perhaps that I might be able to score a job interview or two off the back of it. I had certainly invested enough time and effort into it and, at over 40 pages at the time it was hefty enough to be recognised as a solid body of work.
The paper came back with a substantial referee report. We took this report seriously and worked through the suggested revisions, resubmitting the paper in early April. At this point I was still hopeful that the paper would get out quickly and be useful in my ongoing job-hunting. A further referee's report came back, again fairly substantial, although this time the report asked only for some moderate revisions. The editor told us at this point that the referee was happy with the paper assuming that we took care of the second set of revisions and that they didn't want to review it a third time. We received assurance from the editor that, so long as we were conscientious about our revisions, the paper would be accepted and published with our next submission.
So, thinking we were on the home straight, I took care of the changes and resubmitted the paper. At this point the editor, without consulting us, forwarded the manuscript to another referee altogether. This was unexpected, especially considering that the editor had specifically told us this wouldn't happen. We were a bit taken aback by this turn of events but not particularly worried, given that we had already been through two revisions of the paper.
This is where the random and capricious nature of the peer review process struck. The second (anonymous) referee rejected our paper outright with a fairly cursory explanation as to why. Given that the paper now represented over two years of work on my part I was understandably upset. Challenging this resulted in a rather hostile statement that the paper did not present results worthy of publication. This seemed a rather odd thing to say after the paper had already been under review for eight months and had been essentially accepted, even praised by the first referee. We were able to win an agreement to send the paper to a third, deciding referee. This referee was sympathetic to both previous reports and suggested re-writes of their own to make the paper somewhat more concise. At this point the paper had been 'in the works' for nearly eight months and I was in the middle of a move (both in career and location) and my wife's pregnancy. All of the impetus of the paper had been taken away and so it took a good deal of time to make yet another set of revisions and resubmit the paper. This was done 13 months after its first submission, when it was finally accepted.
I hope that this kind of experience is rare but I do believe that the journals and editors need to look carefully at their processes. In a field in which publication is so vital to the careers of the submitting authors it seems irresponsible to take less than the utmost care and diligence in publishing their works. In the scientific realm scientists are supposed to be above reproach when presenting their results. Falsifying data is one of the most heinous academic crimes possible and, along with plagarism, about the only things one can do to have their PhD revoked. I think that editors should be held to similar standards and that misleading authors and/or delaying the publication of results (intentionally or not) are arguably just as serious.
I'm not sure of the point of blogging this story, maybe it's just to complain about the peer review process. Perhaps I just want to get a dig in at the editors that held up my paper when my attention should have been elsewhere. In the end I'm just trying to give an impression of some of the pain that a scientist goes through when trying to get their work published. It is not an easy process and, akin to writing or other artistic processes is rife with criticisms and other obstacles. It seems that the public's perception of science is that it has a clear answer, things are right or they are wrong. If this were the case then publishing scientific results would be a much easier process, a paper would be right or wrong and could be published or rejected. As it is, it often comes down to a friendly or hostile editor combined with a friendly or hostile referee (or a majority of such!) to decide whether or not you will be published and how quickly. When your career can depend on these decisions it might be just as well to let the criticisms go and, like any endeavour, have faith in your work. You will know the effort that you have poured into it and you will know what that is worth, even if others don't always recognise it.
No comments:
Post a Comment