Publishing in Science and Peer Review

As I stated in my last post, I wanted to write something about how my last astronomical paper went through the publication process. As promised, here is the second part of that post.

Part II

A question often asked by a scientist's friends and family is 'what do you actually do?' A pretty fair question really, especially as many of us only get to be career scientists through the support of the government and thus the tax payer. The truth is that, most often, scientists are writing papers. A (largely) complete description of an experiment, the results and some analysis/summary based on those results. Scientists are always writing at least one paper because this is what we are really judged on, really, to a degree that would probably astonish you.

Getting these papers published means submitting them to peer review, something pretty much any scientist can tell you is a fairly painful process. We have all at some time created a work of art, a piece of writing, a meal or anything at all that is then subjected to some level of criticism. For some people this is a more personal and profound experience than for others. It is easy to imagine that a painter or playwright may feel quite affected by a series of poor reviews of their work. It may be somewhat harder to imagine that scientists often go through a similar experience.

The process of peer review has grown from the earliest ideas that any judgement may be best done by a collection of your peers. In the 18th century this idea began to be applied to the publication of scientific results. As early as 1731 scientific results were being distributed amongst individuals deemed by the editor to be worthy to evaluate them. The idea being not so much that peer review could police fraudulent results or the like but simply that a filter was in place to help guide the editors towards relevant and interesting results. It is almost certainly a coincidence but only one year earlier a tightening of control over judicial peer review was also put into place.

The utopian model of peer review is one in which the journal you want your paper published in is run by a 'benevolent dictator' of an editor. This editor would be all-knowing, all-wise, fair, ethical and, above all has the time and energy to read and evaluate my paper along with the thousands of others submitted every year. This actually wasn't far from the case when Albert Einstein himself published his 'Annus Mirablis' papers in 1905. Each of Einstein's 'Miracle Year' papers was read and reviewed by Max Planck himself, the associate editor of Annalen der Physik at the time. Max Planck was the genius physicist on whose work Einstein had based his first paper and who would later win the Nobel Prize himself.

These days papers are doled out by the editors (or sub-editors) to referees whom they deem to be experts in the field. I have received papers myself to review, although that is another story for another time, and I don't intend to criticise this process. After all, Churchill's quote on democracy is particularly apt here in that peer review may arguably be a horrible process but it does have the advantage of being better than any others that have been tried.

Once an article has been placed with a referee, that referee evaluates it and submits a report, along with one of three recommendations.

1. Firstly, that the journal should accept the article 'as is'; this does actually happen, although extremely rarely.
2. Secondly, that the article should be accepted, though some revisions should be made first. This is the most common result in my experience, particularly when this recommendation is subdivided further into decisions ranging from 'major revisions with subsequent review' to 'minor revisions with no subsequent review'.
3. Finally, an article may be outright rejected if it is considered to contribute nothing of any publishable value and/or be so poorly written that a complete rewrite is necessary.

I should point out that this entire process is anonymous, at least from the side of the author. The person reviewing my work knows who I am (I flatter myself, I mean that they know my name at least) while they remain anonymous to me.

As I alluded to in my last post, my most recent paper took a great deal of time to publish. The reasons for that are many, some of which are my own fault, some are just happenstance and some are the fault of the journal (in my opinion). I submitted the paper just before Christmas 2012 with high hopes that this paper would draw a little attention, enough perhaps that I might be able to score a job interview or two off the back of it. I had certainly invested enough time and effort into it and, at over 40 pages at the time it was hefty enough to be recognised as a solid body of work.

The paper came back with a substantial referee report. We took this report seriously and worked through the suggested revisions, resubmitting the paper in early April. At this point I was still hopeful that the paper would get out quickly and be useful in my ongoing job-hunting. A further referee's report came back, again fairly substantial, although this time the report asked only for some moderate revisions. The editor told us at this point that the referee was happy with the paper assuming that we took care of the second set of revisions and that they didn't want to review it a third time. We received assurance from the editor that, so long as we were conscientious about our revisions, the paper would be accepted and published with our next submission.

So, thinking we were on the home straight, I took care of the changes and resubmitted the paper. At this point the editor, without consulting us, forwarded the manuscript to another referee altogether. This was unexpected, especially considering that the editor had specifically told us this wouldn't happen. We were a bit taken aback by this turn of events but not particularly worried, given that we had already been through two revisions of the paper.

This is where the random and capricious nature of the peer review process struck. The second (anonymous) referee rejected our paper outright with a fairly cursory explanation as to why. Given that the paper now represented over two years of work on my part I was understandably upset. Challenging this resulted in a rather hostile statement that the paper did not present results worthy of publication. This seemed a rather odd thing to say after the paper had already been under review for eight months and had been essentially accepted, even praised by the first referee. We were able to win an agreement to send the paper to a third, deciding referee. This referee was sympathetic to both previous reports and suggested re-writes of their own to make the paper somewhat more concise. At this point the paper had been 'in the works' for nearly eight months and I was in the middle of a move (both in career and location) and my wife's pregnancy. All of the impetus of the paper had been taken away and so it took a good deal of time to make yet another set of revisions and resubmit the paper. This was done 13 months after its first submission, when it was finally accepted.

I hope that this kind of experience is rare but I do believe that the journals and editors need to look carefully at their processes. In a field in which publication is so vital to the careers of the submitting authors it seems irresponsible to take less than the utmost care and diligence in publishing their works. In the scientific realm scientists are supposed to be above reproach when presenting their results. Falsifying data is one of the most heinous academic crimes possible and, along with plagarism, about the only things one can do to have their PhD revoked. I think that editors should be held to similar standards and that misleading authors and/or delaying the publication of results (intentionally or not) are arguably just as serious.

I'm not sure of the point of blogging this story, maybe it's just to complain about the peer review process. Perhaps I just want to get a dig in at the editors that held up my paper when my attention should have been elsewhere. In the end I'm just trying to give an impression of some of the pain that a scientist goes through when trying to get their work published. It is not an easy process and, akin to writing or other artistic processes is rife with criticisms and other obstacles. It seems that the public's perception of science is that it has a clear answer, things are right or they are wrong. If this were the case then publishing scientific results would be a much easier process, a paper would be right or wrong and could be published or rejected. As it is, it often comes down to a friendly or hostile editor combined with a friendly or hostile referee (or a majority of such!) to decide whether or not you will be published and how quickly. When your career can depend on these decisions it might be just as well to let the criticisms go and, like any endeavour, have faith in your work. You will know the effort that you have poured into it and you will know what that is worth, even if others don't always recognise it.

Big Stars and Little Stars

After an amount of time that I'd really rather not think about, my most recent astronomical paper has been published. This paper is particularly special because it is likely to be my last astronomical paper (at least as first author). I thought that this paper would be good to blog about not only because the subject matter is interesting in itself but because I thought that the story of this paper's journey to publication might make an interesting one. In an effort to keep my posts at a readable length as well as milking a subject for all it's worth, I'm going to split the post into two, the first dealing with the subject of the paper, the second with the story of how the paper was published.

Part I
As this is a blog and intended to be at least partly accessible to the average member of the public, I am only going to write about the subject of my paper in fairly broad terms. If you're interested in the details of the experiments, analysis and so on, please go and read the paper for all the gritty details. It is hosted on the Monthly Notices of the Royal Astronomical Society site here.

A relatively succinct description of the paper might be that we were interested in the potential differences between the Perseus molecular cloud, seen below and the W3 molecular cloud seen, um... belower.

Credit to APoD - http://apod.nasa.gov/apod/ap111021.html

Credit to the Herschel Observatory - http://herschel.cf.ac.uk/results/w3-star-forming-region

The reason we're interested in these differences is that both clouds are host to a lot of young new stars still in the process of forming. However, in Perseus these 'protostars' are generally quite small (around the size of our Sun or smaller) while in W3 they are generally considerably bigger (up to tens of times the size of our sun). Figuring out why one region make the big stars that can disrupt the Galaxy for light years around, while another is happy just churning out smaller, longer-lived stars, can give us insight into the way in which our own solar system formed as well as where we might expect to see future star systems possibly hosting Earth-like planets.

Do big stars form in the same way (more or less) that small stars do? Is it just coincidence and/or chance that big stars form where they do or is there some underlying difference about say, the Perseus cloud versus the W3 cloud that leads to them forming stars with different properties? This question is one that has been leading a large part of star formation research over recent years.

By investigating this question we found that the 'dusty clumps' that are the core or nugget of star formation in these clouds do have different appearances, which we suggest implies that they have come from different underlying structures. Our images of sites of star formation, like this -

are like looking at an island in a sea. Picture the island sticking up out of the sea in a way that the height of the island corresponds to the brightness of our star-forming region which, when you're looking at them in the way that we are, implies the density of the protostar.

You can see here that the surface of the island might continue below the sea so, let's say the sea level rises to the height shown by the dashed line in my cartoon. All of a sudden we can't see as much of the island, though the island could be said to be just as big as it was before. Star formation is much like this, the protostar is a big clump of dust and gas which sits in a large 'sea' of dust and gas. The larger 'sea' makes it hard for us to determine exactly how big our island is and what it looks like. In order to determine anything from our observations we have to go off what small part of the island we can see, as well as the properties of the sea which we might be able to determine from other observations.

Well, to torture this metaphor a little further, when we look at the stars that are forming in Perseus, we see small islands in large seas. When we look at W3, we see that the islands are quite a bit bigger, even though the seas are around the same level. This implies that the two regions may, in fact be forming stars in quite different ways. A model which might fit our observations has been suggested recently by Phil Myers, who suggests that smaller stars form by drawing in local material, while larger stars form by pulling in more material from the greater store available in the larger cloud.

While our study is quite speculative, it does tell us something important, which is that when we compare different types of observation, we have to be very careful in our assumptions about what we are looking at. The 'island' which we are looking at can often be assumed to bear some relation to the 'sea' in which it is sat. However, the relationship which exists between one set of islands in one sea cannot always be assumed to exist when looking elsewhere. Our paper implies is that there is a distinct difference between the underlying structures surrounding the protostars in each region. If this is the case then the ways in which big stars and little stars form is actually quite different. Maybe this is due to different conditions at the outset, or perhaps other factors come into play which affect the ways that stars form and these factors vary from place to place within our Galaxy. What is clear is that we must be careful when we perform our large scale surveys of the Galaxy, such as the Herschel Space Observatory's Galactic Plane Survey HiGAL. If we are not careful to take account of the underlying differences between star-forming regions in our Galaxy, then we risk misinterpreting results and potentially drawing false conclusions.

I think it's best to leave my convoluted island metaphor there for now. A bit more on the actual observations and physics that we used to make our observations is below, please feel free to read it if you have made it this far! Also, I'd be more than happy to answer any questions you might have in comments.

To investigate these two clouds we looked at them both in two quite different ways, both invisible to the human eye. Firstly we looked at the big clumps of 'dust' that host the protostars in the clouds using the SCUBA instrument on the James Clerk Maxwell Telescope in Hawaii, due to be decommissioned towards the end of this year. This dust is warm by comparison to most of interstellar space at about - 250 °C and at that temperature shows up well at submillimetre wavelengths, well outside of the range of human vision. Looking at this dust tells us a few things about the protostar (usually multiple ones) forming in the molecular clouds, typically we can determine the approximate temperature and mass of them, although some careful analysis and modelling can sometimes tell us more.

In addition to these dusty temperature maps that we made of the young stars in Perseus and W3, we looked at them using the Green Bank Telescope, another (ENORMOUS!) great telescope in danger of being shut down (visit http://www.savethegbt.org to learn more and help save it!). The GBT has a great array detector which is able to map out these young stars using molecular spectroscopy. In this case, the molecule in question is ammonia, found in abundance in many protostars. The places where stars form in our Galaxy may be hot and dense compared to what we call 'empty space' but they are still astonishingly cold compared to Earth and are at a density equivalent to the best ever vacuum achieved by mankind on Earth! These extraordinary conditions mean that ammonia (amongst other molecules) can exist in states that would never be found in normal human experience. This turns out to be very lucky for astronomers as careful analysis of the radiation emitted by ammonia in these states can tell us not only what the temperature of the forming stars is but also what the density is as well as how the gas is moving around within the clouds.

Not only are these two ways of examining protostars very informative on their own but by comparing them we can learn even more. Crudely speaking, by comparing the temperature we find using one method with the temperature we find from the other, we can infer how close a protostar is to becoming a fully-fledged star and what kind of size that star might be. As you can imagine this is a powerful technique that has been used in astronomy for some time in various guises (usually referred to as the virial mass or virial ratio).

Resistance is Futile

I believe that I've been threatening to start writing a blog for around half a decade now. Given my recent career change I thought that this would be as good a time as any, communicating the work being done at the Met Office and within my previous career in astronomy. After all, as the joke among other 'extronomers' at the Met Office goes, British astronomers all end up studying the weather because they spend most of their time looking at clouds anyway...

Since starting my new job at the Met Office in November I've been keeping my eye out for reasonably interesting topics for posts, fortunately I didn't have to wait very long. Over coffee with some of my new colleagues I got invited to a seminar on assimilating seals into the Met Office oceanography models. At this point comes one of my first realisations of the differences between Met Office scientists and astronomers. Out of the astronomers I've known, few could have resisted the opportunity here to make a 'Borg' joke and fewer still wouldn't have gotten the joke (mostly based around the key word 'assimilating' but the Borg connection is almost unavoidable once you see the picture above). Around the coffee table with meteorologists of various stripes, I was met with blank stares following my (ok, not very funny) attempt. I guess Star Trek-awareness within the scientific community is particularly high amongst astronomers, who'd a thunk it?

After the somewhat mandatory back and forth (to be repeated by everyone I told about the talk) along the lines of 'seals? You mean like sea lions? - yeah', 'trained ones? - no', 'wild ones - yeah', etc. I decided that this seminar was something I wanted to see. I wasn't disappointed as, of all the talks/lectures/seminars I've attended over the years this was definitely among the best in terms of cool, interesting science and of generally fresh thinking. The starting premise is that temperature and salinity profiles of the ocean are an important thing for oceanographers, climatologists and the like to know in order to feed and confirm models of ocean currents and the climate generally. This is important for all sorts of reasons, such as how ocean circulation warms or cools parts of the world (like the UK for example) or determining the level of melt-ice in polar regions, important for climate-change and impact studies.

This information has historically been hard to collect and generally has come in the form of 'opportunistic' measurements, such as bathytherm(ograph)s which are dropped from commercial ships or research vessels. There have been large strides in this area in recent years with the Argo network of autonomous floats that spend their lives at sea, surfacing periodically to report their collected data via satellite. While the Argo floats have substantially increased the amount of data available on the temperature and salinity profiles of the upper 2000m or so of global oceans, there are still many locations where these data are hard or impossible to collect. For example, areas which have large amounts of sea ice are particularly difficult to analyse in an automated way. Particularly in recent years, data on the state and variation of sea ice and the associated ocean water are vital for climatalogical studies. Enter the use of seals - opening up a new measurement system which, although is pretty limited geographically, yields information that would be uncollectable in any other way. While hydrographic profiles from seals aren't going to be available where seals aren't, the places where seals are tend to be inaccessible and so other types of data which might be used instead are pretty much non-existent.

What I found particularly cool was that the data coming from the seals isn't necessarily even intended to be used in meteorological studies. For example, the Sea Mammal Research Unit at the University of St. Andrews started collecting these data in order to study the foraging ecology of elephant seals. This allows them to study the seals behaviour in relation to the ocean conditions as well as differences in population trends. This paper might be a starting point if you wanted to find out more about this connection. Finally, the use of seals in this way might be objectionable to some people. The seals are tranquilised before the data loggers/transmitters are attached and I'm told that the glue that is used is non-toxic to them and dissolves after six months to a year. It's true that shooting a dart at a seal and gluing a circuit board to their head can't really be described as 'zero impact' but, on a scale from 'none' to 'Loreal' I think that the level of harm to the seals is minimal. The fact that the collected data aid in the study of the seals behaviour and their protection further supports the use of this method.