Peer Review

Anyone that has read Ben Goldacre’s Bad Science will know the importance of the peer review system to science.  Scientists will write papers on any discoveries and submit these papers to journals for publication.  This way their work can be scrutinised and critically assessed by other experts in the field.  These journals are rarely read by the general public though.  Publications such as New Scientist and Nature provide a link from scientists to the public, but much of the science news we digest is reported by the media without verification of facts and figures.  We wouldn’t expect important decisions to be based on hearsay?

In 1998 Syed Hasnain, an indian glaciologist, stated he thought all the glaciers in the eastern and central portions of the himalayas would disappear by 2035.  These claims were never repeated in a peer-reviewed journal and Syed now says his claims were speculative.  However these claims were reported as “very likely” in a 2007 report from the Intergovernmental Panel on Climate Change.  The IPCC have now apologised for the mistake, however the question of how and why the claim was included in the first place still remains.

At times the world of science can seem murky and underhand.  Occurrences like the Climategate scandal show that information isn’t always freely available to those that want to or need to see it.  The peer-review system can lead to some scientists being frozen out if they want to publish unpopular views.  A balance must be struck, we want reliable data that has been critical assessed by experts, but we want that information widely available too.  It is important though that any science news story is supplied with appropriate published sources of the original study, so look out for mentions of papers and where they are published, so you don’t get caught out like the IPCC.

Professor Simon


Tags: , , , , , ,

6 Responses to “Peer Review”

  1. Charlie Says:

    The problem is that more often than not peer review is used to freeze out views challenging orthodoxy than ensure some kind of academic scrutiny. The history of peer review extending into the present has been extremely wound up in protecting positions of authority and monetary gain (especially in regards to medicine). This is even more so now when a ‘fraud’ coming to light cost people jobs, grants, prestige, essentially their entire life (cos lets not pretend academics in any field have any life outside of academia). When submitting work to journals, even though the reviewer is anonymous to the submitter and vice versa you are not anonymous to the editor of the journal (who makes the final decision), and those editors normally have a lot of interests of their own that would suffer from critical material being published. I think economics is worst for this but it is also prominent in science such as the effective ban on ‘anti’ string theory research (not that I claim to know anything about string theory.

    Also for all its bravado peer review has often failed in preventing fraudulent work getting through, eg. (once again not that I claim to know anything about the subject matter). Probably loads more here;

    Richard Horton; ‘The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.’

    It is also of course noting there are alternatives to peer review, such as based along lines of

    As a last point though the fact that I got all my sources here from wikipedia could probably open a whole new debate about different types of peer review and their validity…

    • simplesciman Says:

      Currently scientific information usually disseminates through either a system devised to check and validate information or spuriously through one off comments to the media/governments. Though the first way isn’t perfect it’s infinitely better than the second. The problems with the peer review system are there, and with the internet now pretty much widely available in the western world maybe there should be a system there should be some way of papers being published on an online database for anyone to scrutinise. I would welcome that, it would probably make it easier for me to get information/sources too. What is important is that any claim is associated with a methodology, set of data points and explanation of the analysis that can used to assess the quality of the work.

  2. Charlie Says:

    The problem with government data is not just that it comes from ‘one off comments’, the government has been pretty good at having scientific bodies to look at data… only thing is they tell them what to do then ignore them when they say something different (look at drugs for example).

    I get what you are saying about using the internet for a database and that does make more sense, problem is though that not everyone is starting from an equal base in checking/validating information in that 100 ‘database guys’ could check something and slate it for inaccuracies but one professor with an interest in seeing it published could say the opposite and bam its in a journal. I also read about a procedure where you put a small but deliberate mistake in a submission to a journal so loads of people will reply correcting it, so then if anyone looks there will be loads of articles citing your one making it look like your one is really important (but this is more of a side point).

    As is always obvious I know jack shit about science but your last sentence seems to make sense, and if news sources/the government always had to check for these things it would be good, however as it stands they don’t (and won’t ever have to), for example when you see something in the paper they never mention using ‘x method of y data’ and people don’t want to see that either, all you need is ‘professor abc notes’ or even just ‘an expert says’, but these experts are never neutral. Someone peer reviewing an article doesn’t necessarily have to check for these things either if they can get around it but like the way the article is headed and perhaps most importantly even if all those things are there but they don’t like what the article is actually saying they are perfectly free to ignore it.

    Some things are better than others but as long as peer review is tied up in webs of academic and political authority and monetary gain I can’t see a way in which it won’t be heavily skewed.

  3. freddieMaize Says:

    May be they wanna prove that you never landed on the moon. 😉

    Jokes apart –

    By your post, do you mean America has found everything about space science and there is nothing left for the rest of other nations to find/gain anything?

    This is science. Please don’t give room for Politics.

    I’m here for a healthy conversation (if you like)

    Please don’t get offended. I’m trying to be practical.


    • simplesciman Says:

      Not at all, the universe is, well bloody massive and there is so little we know about it. The scientific community definitely what to keep exploring, just with probes, satellites and robots. I commend any enough to find out more about the universe beyond earth, though there is very little advantage to sending men into space. For the money China may spend sending a man to the moon they could send tens of robots to Io, robots that could make amazing discoveries such as traces of life.

      It is unfortunate when science does get entangled in the web of politics, as then it is so often it is abused.

      Thanks for commenting.

  4. freddieMaize Says:

    OOPS. Sry, i gave comment in the wrong place. This is for the “Why Would You Go Into Space?”. Sry again.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: