30 September 2009

Has Steve McIntyre Found Something Really Important?

A commenter here asks me to discuss recent goings on over at Climate Audit, where Steve McIntyre thinks he is on to something rather important. I've followed Steve's work for years, and I think I have a pretty good sense of what he is up to and why it might matter for climate science and the nexus of science and political debate. And if you don't know what this is about, good luck catching up to speed (but if you want to try, there will be no better place than Bishop Hill's recounting). Such is the complexity of the issue and its history.

But let me say this: If Steve has indeed come across new information that forces a significant re-evaluation of a major branch of climate science, then there is no excuse for this not to appear in the peer-reviewed scientific literature. I think that Steve can easily separate out the quantitative implications of his work from the messy science-politics part. If Steve has discovered a smoking gun, then I'd expect Nature and Science to both be candidate publications. And I would really hope that some of those members of the relevant expert community who (I know) frequent his blog would join with him, perhaps as co-authors, to help bring the new analysis into the mainstream scientific discussion. That is how science moves forward.

Meantime, all we have is some interesting analyses and speculation on a well-read and thoughtful blog. I'm happy to wait and see what develops, and to let Steve get on with his work in progress.

24 comments:

  1. Given the editorial practices at both Science and Nature regarding Steve and his requests for data archiving, of which I'm sure you are aware. your confidence is remarkable.

    ReplyDelete
  2. I second what Whitey Bulger says. If either publication were to do so, it would mark a turning point in the debate.

    ReplyDelete
  3. -1, 2-

    This is all just speculation until a paper is written and submitted. For now at least the onus is on McIntyre to produce a paper.

    ReplyDelete
  4. Roger

    Didn't Nature refuse to publish McIntyre's original analysis of Mann et al ?

    http://news.bbc.co.uk/1/low/sci/tech/7092614.stm

    Nature's refusal to publish a re-analysis by Stephen McIntyre and Ross McKitrick of the famous (or infamous, depending on your point of view) "hockey stick" graph has been so well documented elsewhere, not least in hearings instigated by US congressmen, that there is really nothing new to say.

    ReplyDelete
  5. Times have changed, Nature and Science to refuse McIntyre's paper now would be very unwise and not very scientific.

    ReplyDelete
  6. "Nature's refusal to publish a re-analysis by Stephen McIntyre and Ross McKitrick of the famous (or infamous, depending on your point of view) "hockey stick" graph has been so well documented elsewhere, not least in hearings instigated by US congressmen, that there is really nothing new to say."

    Bishopp Hill has a nice article about this here: http://bishophill.squarespace.com/blog/2008/8/11/caspar-and-the-jesus-paper.html

    and about the latest developments: http://bishophill.squarespace.com/blog/2009/9/29/the-yamal-implosion.html

    ReplyDelete
  7. The integrity of the scientific process, and of Science and Nature in particular, is directly implicated by Steve's work on this and other data sets.

    There is nothing even remotely surprising about individuals researchers using dubious practices to exaggerate the importance of their results.

    What is shocking is the failure of these journals to use the tools at their disposal to discourage and otherwise expose such research.

    Amongst these tools are:

    1. Requiring that published papers be peer reviewed by respected academics with expertise in the disciplines used in the paper.

    We have repeatedly seen papers that are mostly (or entirely) statistical in nature be published without ever being reviewed by a statistician. The result has been a series of glaring errors, and a failure to reference (and draw upon) decades of work by statisticians.

    2. Enforcing written requirements that data be shared.

    Although Science and Nature (amongst others) have long had written policies requiring the disclosure of data used in published papers, they have failed to enforce these policies. Science in particular refused to require the disclosure of the data currently being analysed by Steve despite publishing a 2006 paper based on this data.

    There are over a dozen well documented cases of researchers being unable to obtain the raw data used in papers having a significant impact on climate policy. There are almost certainly many others. Journals such as Nature and Science could require disclosure of data and methods as a condition of publication. They do not.

    Hat tip to the Royal Society which DID enforce its procedures with respect to Phil Trans B, resulting in the disclosure that permitted Steve's current analysis.


    3. Facilitating the timely publication of effective comments.

    As documented in many places (http://www.climateaudit.org/?p=7134), it is extremely difficult to publish comments correcting errors (even obvious ones) in journals. When comments are published they often lack essential details due to space constraints, thus obfuscating the magnitude of the errors or impropriety.

    Steve has attempted (and sometimes succeeded) at publishing comments. Both his successes and difficulties seem to attest to this point.

    4. Deciding which papers to publish (and where to publish them) based on their actual scientific content.

    The recent Steig et al was a purely statistical analysis [using no new data], covering a topic which had already been the subject of many publications. It and used questionable statistical practices which had most likely never been reviewed by a statistician, and failed to reference a wealth of relevant (albeit old) statistical papers which might have shone light on their errors.

    None of this keet Nature from putting it on the cover.


    In all obvious respects, instead of using the tools at their disposal to encourage basic scientific honesty, integrity and openness, the major journals have instead become cheerleaders for a particular point of view. In the process they have permitted to be published (without comment) lapses in scientific integrity that would be positively scandalous in any other discipline.


    The importance of Steve's discoveries is NOT that he "forces a significant re-evaluation of a major branch of climate science".

    Steve's discoveries force a significant re-evaluation of the role that journals play in the scientific community and whether or not their recent conduct permits us to continue to ascribe great value to the fact that a particular article has been published in a peer reviewed publication.

    I'm not certain that publication in a major journal is the optimal method of curing this last defect.

    ReplyDelete
  8. Roger, sorry, but I disagree with your recommendation to Steve re: having a paper printed in one of the journals. His work has established the reduced value of doing so, especially in light of those journal failures to perform due diligence.

    The alternative media, his, yours, your father's, Watts, Marohasy's, Lucia's etc. blogs are making progress in real time reviews. Some are even peer reviews.

    This might also make the Publish or Perish meme less important.

    Dunno. Just saying!

    ReplyDelete
  9. Roger:
    My immediate response was to disagree with you - and I still do if you are saying that getting the paper accepted is some kind better test as to the robustness of his findings than what is currently happening on a wide range of blogs (and we both know that the guys at RC are working hard to check Steve's conclusions - but, if Steve and Ross and whoever can pull it together quickly, then I think it will test the political neutrality of Science and Nature and that is a prize worth the effort.

    ReplyDelete
  10. I'm not objecting to Steve publishing his work in a major journal. I suspect that there are few who would object to such a decision.

    Far more objectionable is Roger's apparent belief that Steve is obligated to publish his work in a major journal (the very entities responsible for the current mess).

    ReplyDelete
  11. Of course it would be nice if a paper was submitted, but maybe Steve knows too much about what to expect. Why should he volunteer for this:

    http://www.scribd.com/doc/18773744/How-to-Publish-a-Scientific-Comment-in-1-2-3-Easy-Steps

    ReplyDelete
  12. I second all of Jason S's contraindications and demurs.

    More improbably, why not just grant industrious - and "against all the odds" - Steve a MacArthur "genius" grant, and let him keep going...?

    ReplyDelete
  13. Nature and Science are potentially implicated in the most important ramifications of this new information; the complete failure and possibly duplicity and falsification of science.

    This needs to be thrashed out in public right now, with no more obsfucation from any of the relevant parties (including Nature and Science) to get to the bottom of what has been going on.

    I am astonished that this is being treated as just another technical issue. Yet the statistics is but a sideshow.

    ReplyDelete
  14. Just to add Roger. Both Nature and Science have been at the centre of difficulties in obtaiing full and complete data disclosure in the area of climate science.

    They are in the dock here.

    ReplyDelete
  15. Roger,
    Steve’s work has shown that the U.N. climate scientists are willing to sacrifice the credibility of scientists to achieve their policy goals. Now you expect that scientific journals will publish an article that exposes their lack of credibility?

    I don’t think so.

    Once money is spent you can always earn more. Once credibility is lost there are no words that will regain it.

    ReplyDelete
  16. When the small child said that the emperor had no clothes, it wasn't necessary for the statement to be sworned and notarized for the people to be able to recognize the truth.

    Policymakers around the globe are being hounded to make enormous changes in our way of life on the basis of "science" studies. Those demanding change have a burden of proof and the burden is very high (I'd recommend the same beyond-reasonable-doubt standard required to punish a criminal. After all, those who will be punished by the advocated change haven't even been charged with a crime.)

    From a policy standpoint, McIntyre has NO burden to publish something in order for his point to be considered valid. The alarmist scientists have relied on suspect data. Steve showed that the data comes from a ridiculously small sample and that it is contradicted by other data that should have been included in any responsible studies. These are facts (or assertions which are subject to easy checking). The contention that these facts have no validity until and unless gatekeeper journals choose to give them blessing is ridiculous. Especially since the gatekeepers have disqualifying conflicts of interest given their role in publication of the offending studies, their failure to require adherence to their own data rules, and their well-established record of stonewalling previous attempts to publish work which contradicts the AGW litany.

    This same point applies equally to Watts and the discovery that 90% of the US temperature stations fail minimal scientific standards. He has no burden to show what difference the bad data has made on temperature histories. The burden has to fall on those who have used the bad data. It is they who must produce new studies which adequately account for the bad data (if that is even possible). Otherwise, their work is garbage and should be discarded.

    And note, that the revelations by both Watts and McIntyre go to an even more basic point when it comes to the burden of proof which alarmists must satisfy in any sensible policymaking arena. The particular alarmist scientists involved here have been shown to be either incompetent or fraudulent in the design of their studies. No one should assume any longer that ANY of their work is quality science. Perhaps it is quality, but given their error-filled record, no one should operate under that assumption. ALL their work should be audited and replicated before it is relied upon by policymakers.

    ReplyDelete
  17. To those suggesting that McIntyre need not publish in the peer-reviewed literature ... while I certainly understand the frustration at seeing (some) scientists behaving badly (and trust me, I've seen more than you), the answer is not to do away with institutions of science, but to fix them. Steve's publishing is important not just for him and his ideas (which it is) but also for the integrity of scientific institutions that we will will continue to rely on.

    ReplyDelete
  18. Maurice (and others),

    I think it is absolutely plausible that a major journal will publish Steve's work (although Steve should be reluctant to do so since he still has access to very little of the relevant data).

    I am personally reluctant to give the journals such a chance at redemption. After such a publication we'll not doubt hear that the scientific process worked as it was supposed to.

    The process has NOT worked. Any future publication of Steve's work by Nature will NOT render what has gone before acceptable.

    This is a low point in the history of science. Major journals are complicit in the doctoring of results for the purpose of influencing policy.

    It would be a terrible mistake for us to simply "move on" as if nothing out of the ordinary had happened.

    ReplyDelete
  19. Roger, you said that "Steve's publishing is important not just for him and his ideas (which it is) but also for the integrity of scientific institutions that we will will continue to rely on."

    Which is more important to the integrity of science:

    A: Publishing Steve's work

    B: Reforming scientific institutions so that efforts like Steve's will be less necessary in the future

    ?

    ReplyDelete
  20. -19-Jason S

    I see A and B as tied together

    ReplyDelete
  21. I suggest Steve writes a very short "Comment on Trees Tell of Past Climates: But Are They Speaking Less Clearly Today?" to the Philosophical Transactions: Biological Sciences in which he demonstrates the lack of robustness in the Briffa hockeystick. In that way he also can make his version of the merged Yamal dataset official and quotable.

    After acceptance, Steve can submit comments on all other publications that did use Briffa's Yamal series to these specific journals, citing his then peer reviewed work in Phil. Trans..

    Also Steve will have a solid reference everytime when people do continue to use Briffa's Yamal series in future publications.

    Finally Phil. Trans. is most probably friendly inclined towards the subission of a comment by Steve, as the original paper appeared in Phil. Trans.

    ReplyDelete
  22. I agree that submitting a Comment to Phil Trans B is logical. Phil Trans B has demonstrated a greater to data availability than other journals. Based on past experience, my inclination will be to carry out the submission in full view, simultaneously placing the submission at arxiv and placing any reviews on record as well.

    ReplyDelete
  23. Jason S.

    I really appreciated your four tools. Two things 1) I think all journals should have comment posts for all their papers to allow people to ask questions and comment.
    2) Just so all you climate folks know: the behaviors you describe of Nature and Science are not unique to climate science; I have found those kinds of behaviors frequently in my own field, when politically hot topics arise.

    ReplyDelete
  24. Of course it should be the "Comment on Trends in recent temperature and radial tree growth spanning 2000 years across northwest Eurasia".

    "Trees Tell of Past Climates: But Are They Speaking Less Clearly Today?" is the Phil. Trans. B paper that Briffa co-authored with Schweingruber (!) in 1998...

    ReplyDelete