25 November 2009

Redefining Peer Review

In 2005 Gavin Schmidt and Michael Mann, of Real Climate and CRU email fame, carefully explained that the process of peer review is a messy, incremental way to advance knowledge in fits and starts:
The current thinking of scientists on climate change is based on thousands of studies (Google Scholar gives 19,000 scientific articles for the full search phrase “global climate change”). Any new study will be one small grain of evidence that adds to this big pile, and it will shift the thinking of scientists slightly. Science proceeds like this in a slow, incremental way. It is extremely unlikely that any new study will immediately overthrow all the past knowledge.
They explained that even when results are published that do not stand the test of time, the process of peer review can successfully winnow out those arguments with the greatest merit:
. . . even when it initially breaks down, the process of peer-review does usually work in the end. But sometimes it can take a while.
With this perspective as background, one of the most damning aspects of the CRU emails was the behind-the-scenes efforts of the activist scientists to -- in their own words -- "redefine what the peer reviewed literature is."

Peer review as related to scientific publishing is a process in which experts are asked to judge the appropriateness of a paper for publication in a scientific journal. It is often cursory and focused on the merits of an argument, rather than a detailed replication or decomposition of the data or methods. Peer review does not mean that a result is right or will stand the test of time, but that it has met some minimal standards of acceptability for publication. The scientific community is replete with vignettes about papers that were rejected for publication in one venue only to be published elsewhere and which later turned out to be seminal. Similarly, every so often even Science and Nature find themselves in trouble with a paper that is badly wrong or even fraudulent. But despite these shortcomings in the process, peer review is widely viewed much as Winston Churchill viewed democracy: the worst possible system except for all the others.

Peer review works because over the long-term good ideas win out, and this process happens organically and through a decentralized process. Peer review takes place through many independent journals, with editing and reviewing conducted by many independent scholars from a diversity of disciplinary and experiential backgrounds, and with their own idiosyncratic biases and views. No one group or perspective owns the peer review process, and that diversity is part of its core strength. Truth -- meaning a convergence to agreement on scientific questions -- thus is a product of the peer review process over time. Of course the path to truth can be convoluted and indirect. For instance, it used to be true that there were 9 planets in our solar system. Now that is less true.

Some issues relevant to decisions are characterized by uncertainties and contested certainties making the distribution of scientific views not readily apparent simply by looking at the sprawling literature. In such situations a formal assessment can provide a useful perspective on the degree of consensus or disagreement among relevant experts on various claims. Such assessments are nothing more than a snapshot in time, as science is continuously evolving. When done well, an assessment will reflect the full range of views held by relevant experts, including minority views (see PDF), as well as the connections of scientific understandings to alternative possible courses of action.

Now back to the CRU emails. The emails show a consistent pattern of behavior among the activist scientists to redefine peer review in accordance with their own views of climate science. In doing so, they sought to turn the entire notion of peer review on its head.

The emails show a group of scientists frustrated with the peer review process, seeking to change how it is practiced. How so? The emails indicate concerted efforts to reshape the peer review process by managing and coordinating reviews of individual papers, by putting pressure on journal editors and editorial boards, by seeking to stack editorial boards with like-minded colleagues, by arranging boycotts of journals and other actions involving highly questionable ethics. But we might wonder why these scientists would take such steps to change peer review if, as Schmidt and Mann explained at Real Climate -- "peer review usually does work in the end." Why depart from a process that works? The answer is obvious: the short-term politics of climate change.

The activist scientists decided that the peer review process would work better in service of their political agenda if it used "truth" to determine whose views would be allowed to be published in the literature and reflected in assessments. In this case "truth" simply means the views deemed acceptable among the activist scientists and their close clique of colleagues. In an interview with NPR Real Climate's Gavin Schmidt defended this very backwards view of peer review:

Journals are supposed to be impartial filters that let good ideas rise to the top and bad ideas sink to the bottom. But the stolen emails show that a group of scientists has decided that's not working well enough. So they have resorted to strong tactics — including possible boycotts — to keep any paper they think is dubious from reaching the pages of a journal.

"In any other field (a bad paper) would just be ignored," says Gavin Schmidt at NASA's Goddard Institute for Space Studies in New York. "The problem is in the climate field has become extremely politicized, and every time some nonsense paper gets into a proper journal, it gets blown out of all proportion."

Most of the papers Schmidt and his colleagues object to challenge the mainstream view of climate science. Schmidt says they may be wrong or even deceptive, but they are still picked up by politicians, pundits and businesses who are skeptical of climate change.

So Schmidt suggests that in order to short circuit the ability of their political opponents to cherry pick and blow out of proportion studies that the activists scientists did not agree with, they saw a convenient short cut: Simply reshape the peer review system such that those papers don't ever appear or go unmentioned in scientific assessments.

The problem with this strategy, of course, is that many climate scientists (and presumably others inside and outside of the scientific establishment) are unwilling to cede ownership of the "truth" to a small clique of scientists. In fact, peer review exists in the first place because there are no short cuts to the truth, and any such short cut will inevitably fail. Consider that the efforts revealed in the CRU emails to manage the peer reviewed literature went well beyond efforts to prevent so-called "skeptical" papers from being published, but included a focus on papers that fully accepted a human influence on climate, but which offered views that differed in some degree (e.g., here) from those preferred by the activist scientists. The emails reveal activist scientists busy extolling the virtues of peer review to journalists and the public, while at the same time they were busy behind the scenes working to corrupt the peer review process in a way that favored their views on the science and politics of climate change. Here we have a case study in the politicization of climate science by climate scientists.

The clique of activist scientists sees absolutely nothing wrong in what they are doing -- they are after all justifying their actions in terms of "truth" in support of the greater good. And the issue is made even more complex because those who share the political agenda of the activist scientists are ready to join their peer review coup whereas those opposed to that political agenda are happy to try to exploit for political gain the scientists' ethical lapses and failure to appreciate their role in politicizing climate science. So much of the discussion gets wrapped up in these distractions, rather than the issue of the integrity of climate science.

The sustainability of climate science depends upon our ability to distinguish the health of the scientific enterprise from the politics of climate change. The need to respond to climate change (which I support) does not justify sacrificing standards of scientific integrity for political ends. In fact, as the events of the past week show, when standards of scientific integrity are compromised, the political consequences can be double edged.


  1. thank you for this great post

    However let me suggest that yoour definition of truth is slightly misleading
    "Truth -- meaning a convergence to agreement on scientific questions"

    I mean if something is true it might create convergence (think about he truth of newtonian physics ) but convergence is not a proof nor a criteria of truth.

    As we all know from Karl Popper: Truth exist but you are never sure you have it. In fact the only thing you can only be sure of, is that something is NOT true.

    This is very important as it shows that people "justifying their actions in terms of "truth" in support of the greater good" are just religious people unable to understand that the greatest value of science is to invite refutation by others.

    in this context where climate science seems so confused with climate politics, I think it is important to be very precise about the epistemological foundation and meaning of scientific concept like truth.

    The danger of providing a "consensual" defintion of truth is best illustrated by IPCC repeated attempt to argue for the "truth" of its position because there is a "widespread consensus among the majority of scientist".

  2. Politics is such a relentlessly corrosive force. Aided by greed, corruption and fear I wonder if any "truth" can stand against it. Truth becomes a relative situation, an a priori interpretation, with a distinct goal. The political truth is not dependent upon or revealed by any concerted rational, empirical analysis of data, but one that is created for a purpose. It is entrenched with credentials, fortified through peer review and defended by charismatic opportunists. That is until they are caught with their hands in the cookie jar. I do believe this will end poorly for all parties, including those who are trying to edge the climate science process forward through ethical/professional/moral efforts.

  3. Do you really think that peer review is the best system, except for all the others? From the perspective of outsiders who want some way to establish scientific "authority", and from the perspective of universities who want "objective" standards to base their personnel decisions on, I'll grant you that it seems indispensable. But, from the perspective of the actual process of science-- exchanging ideas and results with others working on the same problems-- the peer review process seems much more like a relic of an age when it was expensive to put things into print and get them distributed. Now, we've got the internet. Distribution and archival is cheap, but it seems like most fields outside of math, physics, and computer science have been extremely slow to take advantage of it and are even downright hostile to it (and even in CS, we've been woefully slow). Managing reputation takes a bit more work, but it can certainly be done.

    The thought experiment I like to run is to imagine we'd had the internet for the last 400 years. Would we really have decided that expensive, year-long publication cycles was the way to go?

  4. Phil Jones (29/1/2009) “With free wifi in my room, I’ve just seen that M+M have submitted a paper to IJC on your H2 statistic – using more years, up to 2007. They have also found your PCMDI data – laughing at the directory name – FOIA?….Anyway you’ll likely get this for review, or poor Francis will. Best if both Francis and Myles did this. If I get an email from Glenn I’ll suggest this.”

    Ben Santer replied: “It would be great if Francis and Myles got McIntyre’s paper for review.”

    Keith Briffa (22/9/1999) “I know there is pressure to present a nice tidy story as regards ‘apparent unprecedented warming in a thousand years or more in the proxy data’ but in reality the situation is not quite so simple. We don’t have a lot of proxies that come right up to date and those that do (at least a significant number of tree proxies) some unexpected changes in response that do not match the recent warming. I do not think it wise that this issue be ignored in the chapter. For the record, I do believe that the proxy data do show unusually warm conditions in recent decades. I am not sure that this unusual warming is so clear in the summer responsive data. I believe that the recent warmth was probably matched about 1000 years ago. I do not believe that global mean annual temperatures have simply cooled progressively over thousands of years as Mike appears to and I contend that that there is strong evidence for major changes in climate over the Holocene (not Milankovich cycles) that require explanation and that could represent part of the current or future background variability of our climate.”

    Giorgio Filippo (University of Trieste) (11/9/2000) “Essentially, I feel that at this point there are very little rules and almost anything goes. I think this will set a dangerous precedent which might mine the IPCC credibility, and I am a bit uncomfortable that now nearly everybody seems to think that it is just ok to do this.”

    Phil Jones (8/7/2004): “The other paper by MM is just garbage – as you knew. De Freitas again. Pielke is also losing all credibility as well by replying to the mad Finn as well – frequently as I see it. I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow – even if we have to redefine what the peer-review literature is!”

    Keith Briffa (29/4/2007): “I tried hard to balance the needs of the science and the IPCC, which were not always the same.”

  5. Mr. Pielke:

    Excellent assessment!

    The Freedom to express alternate scientific findings, to have the alternate findings given a fair review, is surely the correct path to Scientific Discovery. The Freedom to discover data of others and to test their data/findings is the transparency of Science. When Political Agenda through a Gatekeeper mentality stifles Freedom, then the path to Scientific Discovery becomes blocked.

    Maybe taking the entire proposition of Climate Change and looking at the current process from back to front is educational. That is, Cap and Trade is the end game. Cap and Trade is Political-Economy. The Political-Economy of Cap and Trade has an Agenda. The Agenda is Political and the Agenda wants to accomplish Economic Outcomes. The Agenda then needs Science that supports the Agenda. Only the Science that supports the Agenda is held out to be true. In order to block alternate scientific findings, Freedom to Express needs negated. Negating Freedom is done through manipulating review criteria and invoking the method of Gatekeeper of Information.

    What about this Economic Proposition: what is the simplest and cheapest solution to a problem? Apparently “Agenda” is causing Freedom to be negated and leading to Gate Keeping of information. Removing “Agenda” then allows Scientific Information to flow freely leading to true Scientific Discovery.

  6. Roger, I'm sensing a change in your attitude here. Leg one, the emails, of this 3 legged stool of Climategate was bad enough. I believe, however, that the other two legs, the S/W codes used to process the data, and the data (processed versus the raw versions) will be even worse.

    The bottom line here? Climategate will show that the temps have been manipulated to be higher than they actually were and were increasing at a rate higher than normal, AND, that the message from them resulted in predictions of exaggerated impacts (alarmist.)

    It will take a long time to reverse the damage from these exaggerations, but we should see a slow acceptance that climate change is happening within the range of natural variability.

  7. Roger what exactly do you mean by the 'sustainability' of climate science?

  8. These guys are what you might call "ethically challenged" . . . very, very ethically challenged

    The corruption of the PRP is a very important component to this ClimateGate story as "peer review" was used as the biggest cudgel in backing up theri claims of quality assurance in their work.

    The code reviews continue and are very revealing. Code comments such as:

    "From the file pl_decline.pro: check what the code is doing! It's reducing the temperatures in the 1930s, and introducing a parabolic trend into the data to make the temperatures in the 1990s look more dramatic."

    will be hard to explain, harder than the use of the word "trick"

  9. -4-Chris

    "Do you really think that peer review is the best system, except for all the others?"

    Yes. Which is why it needs to be taken care of.


    "However let me suggest that yoour definition of truth is slightly misleading"

    No not, misleading, but it does reveal me as a pragmatist;-)

    -7-Marlowe Johnson

    "Roger what exactly do you mean by the 'sustainability' of climate science?"

    I mean the ability to maintain trust and credibility from those outside climate science. That trust was deeply damaged this past week among many (e.g., see Monbiot).

  10. Gavin Schmidt responds at my colleague Ben Hale's blog, calling this post "pathetic" and "juvenile" but doesn't explain how so (funny that):


    I respond to him over there. If Gavin wants to respond here, to set the record straight, he is welcome to do so.

  11. Roger,

    Thanks for clarifying. Wrt this particular episode I tend to agree with the mohehill side and find Spencer Weart's take particularly appropriate (i.e. people's attitudes about climate change are pretty well entrenched one way or the other).

    If there is a silver lining to found in any of this I'd suggest that it's that the AGU may be forced to reconsider it's stance on open peer review ala EGU as Curry and Annan have suggested...

    On a related note, would you charaterize your own efforts here and on Prometheus as having a positive effect on the sustainibility of climate science? If so, why? I ask because I suspect that a number of your detractors likely feel differently than you do...


  12. -10-

    "Gavin Schmidt responds at my colleague Ben Hale's blog, . . . "

    By this very action, that makes the characterizations a self-reference.

  13. -12-Marlowe Johnson

    "would you charaterize your own efforts here and on Prometheus as having a positive effect on the sustainibility of climate science?"

    Well, since the very early days at Prometheus that has been one of my goals. Some of the partisans in the climate debate dislike the spotlight, and take exception. (No one likes being called unethical, and I fully understand this. But at the same time, if the shoe fits ...)

    One way to see how my work is interpreted and judged outside the hyper-politicized climate debate is to take a look at the many reviews of my book The Honest Broker, which was focused on the sustainability of science more generally:


    I am happy to have my work judged over the long term and am not so concerned about inflammatory comments by a very few angry, activist scientists. They too will be judged over the long term.

  14. Tangental to this is the post on your father's blog, including email correspondence he had during the drafting of the CCSP Assessment of Surface Temperature Trends. It would definitaley appear in that case to have been peer-filter.

    Most astonishing was the revelation that the entire chapter - which was supposed to be the work of a team of authors - ended up being the product of a single, junior, team member over 3 hours.

    More revalatory was that the email in which he introduced the proposed chapter included the following two statements in a single paragraph:

    "I have the most to lose through Chapter 6 in its current form in terms of future research career" Statement of motive.

    followed by:

    "Therefore I took the liberty of spending 3 hours this morning developing an alternative" Statement of action

  15. Peer review does have a fair number of problems. One is that for less highly visible fields, the quality of the review may be questionable as it is just one more thing to do for researchers that are very busy just trying to stay afloat applying for grants, teaching, serving on committees, taking care of their families, volunteering in their community, working out, etc. Generally, if you think some activity is important in our society, people get paid to do it. What does it say that people aren't paid to do reviews? Just sayin'.

    There is a body of literature concerning peer review as practiced- I remember some gender studies about 20-30 years ago; the take home lesson is that humans are inevitable human, and science is a society of humans.

    In my view, peer review is probably "good enough" for topics that are not politically important; but politically charged topics need more than peer review, and more than "consensus groups" whose findings are also a function of small p politics, and human group dynamics.

    Perhaps, as Chris says, it is time to design something more robust for these top scientific issues in terms of policy ramifications.

    Finally, I would hope that we would choose honesty, even at the expense of the "sustainability" of a given discipline. You can always work in another discipline, but you only have one soul. In my opinion.

  16. What is striking in much of the discussion of the CRU emails is how frequently the worst motives are attributed to the skeptics in the emails and to the "Team" in the discussion of the emails. It is as if both sides are locked in a deadly embrace of mutually desired destruction.

    Clearly to be an effective process, the peer review process needs to be carefully nurtured and protected, if it is not to become a grand jury-like process, where the outcomes is largely pre-ordained.

  17. Geckko: Do you have a source for your quotes? I had a quick look but couldn't locate the relevant emails.

  18. @Pielke jr
    "”However let me suggest that yoour definition of truth is slightly misleading" No not, misleading, but it does reveal me as a pragmatist;-)"

    Was Einstein wrong when he formulated 'anti-consensual" ideas about space and time which were indeed against what every other scientific believed about the basic of space and time?
    how useful & productive was it to have this “pragmatic” definition of "truth as consensus" as a criteria to judge the validity of relativity theory at the begining of the XXth century (say before Eddington's first verification of one of relativity prediction in 1918)?

    What science is good in practice is good at is establishing the falsity of a theory (or the partial falsity of a theory).

    This is a very pragmatic definition of science and it avoids all that crap about "data supporting a theory" that is constantly heard in climate science.

    Data can never supports a theory, it can only invalidate it: a good example is the article "two natural components of recent climate" change by Akasofu that shows a slight divergence between IPCC prediction and recent climate evolution. Now if the temperature down trend continues it will be more and more a refutation of IPCC prediction. It's probably too soon to conclude but this is how scientific refutation works as you know.

    It has nothing to do with consensus.

    The reason I insist on this point is that I believe that the "truth as consensus" theory is widely believed by the general public and has been used (and abused in fact) by IPCC to create an illusion of truth.

    Just my two cents.

    Thanks for a great site and great articles anyway

  19. James Hansen also sees poor judgment here, expressing a view similar to my own:

    "Question: Do the e-mails indicate any unethical efforts to hide data that do not support the idea of anthropogenic global warming, or to keep contrary ideas out of the scientific literature and the Intergovernmental Panel of Climate Change reports?

    Hansen: They indicate poor judgment in specific cases. First, the data behind any analysis should be made publicly available. Second, rather than trying so hard to prohibit publication of shoddy science, which is impossible, it is better that reviews, such as by IPCC and the National Academy of Sciences, summarize the full range of opinions and explain clearly the basis of the scientific assessment. The contrarians or deniers do not have a scientific leg to stand on. Their aim is to win a public-relations battle, or at least get a draw, which may be enough to stymie the actions that are needed to stabilize climate."


  20. -15- Sharon F.

    The fundamental issue here is not political importance. Instead the fundamental issue is that decisions that have the potential to significantly impact the health and safety of the public are being made. In this instance, the potential impacts extend literally to everyone on the entire planet.

    There are procedures and processes in place to handle such decisions, and they have been available for many decades. Take the certification of the airworthiness of commercial aircraft, new drugs and medical devices, power plants, building materials and construction, and a host of others.

    These are regulated by means of independent agencies established by local, state and, most importantly, the federal government. The examples above are handled by the FAA, FDA, NRC, and other independent agencies.

    The most important characteristics of these agencies is first and foremost their independence from both politics and the associated private industry that they regulate. And, they provide an important buffer between those who have a vested interest in the science / engineering / products / services and the public. All these independent organizations insist that everyone, without exception, associated with the organization avoid even the appearance of conflicts of interest. Other critically important characteristics of these agencies include the complete open-ness of the process and the extreme depth of the reviews, verifications, validations and quality assurance aspects of the material submitted.

    Notably, these independent agencies focus on the application / implementation of science and engineering results as embodied in a product or service. The amount of work that is required to make the transition from complete understanding of underlying principles to viable product or service is enormous. In the case of climate science, not only do the proponents openly admit that a firm grip on all the underlying processes is not yet in hand, but they propose that we move directly from this state to the final implementation of a solution; the precautionary principle and just do something.

    Not only is failure assured, but the regulatory process has been bypassed and the work required to obtain viable solution(s) has been simply ignored. The interests of only those who have a vested interest in the front-end, problem-definition work are being served. Everyone will pay for the certain failure.

    It is clearly evident to everyone that very important steps are being simply ignored and more importantly, a pre-ordained, top-down 'solution' is being enforced. The only persons to benefit by the proposed 'solution' are those who have defined the problem and 'solution'. And rich people.

    I have suggested that a Carbon Regulatory Agency ( Administration, Commission, ) be established patterned after the many existing, and very successful, independent regulators; FDA, FAA, NRC, etc. The template(s) for such an organization has been in place for decades, so the mechanics of the implementation are straightforward and easy.

    I'm not making any headway with my suggestion.

  21. Never mind Geckko, just found they came from the latest blog post by Pielke Sr., the text of which can be found here http://pielkeclimatesci.wordpress.com/2009/11/26/e-mail-documentation-of-the-successful-attempt-by-thomas-karl-director-of-the-u-s-national-climate-data-center-to-suppress-biases-and-uncertainties-in-the-assessment-surface-temperature-trends/

  22. The big question going forward is whether main stream Journals will allow these people to participate in peer review.

  23. Hansen's statement is most unsatisfactory.

    "Contrarians and deniers" is childish name calling of people who disagree with your point of view, which is symptomatic of what is wrong with climate science. That's not how science should work - the views of others should be respected, not insulted and hounded out of existence in order to construct a false consensus.

    As we have seen from the nasty tone and content of the CRU emails, the scientific process - peer review and assessment reports have been corrupted - that's not "poor judgement" it's deliberate and calculated due to politics leading science. People like Hansen and the rest of 'Climategate' crowd need to decide whether they want to be scientists or politicians - seems they've decided to be politicians dressed up as scientists.

  24. Hansen employs his trademark slander and demonstrates his inability to discern the obvious. Why would anyone find him credible?

    The databases he's responsible for are a mess. Studies he's co-authored have been revealed to be incompentent garbage (see e.g. Rahmstorf joke -- our made up numbers prove it's worse than we thought). His predictions are wildly off. And his over-the-top remarks calling for skeptics to be jailed prove he is totally unfit for any role in government or scientific assessments.

    Again, why would anyone consider his opinions worthy of any respect? If he's part of the scientific establishment, the scientific establishment isn't worthy of our respect.

  25. I'm interested in the human underbelly of this fiasco. Is it restricted to CRU? What is it that leads some scientists to a level hubris which brings then to a point in which they have total disregard for the transparent application of the scientific method. I have a family member in this field and he would never act the way some of these men acted. He has too much respect for his craft. Will we ever learn the motivations of such men?

  26. I think it is easy to see why they did it. These folks put their reputation on the line when they signed on to the predictions. Second there was much discussion in the early 1990's to wait for a climate"signal". The signal became the holy grail to prove the predictions.

  27. nanodots - not restricted to CRU - the emails were often copied or sent to or by other scientists in the USA and elsewhere.

  28. @Nanodots
    This is not restricted to CRU. There are other, supposedly respectable scientists engaged in similar behaviour.

  29. In theory, peer review is an effective error management tool. Its goal is defensive, to identify papers containing false observational data, hypotheses that are inconsistent with the accepted data, and hypotheses that are unable to predict/postdict experimental outcomes. Such a goal is politically agnostic.

    IMHO, in the CRU emails we see agnosticism abandoned. We see an effort to use peer review offensively, as a tool for weeding out papers that are inconsistent with the "established consensus." Peer review becomes an accreditation tool. That we find this has a deleterious effect on error management is, of course, not unexpected.

    As an aside, new technologies often allow the adoption of new strategies and new tools. IMHO, the so-called "skeptical" blogs have done a much more effective and timely job of "climate clique" error identification than the current "climate clique" peer review process. Perhaps it is time for the "climate clique" peer review process to join the web. I suggest they submit their next paper to ClimateScience and ClimateAudit for review :-)

  30. This is from a statement by Trevor Davies, pro-vice-chancellor (vice-president) for research at U East Anglia:

    The publication of a selection of stolen data is the latest example of a sustained and, in some instances, a vexatious campaign which may have been designed to distract from reasoned debate about the nature of the urgent action which world governments must consider to mitigate, and adapt to, climate change.

    Full text here: http://www.uea.ac.uk/mac/comm/media/press/2009/nov/homepagenews/CRUupdate

    Essentially, Davies argues that this whole thing is a distraction. He also says that they are willing to debate with anyone about anything, as long as it is about urgent action. In other words, Davies starts from the conclusion.

  31. Apparently JOHN HOLDREN is an email participant.

  32. Richard,

    Regardless of where one stands wrt to the content of the emails, do you not agree that the timing of their release (i.e. right before coppenhagen) is suggestive of other motives?

    Roger, you once alluded to tone and tastes when I asked you why you chose to focus on certain topics and not others. In this particular instance I'm surprised that you haven't had anything to say about private e-mails being released into the public domain and the impact that this is likely to have on practitioners of academic/public research...

  33. I suggest reading a couple of articles: "Why Most Published Research Findings Are False" http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
    Also a more recent article: "Genetic 'breakthroughs' in medicine are often nothing of the sort" http://www.guardian.co.uk/science/2009/nov/06/genetics-medicine-depression-significance

    We have several things happening here:
    1. Journals don't want to use space reporting negative results.
    2. Scientists need to be published for career reasons, so they have an incentive to find statistically significant results in their data.
    3. They also know that it will be harder to get published if they publish results that will contradict the results of likely reviewers, so they will be less likely to come to conclusions that challenge the accepted wisdom.

    One way to get around this is to get rid of the whole dead tree mode of publishing. Let scientists put their work in an electronic database where it can be reviewed, ranked and referenced by other scientists and all results, positive and negative are available.

  34. During the Ozone controversies of the 1970s and 1980s, there were similar things going on. I have atmospheric scientists on record who said as much, e.g.:

    "The people who I knew who I would have thrown out of the review process were the perennial troublemakers. In other words, they were given four chances to shoot it down and they each try, and after four times you say: Do a better job! People should not bring you disproofs that have obvious flaws in them all the time."

    One prominent scientist who was on the receiving end put it this way:

    "Unconsciously, they formed a tribe that has a mutual self-interest in sustaining the ozone crusade. They do not like to have critical comments made from outside, it’s understandable, it’s human. The scientists formed a community, they were ably led and that was it. They all agreed. Nobody would want to disagree. If you had been at some of those meetings you would know it. I will never forget the hostility if you got up and suggested anything that was contrary to the message."

    You can read more in my 'Transnational Environmental Policy', published by Routledge and available as Google Book.

  35. @Mike
    There's indeed an email by Holdren. Nothing untoward, but it does mean that people with no interest in climate will get involved too.

    There's also an email discussing NOAA funding. This may lead to a financial audit.

    There's another email about ante-dating an article so that it could be included in the IPCC.

    The BBC had these emails a month before anybody else, so the timing of the release was planned.

  36. Marlowe- re the timing of the emails..it sounds like they may have been trying to influence discussions in Copenhagen.
    Lancet published its "health and climate" yesterday..it published specifically to affect Copenhagen.
    "The Lancet Series on Health and Climate Change examines the public health benefits of reducing greenhouse-gas emissions ahead of the UN Climate Change Conference in Copenhagen, December 7–18. "

    Since climate change policy is a political issue, scientists on both sides (and journals) are behaving politically and following political sets of rules (which there don't seem to be many of, in this world;)).

    If folks are cloaking a political agenda under the blanket of science, is it wrong to attempt to see what's under the blanket?

  37. Timing of the release planned? What about the intensifying climate alarmism as December 11th approaches? This is only up to 24th November:

    'Teenager Wins Trip to World Climate Change Summit' - 24th Nov: http://news.bbc.co.uk/1/hi/england/gloucestershire...

    'Australia PM Presses revised Carbon Emissions Scheme' - 24th Nov: http://news.bbc.co.uk/1/hi/world/asia-pacific/8375...

    'Copenhagen: What's your Solution?' - Tue 24th Nov: http://news.bbc.co.uk/1/hi/talking_point/8375378.s...

    'Climate is major cause of African conflict' - Tue 24th Nov: http://news.bbc.co.uk/1/hi/sci/tech/8375949.stm

    'US will Announce Climate Targets' - Tue 24th Nov: http://news.bbc.co.uk/1/hi/sci/tech/8375248.stm

    'Global Warming Science Alarming' - Tue 24th Nov: http://news.bbc.co.uk/1/hi/sci/tech/8375576.stm

    'Climate Change: Copenhagen in Graphics' - Tue 24th Nov: http://news.bbc.co.uk/1/hi/sci/tech/8359629.stm

    'State Leaders boost to Copenhagen' - Mon 23rd Nov: http://news.bbc.co.uk/1/hi/world/europe/8373551.st...

    'Climate Change: What Price Will Future Generations Pay?' - Mon 23rd Nov: http://news.bbc.co.uk/1/hi/sci/tech/8374965.stm

    'Is Cumbria a victim of Climate Change?' - Mon 23rd Nov: http://www.bbc.co.uk/blogs/thereporters/markeaston...

    'East Antarctica Ice Sheet Losing Mass' - Sun 22nd Nov: http://news.bbc.co.uk/1/hi/sci/tech/8371773.stm

    'Fishermen Spot Climate Change' - Fri 20th Nov: http://news.bbc.co.uk/1/hi/in_depth/wales/2009/gre...

    And pretty lame reporting on 'Climategate':

    'UK's Climate Units Emails Hacked' - Fri 20th Nov: http://news.bbc.co.uk/1/hi/sci/tech/8370282.stm

    'Email Arguments' - Sat 21st Nov: http://news.bbc.co.uk/1/hi/sci/tech/8371597.stm

    'Hacker Leaks Scientists Emails: - Mon 23rd Nov: http://news.bbc.co.uk/1/hi/england/norfolk/8374721...

  38. From The UK Telegraph - James Delingpole blog:

    "I’ve just had a great, very sympathetic interview about Climategate on LBC radio (London’s main commercial news and talk station) with Petrie Hosken. She told me she has been simply inundated with callers, all of them utterly unconvinced that human influence has made any significant on so-called “Global Warming”. She was desperate to get a few balancing calls from people who do believe in AGW but just couldn’t find any.

    Can you imagine this happening a year ago? Or even a month ago? Until Climategate, we “Sceptics” were considered freaks – almost as bad as Holocaust deniers – beyond the pale of reasonable balanced discussion. Suddenly we’re the norm. Climategate has finally given us the chance to express openly what many of us secretly felt all along:

    AGW is about raising taxes; increasing state control; about a few canny hucksters who’ve leapt on the bandwagon fleecing us rotten with their taxpayer subsidised windfarms and their carbon-trading; about the sour, anti-capitalist impulses of sandal-wearing vegans and lapsed Communists who loathe the idea of freedom and a functioning market economy.

    We know it’s all a crock and we’re not going to take it.

    This is our Berlin Wall moment! They can’t stop us now!"

  39. "There's another email about ante-dating an article so that it could be included in the IPCC."

    That was the Wahl and Amman attempt to stick Humpty-Dumpty back together again. You can read the story here:


  40. Richard Tol, it looks like the story about the BBC sitting on the emails is either a typo (Hudson says October 12, RC got knowledge Nov. 12), or Hudson was cced in Oct. by someone at CRU re his earlier piece with a series of emails critical of it, or Hudson was sent a set of emails authored by the CRU people critical of his earlier piece FROM the hacker. Last option seems most interesting but least likely right now.

    PS. How's the hair? Still freaky?

    Otherwise, Roger, objecting to this:

    "The emails indicate concerted efforts to reshape the peer review process by managing and coordinating reviews of individual papers, by putting pressure on journal editors and editorial boards, by seeking to stack editorial boards with like-minded colleagues, by arranging boycotts of journals and other actions involving highly questionable ethics."

    ...seems more than a bit sucky. You want to read about bitching and back biting and plotting and scheming among scientists, Google "Clade Wars" or read Hull's "Science as Process". Conclusion: a 1000 times worse than this everything came out all right in the end.

    This stuff is child's play, which is why most people with any practical knowlege of how science actually operates aren't terribly shocked by it. Don't know what your problem is.

  41. Roger. You wrote "The sustainability of climate science depends upon our ability to distinguish the health of the scientific enterprise from the politics of climate change. "

    In my cynical view, there is essentially no difference between the two things. Margaret Thatcher set up the CRU and Hadley precisely to promote AGW. She appointed John Houghton (an AGW enthusiast) and he appointed the rest of them.

    It is clear that Britain plays a dominant role in this field and they managed to gather a cabal that included Hansen, Schmidt, Mann and the rest of the RC squad around themselves. The information from the leak confirmed the above view.

    Post Thatcher, Tony Blair set up and pushed through Kyoto (actually deputy prime minister John Prescott) and Gordon Brown has held pre Copenhagen meetings.

    More generally, we no longer live in the '70s and '80s where we contrasted socialism with capitalism or government and private enterprise. Today there is only international capital and those that serve it. They played national governments against each other until they became powerless.

    The power of the individual employee to be independent, honest and live by principles has vastly diminished whether he/she cares to admit it or not. I know that in my former employment as a college lecturer the offer of an early retirement scheme gave rise to a stampede of the most respected, longest serving and hardest working lecturers and managers.

    We are no longer in Kansas

  42. Yo, eric144, I'm not sure you're even on planet Earth anymore.

  43. Richard Tol said..."This is from a statement by Trevor Davies at U East Anglia:

    "The publication of a selection of stolen data is the latest example of a sustained and, in some instances, a vexatious campaign which may have been designed to distract from reasoned debate about the nature of the urgent action which world governments must consider to mitigate, and adapt to, climate change."

    This is where I have a profound problem. Why is this individual in charge of research, expressing a political view prefaced on the science debate being over ?

    Why has virtually every science representative body in the world expressed an opinion about global warming ? From chemistry to biological warfare, they know that AGW is an absolute certainty.

    Do they have a view on the extinction of the Bulgarian wombat, the colour of black holes or the application of cold fusion to the soft drinks industry ? Apparently not.

    It's politics, not science.

  44. Marlowe, if someone sincerely and honestly believed there had been misconduct as implied by these emails, wouldn't they naturally want them released before the Copenhagen meeting? It would appear that your complaint about the timing comes down to being unhappy that the damaging emails were released when they would have the most effect.

    Similarly, on the alleged release of this data as the result of an external attack, something I've noted in, eg, the coverage by the New York Times is that there seems to be a certain fastidiousness about the release of this data, which may have been obtained illegally, but which appears very likely was being withheld unlawfully; compare that with the rather divergent reaction to the Times' publication of various US military secrets, which were almost certainly obtained illegally, but were being withheld lawfully.

    Or, to put it more concisely: convince us that your concern doesn't reduce to "but it was my ox that was gored this time!"

  45. bigcitylib # 40 -- "Don't know what your problem is."

    The problem is that these people are trying to rush us into spending trillions of dollars without properly considering all the appropriate aspects and uncertainties of the problem. I don't remember the exact quote, but someone once said that academic politics are so vicious because so little is at stake. That is not the case here...

  46. Peer review is sometimes useful in improving papers but just as often I think it makes papers worse as authors have to respond to inane referee comments. Online working papers are very big in economics now (RePEc and SSRN) and get far more citations than book chapters but fewer still than journal articles. I'm increasingly thinking that journals are redundant (though I do a lot of reviewing an am associate editor of Ecological Economics) and we should just rely on citations to assess quality.

  47. At a luncheon this week, spoke to multiple colleagues about the recent revelations going on in the climate science arena. To the person, all were stunned at how corrupted the whole process had become. Although none of us are in the climate science field, this latest lapse in the scientific and peer review process was more than enough to push the remaining few supporters to start muttering heretical and skeptical thoughts.

    This type of "child's play" is usually stomped down fairly quickly in other science endeavors. Climate science needs some adults to quickly assert control over their immature children. Truly an embarrassment, and a warning, for all scientists.

  48. As many of us feared, the over-hype bubble has burst and it's taking down the reputations of general science and environmentalism.

    Meanwhile there are still serious environmental issues such as the destruction of mangroves, coral, dead zones and overfishing. Somehow these aren't considered as apocalytic, though they are probably more serious than any AGW.

    Bangladesh needs artificial dykes built, Africa needs basic water drilling and the first world needs to deal with fertilizer runoff and coastline pollution, but where's the money for that? It disappears in penpusher impacts reports and costly talks about talks about trading schemes to reward the rich.

    While the activist scientists stroked their collective egos and (with the bureaucrats) bled the public purse, and while politicians pretended to care about future generations, millions of the current generation died already from more simple and curable problems like contaminated water, mosquito bites, fishing ground contamination, pointless wars or just basic, grinding poverty.

  49. bigcitylib

    Yo,yo,yo dude

    I deliberately kept it extremely short and suggestive, rather than literal. Probably the wrong tactics on a science forum. The literal version follows.

    The message was that Margaret Thatcher created global warming as a political issue in order to further her (extremely well recorded) war against coal. She created the CRU and Hadley precisely for that purpose. Tony Blair and Gordon Brown have continued to lead global warming as a political issue.

    Science is 100% owned by government/big business. Scientists are employees. They do what they are told. Any talk of the integrity of science or scientists is therefore largely redundant.

    Most scientists aren't under this intense political pressure so behave with the highest probity, but there are large areas where their 'morality' is extremely questionable like GM food and an incredible range of very, very nasty weapons.

    Further, the idea that so called left wing figures like Soros (arguably the most successful financial criminal in human history), Gore or the current American president (biggest donor Goldman Sachs) represent the citizens against big business is completely insane.

    That is the political premise on which global warming has been sold. Gore vs Exxon. Gore, the man, who like his father, spent his whole senatorial career as a front for Occidental Oil.

    The emails confirm that climate science is a cesspit of manipulation and desperate attempts to make the 'science' fit the policy. I was also speculating that the CRU was the centre of the cabal we know so well, with British scientist Gavin Schmidt also running RealClimate.



    Thatcher at the UN


    Margaret Thatcher: Speech opening Hadley Centre for Climate Prediction and Research


    Sir John Houghton and Phil Jones (Thatcher)


  50. Prof Peilke and interested others:

    Willis Essenbach has uncovered and circulated an email I sent 6 years ago but I had forgotten. It is directly relevant to the actions of CRU staff being discussed here, so I copy it here. Please note its original circulation list and contents, especially its final sentence.


    From: RichardSCourtney@aol.com
    To: t.osborn@uea.ac.uk, m.allen1@physics.ox.ac.uk, Russell.Vose@noaa.gov

    Subject: Re: Workshop: Reconciling Vertical Temperature Trends

    Date: Sun, 23 Nov 2003 18:42:59 EST

    Cc: trenbert@cgd.ucar.edu, timo.hameranta@pp.inet.fi, Thomas.R.Karl@noaa.gov, ceforest@mit.edu, sokolov@mit.edu, phstone@mit.edu, ekalnay@atmos.umd.edu, richard.w.reynolds@noaa.gov, christy@atmos.uah.edu, roy.spencer@msfc.nasa.gov, benjie.norris@nsstc.uah.edu, kostya@atmos.umd.edu, Norman.Grody@noaa.gov, Thomas.C.Peterson@noaa.gov, sfbtett@metoffice.com, penner@umich.edu, dian.seidel@noaa.gov, trenbert@ucar.edu, wigley@ucar.edu, pielke@atmos.colostate.edu, climatesceptics@yahoogroups.com, aarking1@jhu.edu, bjorn@ps.au.dk, cfk @lanl.gov, c.defreitas@auckland.ac.nz, cidso@co2science.org, dwojick@shentel.net, douglass@pas.rochester.edu, dkaroly@ou.edu, mercurio@jafar.hartnell.cc.ca.us, fredev@mobilixnet.dk, seitz@rockvax.rockefeller.edu, Heinz.Hug@t-online.de, hughel@comcast.net, jahlbeck@ab

    Dear All:

    The excuses seem to be becoming desperate. Unjustified assertion that I fail to understand “Myles’ comments and/or work on trying the detect/attribute climate change” does not stop the attribution study being an error. The problem is that I do understand what is being done, and I am willing to say why it is GIGO.

    Tim Allen said;
    In a message dated 19/11/03 08:47:16 GMT Standard Time, m.allen1@physics.ox.ac.uk writes:

    “I would just like to add that those of us working on climate change detection and attribution are careful to mask model simulations in the same way that the observations have been sampled, so these well-known dependencies of nominal trends on the trend-estimation technique have no bearing on formal detection and attribution results as quoted, for example, in the IPCC TAR.”

    I rejected this saying:
    At 09:31 21/11/2003, RichardSCourtney@aol.com wrote:

    “It cannot be known that the ‘masking’ does not generate additional spurious trends. Anyway, why assume the errors in the data sets are geographical and not ‘?’. The masking is a ‘fix’ applied to the model simulations to adjust them to fit the surface data known to contain spurious trends. This is simple GIGO.”

    Now, Tim Osborn says of my comment;
    In a message dated 21/11/03 10:04:56 GMT Standard Time, t.osborn@uea.ac.uk writes:

    “Richard’s statement makes it clear, to me at least, that he misunderstands Myles’ comments and/or work on trying the detect/attribute climate change.

    As far as I understand it, the masking is applied to the model to remove those locations/times when there are no observations. This is quite different to removing those locations which do not match, in some way, with the observations – that would clearly be the wrong thing to do. To mask those that have no observations, however, is clearly the right thing to do – what is the point of attempting to detect a simulated signal of climate change over some part of (e.g.) the Southern Ocean if there are no observations there in which to detect the expected signal? That would clearly be pointless.”

    Yes it would. And I fully understand Myles’ comments. Indeed, my comments clearly and unarguably relate to Myles comments. But, as my response states, Myles’ comments do not alter the fact that the masked data and the unmasked data contain demonstrated false trends. And the masking may introduce other spurious trends. So, the conducted attribution study is pointless because it is GIGO. Ad hominem insults don’t change that.

    And nor does the use of peer review to block my publication of the facts of these matters.

    Richard S Courtney

  51. It’s astounding that even now, there is so much bending over backwards to politely justify what Mann and his accomplices at the CRU have done or to "understand" it in the same manner of an indulgent parent does a misbehaved child. The CRU files are clearly damning. These people have committed criminal acts; namely, that they have repeatedly acquired and used public funds (over many years in the $10Ms) to deliberately falsify, deceive, and mislead, and they have colluded to do the same. On top of this, they have colluded to resist challenges and discovery of their acts and the bankruptcy of their work and results. To add insult to injury, when challenged with valid data and calm analysis by those who disagree with them, they call names, rather than replying in kind (as would true scientists). If the base issue dealt with accurately counting the number of fairies on the head of a pin, the consequences of their behavior would simply be a sad footnote. But given the scope and magnitude of the base issue, the consequences could be globally catastrophic, for billions of people. They should be prosecuted to the maximum extent in as many jurisdictions as available.

  52. I agree with other posts that peer review is necessary but not sufficient to ensure that information is correct.
    In addition to David's point about the questionable comments of the reviewers, wouldn't everyone learn more if publicly a variety of reviewers could put out his or her comments and the author and others could respond in a transparent way?

    The authors would be required to post the model, raw data and the procedures they used for quality assurance, quality control and model verification and validation, where appropriate.

    Government could then make a rule that any scientific studies which have strong evidence requiring policy responses that would involve 1 or more billion dollars, would have to go through this process.
    Official scientific or policy advisory panels could only use scientific studies which had been through this screening process.

    (PS in rulemaking in the US, we already have to pass a higher bar of analysis when the economic impacts of the rulemaking are greater)

  53. There is one point about which we should not be confused. Contrary to the claims of certain media outlets and certain individuals, the CRU emails are NOT private correspondence.

    Emails sent at the place of one's employment, bearing the name of one's employer, and pertaining to one's on-the-job responsibilities, belong to said employer.

    If that employer happens to be funded with public money, such emails are almost certainly subject to Freedom of Information legislation.

    See this link, on the University of East Anglia's website http://www.eua.ac.uk/is/foi/guidance The section titled "5 key facts that all staff should know about Freedom of Information" explictly advises that all email could potentially be released to the public and should, therefore, be "clear and professional."

    As others have observed, none of this email appears to discuss staff meetings, picking up milk on the way home, or the cost off little Johnny's braces.

    These are NOT private email. Repeated attempts to suggest that they are is spin, plain and simple.

  54. Dr. Pielke, in light of some of your comments reflecting what is perhaps an idealized view of the peer review process in science as a whole, you may wish to consider the paper "New Ideas in Science" by Dr. Thomas Gold.


  55. "The problem with this strategy, of course, is that many climate scientists (and presumably others inside and outside of the scientific establishment) are unwilling to cede ownership of the "truth" to a small clique of scientists."

    And so they enfeeble the self-correcting mechanism of Science. Lysenkoism was finally done away with eventually because it did not fit reality (in painful ways).

    I hope people are very careful in taking short cuts to truth.

  56. The question of whether these emails should have been private, or whether peer review or even science as a whole are politicized and subject to sorruption is not what is critical. There have been numerous instances in the history of science were either a "concensus", "religious affliction", or the emegerence of powerful groups and interests have suppressed the critical advance of science for perhaps as much as a generation. Some of us grasp that as normal and that eventually real science wins out anyway. But that can take decades, and the AGW crowd is asking the entire planet to place an enormous bet - they are claiming that the single thing - energy, most responsible for every benefit of the modern era. What has not only created fabulous wealth but increased standards of living and life expectancy accross the entire planet. That we must radically curtail. And they ask our political leaders to do this based on science that it is increasingly clear is poorly tested. This is not about the details of the human family tree, or when did the first people arrive in north america or whether the earth is the center of the universe. Getting this substantially wrong - either way, could have devestating consequences for the entire population of this planet. As bad as the AGW crowd may think the harm of an increase of 2C might be - and no one has really and truly looked at those consequences, 8B people in substantially increased poverty is on hell of a price to pay for the normal politics of science. Outside the spotlights the demand of most skeptics is to subject this entire thesis to real scrutiny. It is not science if the raw data as well as all the details of the data analysis are not public. It is not science when criticism and disent even over minor issues and even inside the AGW camp are not tolerated. It is not science if studies and results can not be independently verified by anyone that wishes. The Hadley debacle is not so damaging because it proves skeptics wrong - it does not. But because it shows even proponents how substantial portions of the AGW argument are religion rather than science. The most damaging revelations personally are the supression of internal disent. The entire climate community - especially beleivers with questions, now has reason to distrust their pinnacles of power.

  57. Thanks for sharing this, Jason. Dr. Gold seems to be saying that scientists behave as other humans and the structure of the current science establishment does not have what it takes to counter that.
    The science court idea, I think, is worth pursuing for highly important scientific issues. With transparency of the discourse and data being fundamental.

  58. It seems to be uncontroversial that peer review is not perfect. By its very nature, most of its daily practices are confidential and therefore not in the public domain. In many cases a journal editor will be able to influence which papers get published and which get rejected. There are initiatives to make the review process public online, with comments from reviewers and others visible. For better or worse most publishing is still done in the old fashioned, secretive way.

    This would normally not be a problem (apart from back biting, effects on academic careers, etc). However, in the field of climate change things are different. Not because the stakes are high (as many commentators say these days, post CRUgate) but because we have the IPCC.

    This intergovernmental body establishes a different set of rules for the game. The game is no longer TRUTH SPEAKING TO POWER but managing a consensus view that is -- by the very definition of the IPCC -- international, scientific, and political. Everyone in it must be ON MESSAGE.

    This means that the 'dominant tribe' operating within the IPCC has far more leverage compared to other instances of influencing knowledge production and dissemination. Because the IPCC consensus is all that counts, outsiders do not normally have a voice.

    In a world where scientific controversies can unfold in the 'normal way' (through peer reviewed papers, without a world wide coordination attempt) we would sooner or later see all relevant research findings expressed.

    The architects of the IPCC did not want this. They thought politicians and the public will be confused if there are too many viewpoints on climate change. But from day one of its existence, the IPCC had to fight 'contrarians' and tried to tighten the grip on the CONSENSUS ever more. Political nervousness was the result within core groups of science.

    The IPCC is counterproductive both for science and for politics.

  59. Another irritating thing about our current approach to dissemination of publicly paid for science- you need to pay journals to view it. So if you as a policy maker or interested member of the public, don't have access to a scientific library, you must rely on the judgment of others to apply it to policy.

  60. True, Reiner Grundmann, that's an angle I have not thought of ... But consensus should not mean majority optinion. Consensus should relate also to disensus - consensus that we disagree. Then, everything would be fine, but if consensus is what a powerful group claims to be consensus, that is different.

  61. Peer review and bunker mentality all in one.


  62. Love the cartoon - really depicts the process. Great blog - good to see you finally vindicated!

  63. -37 Paul Biggs:

    The Daily Telegraph reports that it is no coincidence that the BBC withheld the news about CRU. They quote former news presenter
    Peter Sissons claiming 'it is now "effectively BBC policy" to stifle critics of the consensus view on global warming.

    He said: "The Corporation's most famous interrogators invariably begin by accepting that "the science is settled", when there are countless reputable scientists and climatologists producing work that says it isn't.'

    Good to have freedom of press, although the Telegraph usually is not my cup of tea. Takes a visit to the dentist to find out.

    "But it is effectively BBC policy... that those views should not be heard."

  64. I believe we need to step back and consider what policy changes (in various arenas) would help avoid both the reality and the perception of bias and infidelity to scientific principles. This is not too different from conflict-of-interest policies.

    As a start, some things to consider encouraging:

    * Reproducible Research (google it; Stanford U and others are making this a standard practice)

    * Truly independent peer review (perhaps even going so far as to develop public lists of qualified reviewers in various fields, and using public random selection for selection.)

    * Requiring disclosure of the extent to which analyses make use of cherry-picked data (i.e. it's fine to filter on metadata, but attributes of the data itself should be avoided for selection purposes)

    I'd be more extreme on this last one but apparently a multidisciplinary discussion is needed before climate scientists will accept that their field needs to learn how to better avoid confirmation bias.

  65. I appreciate the quality of the discussion here on the peer review process. There are two things in particular I (as a non-scientist) agree on:

    1) We need a "science court" with coverage like C-SPAN. The government ought to promote education thoughout our lives, and via TV and the Internet ought to strive to make science as accessible as possible.

    2) We need more open access to the journals so that concerned citizens can take the initiative to educate themselves. I'd be happy with either a more widespread micropayment system for per-article access, or a trade organization of journals that provides a low-cost joint membership (or maybe a combination?)

    I found this piece by Isaac Asimov somewhat useful as I considered my own set of bigger questions that have been raised by the CRU incident: The Relativity of Wrong.

  66. The most damning criticism comes from the emails themselves:
    >As for thinking that it is "Better that nothing appear, than something unnacceptable to us" .....as though we are the gatekeepers of all that is acceptable in the world of paleoclimatology seems amazingly arrogant. Science moves forward whether we agree with individiual articles or not....
    -- Raymond S. Bradley, 0924532891.txt