15 February 2013

Science is the Shortcut

UPDATE 20 Feb 2013: Five days after writing this critique I was asked to step down from the GEC editorial board.

There is a new paper out by Brysse et al. in Global Environmental Change (here $) which includes as co-authors Naomi Oreskes (author of Merchants of Doubt) and Michael Oppenheimer (long-time IPCC contributor and a contributing lead author for the AR5). The authors report a remarkable finding -- they have identified a shortcut to divining the true state of knowledge of the science of climate change.

As the authors' explain:
Evidence from recent analyses suggests that scientists, particularly acting in the context of large assessments, may have underestimated the magnitude and rate of expected impacts of anthropogenic climate change. We suggest that this underestimation reflects a systematic bias, which we label “erring on the side of least drama (ESLD)”.
What ESLD therefore means is that when scientists make a claim about climate change, particularly via the IPCC and other assessments, the presence of a systematic bias indicates that the odds are that things are really much, much worse. ESLD therefore offers a short cut to anticipating where climate science is headed.

An important reason for this bias, the authors assert, is of course none other than those evil skeptics:
[O]ne possible reason why scientists may have underestimated the threat of anthropogenic warming is the fear that if they don’t, they will be accused by contrarians (as was Schneider) of being alarmist fear-mongers. That is to say, pressure from skeptics and contrarians and the risk of being accused of alarmism may have caused scientists to understate their results.
Not only is the accusation of a systematic bias an insult to the integrity of practicing scientists, but the entire paper is built on an empirical foundation that does not touch the ground.

Let's take a closer look at the data claimed to support the ESLD hypothesis. The paper examines literature on IPCC predictions for temperature and sea level, hurricanes, and the role of greenhouse gases from permafrost melting in climate models.
  • It finds that sea level is running well below the 1990 IPCC prediction and above the 1995 and 2001 predictions (if anyone can make sense of 2007 IPCC predictions then you get a bonus point). 
  • It finds that observed temperature increases are consistent with the predictions of all 4 IPCC assessments.
  • It finds that the IPCC accurately reflects the community understanding on hurricanes.
  • It finds that peramfrost melting is not included in climate models, representing a "potentially profound bias in the climate projections—not toward overestimation of climate change, but toward its underestimation."
  • They also cite Arctic sea ice and some science on rainfall.
So here is my tally: 3 sea level predictions + 4 temperature predictions + AR4 hurricanes + permafrost + rainfall + Arctic sea ice = 11 data points. Of these 11, according to Brysse et al. the scientific community has been accurate on 5, overestimated the near term evolution in 1 case and underestimated in 5. Are you convinced of ESLD?

Consider also the figure below on global temperatures, courtesy of Ed Hawkins (@ed_hawkins) at the University of Reading (for discussion see his post here). The figure shows that for the CMIP5 climate models (i.e., those to be used in IPCC AR5) observed temperatures are running at the low end of the ensemble of predictions, reflecting the recent "standstill" in global temperature (the figure also calls into question the Brysse et al. conclusion that global temperatures are consistent with all 4 previous IPCC reports, but I digress).

Least drama? Hardly.
What can be said from all this? Well pretty much nothing.

Consider that the 2007 IPCC report alone had 2,744 findings, almost all of which were reported in probabilistic fashion. Evaluating the accuracy of those findings comprehensively against the evolution of the climate system would be difficult if not impossible, both empirically and epistemologically. Further, one could easily pick out a few findings from the report which tell a different story: drought, methane emissions, flooding, disaster costs, Himalayan glacier melt and so on. In 2010, Robert Watson, a former Chair of the IPCC, noted of the errors discovered in the AR4 report: "The mistakes all appear to have gone in the direction of making it seem like climate change is more serious by overstating the impact. That is worrying." A Dutch assessment of the IPCC AR4 found much the same.

For some reason Brysse et al. neglected to consider a 2010 paper (co-authored by Michael Oppenheimer) which warned of the threat of a dramatic increase in poor Mexicans migrating north into the United States due to climate change. Talk about drama! However, an unnoticed 2012 paper in the same journal found that the original migration paper contained some serious methodological flaws, so never mind. This apparently was a case of Erring on the Side of Too-Much Drama (ESTMD - I can make up scientific-sounding concepts too). We could go on like this all day, and it would not provide any enlightenment.

Is there any evidence that climate scientists exhibit a systematic bias in their published work and assessments due to outside pressure or other factors? No.

As the ESLD paper shows, ironically enough, sometimes work that is badly off base gets into the literature and even scientific assessments are far from perfect. Nonetheless, science is the best route we have to gaining an understanding of the world that we live in. I'm sorry to say that there are no shortcuts. Or perhaps put another way, science is the shortcut.