26 January 2010

Lord Stern's Spokesperson Responds

Bob Ward, Policy and Communications Director at the Grantham Research Institute at the London School of Economics, where Nicholas Stern in the Chair, has kindly written in to the comments, responding to my post on how the Stern Review Report misused the Robert Muir-Woods paper and quietly altered a typo that revealed the fuzzy math. Bob's comments are below, with my responses in the inset boxes, with a black line around them. I thank Bob for his engagement.
Roger,

I see you have taken advantage of the media frenzy around the IPCC to gain more coverage for your gripes on disaster losses. However, I’m afraid there are some serious errors in your public statements.
I am happy to see that we both share an interest in correcting errors. If my statements are in error, then I will correct them.
As you know, I formerly worked with Robert Muir-Wood at Risk Management Solutions and I now work with several of the people who prepared the Stern Review, so I have had the chance to speak to them about your comments. I am providing my comments in multiple entries on your blog as it will not allow them to be all posted at once.
I am happy to give your comments top line billing, as all views are welcome here. Based on these comments, I assume that you are writing as the representative of Nicholas Stern.
First, let me clarify a few points about the Stern Review. You are quite right, and as the UK Government acknowledged, there was a typographic error in Table 5.2 of the version of the Stern Review that was published on the Government’s website in October 2006 –the increased costs, as a percentage of GDP, from US hurricanes was inaccurately printed as 1.3% and 0.6%, instead of 0.13% and 0.06%. The error was picked up by members of the team and corrected before the report was printed by Cambridge University Press for publication in January 2007. The error had not other significance for the report other than in Table 5.2. There was no conspiracy and no attempt to deceive – it was a straightforward typo which was corrected very soon after the Review appeared.
The FAQ page for the report says that an Errata was to be prepared explaining why there were multiple versions of the report available. Can you point me to this errata? The problem of course with multiple versions of the report is that it can cause some confusion, as my peer reviewed critique of the report relied on the older version. Surely the Stern Review should adhere to the same standards as a normal academic publication?
Second, you are also right that the costs of extreme weather events globally calculated as 0.5-1.0% of GDP, and presented in Table 5.2 was based on the 2% trend published in the six-page paper by Muir-Wood, Miller and Boissonnade from the Hohenkammer workshop in 2006.
Table 5.2 explains that the results are for the developed world. So far you have explained that I am correct about 2 of my assertions. I'm still waiting for the errors.
However, it is completely wrong to suggest that “as much as 40% of the Stern Review projections for the global costs of unmitigated climate change derive from its misuse of the Muir-Wood et al. paper”. In fact, the modelling of the macroeconomic costs of the impacts of unabated climate change did not make use of the Muir-Wood et al. (2006) abstract – the costs were projected using the PAGE2002 integrated assessment model, as described in Chapter 6 of the Review. The PAGE2002 model drew upon three major sources of estimates of the impacts of climate change (Mendelsohn et al. (1998), Nordhaus and Boyer (2000) and Tol (2002)) – this is explained in Chapter 6 of the Review and also in this 2007 paper from ‘World Economics’ (see particularly Figure 2): http://www.occ.gov.uk/activities/stern_papers/World_Economics1.pdf
If the lower bound of Stern Review cost estimates is 5% of global GDP and disasters are responsible for 2% of that total then, that is 2/5 = 40%. Are you now saying that the damage costs of climate change are insensitive to disaster costs? That is, the disaster costs do not matter in the Stern Review's cost estimates? This would be remarkable if so.
Further, the Stern Review points out on page 170: “Extreme weather events are not fully captured in most existing IAMs; the latest science suggests that extreme events will increase in frequency and severity with climate change.” This statement references a technical paper by Rachel Warren and co-authors (including Richard Tol) which was prepared for the Review. It points out on page 5: “Most models do not take into account the influences of extreme weather events which are likely to contribute very strongly to economic impacts; however the top end of the PAGE input ranges do include them as far as the literature allows, albeit that literature on potential changes in the frequency and intensity extreme [sic] events is in its infancy”: http://www.dfld.de/Presse/PMitt/2006/061030c4.pdf
Again, what have you said here? The Stern Review estimates for the damage costs of climate change do, or do not, include the costs of disasters?
You also complain that the Stern Review should not have cited the Muir-Wood et al. (2006) workshop paper because it was not peer-reviewed.
No. The citation is not a problem. The fact that it was not peer reviewed should have led to some caution. The problem is that the Stern Review grossly misused the Muir-Wood paper to imply a relationship of increasing greenhouse gases and rising disaster costs. Further, the Stern Review went beyond its misuse of Muir-Wood et al. to invent from whole cloth accelerating disaster costs, based on absolutly no scientific foundation.
The abstract of the paper states:

“After 1970 when the global record becomes more comprehensive we find evidence of an annual upward trend for normalized losses of 2% per year that corresponds with a period of rising global temperatures. However over this same period, in some regions, including Australia, India and the Philippines normalized losses have declined. The significance of the trend in global normalized losses is dominated by the affect [sic] of the 2004 and 2005 Atlantic hurricane seasons as well as by the bias in US wealth relative to other developing regions.”

The abstract also states:

“What is presented here provides a short summary of the global results of this study. Full results are in course of publication also covering individual peril regions and the exploration of correlations with global temperatures.”

As you know, these results were published in 2008 as a peer-reviewed paper by Miller, Muir-Wood and Boissonnade in the volume on ‘Climate Extremes and Society’. The paper reached the same main conclusions as the workshop presentation: “After 1970, when the global record becomes more comprehensive, we find evidence of an annual upward trend for normalized losses of 2% per year.” A sensitivity analysis revealed that there was no statistically significant trend between 1970 and 2005 if the losses from the 2004 and 2005 US hurricane seasons were removed. The paper concluded: “In sum, we found statistical evidence of an upward trend in normalized losses from 1970 through 2005 and insufficient evidence to claim a firm link between global warming and disaster losses.”
Did you read that last part Bob? The paper states clearly: "We find insufficient evidence to claim a statistical relationship between global temperature increase and normalized catastrophe losses."

With hindsight, at the very least, it should be clear that the use of the Muir-Wood paper as it was by Stern was simply incorrect.

Further, elsewhere I've seen you complaining about cherrypicking starting dates for assessing global temperature trends, such as when people claim that global warming stopped in 1998. You are engaged in your own cherrypicking here, when you focus on 1970-2005. There are no residual trends 1950-2005, and you can bet 1950-2009.
The paper also noted several factors that should be taken into account when interpreting the results, including “changing vulnerability over time”. This means that losses from US hurricanes and other extreme weather events did not separate out the impacts of changing building standards between 1970 and 2005, which of course, would tend to obscure any trend towards increasing losses from an increase in hazard (ie frequency and/or intensity of weather events).

Of course this limitation also applies to your paper with other co-authors in 2008 on normalized losses from US hurricanes, which simply states: “As strong codes have only been implemented in recent years (and in some cases vary significantly on a county-by-county basis), their effect on overall losses is unlikely to be large, but in future years efforts to improve building practices and encourage retrofit of existing structures could have a large impact on losses”. Although you dismiss any potential impact from improvements in building standards, and so do not take it into account, you will no doubt know that these standards improved after Hurricane Andrew in 1992 and their impact on reducing damage has been documented, for instance, in this report by the University of Florida: http://www.dca.state.fl.us/FBC/Hurricane_Research_Advisory_Committee/Report_SurveyProject_Gurley_62005.pdf

It seems to me that your analysis may have missed the impact of changes in Atlantic hurricane activity on losses (such as the increase in frequency since 1995) due to reductions in vulnerability of US coastal properties. Surely, some research is required to justify the neglect of the impact of changing building standards? Certainly you would be advancing an important area if you were to investigate this issue more thoroughly.
I welcome your thoughts on our work. We have indeed explored the possible biases that might be introduced into the hurricane loss record, and we don't see any such biases. Trends in the damage record match up perfectly with trends in landfall frequencies and intensities -- there are no such trends! So while there may be an effect, it is not large enough to appear in our dataset.

But if you are not convinced, try out this experiment. Take our loss data for 1900-2009, which is readily available online. Simply assume that all Florida (or all US for that matter) hurricanes would have been 10%, 20% or whatever percent more damaging except for those really effective new building codes. (Engineers and planners, it is OK, this is a thought experiment, of course I know the building code effect would not be so large!) Then look for a trend in the dataset since 1900. You won't find one.
Similarly, your conclusion that increases in exposure and value at risk completely explain the trend in rising losses from hurricanes is not shared by other authors. For instance, this paper by Silvio Schmidt and co-authors on US hurricane losses, published last year in the journal ‘Environmental Impact Review’, concluded: “In the period 1971-2005, since the beginning of a trend towards increased intense cyclone activity, losses excluding socio-economic effects show an annual increase of 4% per annum. This increase must therefore be at least due to the impact of natural climate variability but, more likely than not, also due to anthropogenic forcings.”: http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V9G-4W6FNC4-1&_user=1177143&_coverDate=11%2F30%2F2009&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1180772824&_rerunOrigin=google&_acct=C000051857&_version=1&_urlVersion=0&_userid=1177143&md5=2a2c466695839402c959b38b2ebfdcb0
What Silvio did, quite responsibly, is offer up a conditional, speculative statement. If GHGs are responsible for the increasing US landfalls since 1970, then yes, there is a GHG signal in the loss record. Unfortunately, attribution of US landfalls to GHGs remains unknown. As I wrote about in a recent paper on hurricanes, there have been no trends in landfalls, or in the region where landfalls occur, over the period of record. Silvio's recent papers and those I have been invovled in are in no way contradictory. It is fine to hypothesize or speculate about a GHG-disaster loss relationship. But the ultimate arbiter is the data.

Finally, you should be careful presenting only part of the story, as you will be found out. Here is what Silvio's paper actually says: "There is no evidence yet of any trend in tropical cyclone losses that can be attributed directly to anthropogenic climate change."
As I am sure you appreciate, it is important that public debate about climate change is well-informed with accurate information. Unfortunately, your inaccurate and misleading comments on your blog have now generated misleading coverage elsewhere. I urge you to exercise greater care in future to avoid making further mistakes similar to those I have outlined here.
What mistakes?
Best wishes,

Bob Ward
Thanks again.

21 comments:

ljohnson said...

Ouch. I look forward to Ward's rebuttal. When he is done licking his wounds, of course.

Jer said...

That is kind of like reading the IPCC report, a great deal of verbiage and misrepresentation to prove nothing. But it does keep people employed I guess, which in these times has to be worth something

Ian said...

Another superb bit of dissection...

eric144 said...

Excellent response Rodger

Bob Ward writes occasional anti sceptic, fire fighting pieces for the Guardian comments section, in the tabloid style of the esteemed George Monbiot.

http://www.guardian.co.uk/environment/2009/nov/20/climate-sceptics-email-hacking


The man who owns the Grantham Institute (including Bob and Nick Stern) is the very generous controller of an $85 billion investment fund, Jeremy Grantham.

http://en.wikipedia.org/wiki/Jeremy_Grantham

Charlie said...

Roger Pielke, Jr: "I assume that you are writing as the representative of Nicholas Stern."

I recommend that you e-mail Nicholas Stern to ask if Bob Ward accurately represents Stern's position on the above matters.

If the answer is "no", then perhaps Stern can issue a statement in his own words.

Richard Tol said...

Bob Ward is right about one thing: The numbers in Table 5.2 are NOT used in the rest of the Stern Review.

The "cost-benefit analysis" of the Stern Review contains "catastrophic losses" but these are NOT the impacts of extreme weather.

Roger Pielke, Jr. said...

-6-Richard

The issue is not whether the numbers in the Table were used directly in the cost estimates, the issue is one of "additionality" (or maybe "subtractionality";-)

If Stern says the costs are 5%-20% of GDP, does it matter if Stern estimates extreme events as 0.2% or 2%?

If not, that says something about the analysis.

Richard Tol said...

-7-Roger
Stern's 5-20% does not include extreme weather.

As you have shown here, Stern's numbers about extreme weather are nonsense.

As I and other have shown here, Stern's total impact estimates are nonsense too.

What share does one nonsense contribute to another nonsense?

Roger Pielke, Jr. said...

-8-Richard

Well said!

bernie said...

I am beginning to feel like Alice in Wonderland?

Bob Ward said...

Roger,

I am very flattered by the prominence that you have given to my views. But I am still puzzled that you remain so certain that climate change has had no impact on disaster losses. Other researchers have been far more cautious about drawing such a definitive conclusion, given the limited nature of the data. Isn’t there a danger that you are confusing ‘absence of evidence’ with ‘evidence of absence’.

For instance, you insist that the well-documented increase in hurricane activity in the Atlantic basin since 1995 (and it’s not clear to what extent, if any, climate change has played a role in the increase) has not increased the risk of landfall along the US coast, and you point to the absence of any detectable trend in landfalling data. But as my former colleagues at RMS pointed out in this paper on the ‘Five-year Prediction of the Number of Hurricanes that make U.S. Landfall’ (http://www.rms.com/Publications/Research/Jewson.pdf), which was published last year in this volume on ‘Hurricanes and Climate’ (http://www.springer.com/earth+sciences/meteorology/book/978-0-387-09409-0?detailsPage=toc), one would not expect to be able to detect the increase in the landfall record, not because the increase has not happened, but simply because the signal-to-noise ratio is too low for the increase to be distinguished from the variability in the data.

Given that it is so difficult to detect from landfall data alone changes in hurricane activity, and the fact that one cannot quantify the impact of changes in vulnerability on losses, surely you should be more cautious about discounting any impact from climate change?

Best wishes,

Bob

Roger Pielke, Jr. said...

-11-Bob Ward

Thanks for following up.

Recently, I addressed explicitly Steve Jewsen's et al.'s point, which is perfectly accurate, in a recent paper. Here is what I said: "In practical terms, on timescales of decision making a signal that cannot be seen is indistinguishable from a signal that does not exist."

If there is an impact from [human-caused] climate change in current disaster losses, yet that impact is too small to detect, then why are we debating it? The literature is robust and speaks with one voice on this subject. There is no "absence of evidence" on which to reach conclusions.

I have repeatedly said that such a signal may reveal itself in the future, and may in fact be large. However, the fact that we cannot see it now is the way the numbers fall out. Shouldn't we just say as much and move on? It is of course important to carefully distinguish what we see looking backwards in time and what we expect for the future.

This issue has nothing to do with responding to climate change, which is important and necessary both as mitigation and adaptation. This has to do with a faithful representation of the state of the science, which is not only the right thing to do, but helps to build trust and confidence in the experts that we all must reply upon.

Climate policy can survive admission that we don't (yet) see a GHG signal in the disaster record, even though experts have different opinions about if and when we might. When a signal can be seen in the loss record, I'd love to be the first to announce it. Until then, I won't be speculating about signals that can't be seen and trends that have yet to appear. That is not how empirical science is done.

Craig said...

If climate policy can survive missing GHG signals, can it survive the ongoing saga of climategate? The Times of London has the story: http://timesonline.co.uk/tol/news/environment/article7004936.ece

"Scientists in stolen e-mail scandal hid climate data"

Charlie said...

Bob Ward #11 says "Isn't there a danger that you are confusing 'absence of evidence' as 'evidence of absence'.

It appears that you are confusing the inability of the data to prove non-existence of an increase of damage due to AGW as proof positive such an increase does exist.

I cannot prove that leprachauns do not exist. That isn't proof that they do exist.

Oliver said...

-12- Roger

I think your point is well made. But I wonder if "shouldn't we just say [we can't see it] and move on" might not be a little too simple. It seems to me that one might well also want to say how big the effect would have to be if we were to be able to see it, through some sort of monte-carlo/historical record/model-based sensitivity analysis. There may well be reasons why this could not be done convincingly, and if so that's fine, but without knowing that it is impossible it seems to me that saying "there is no discernible trend, but a trend of such and such would not be discernible with current data" (or "there is no discernible trend, and any trend above such and such would, we think, be discernible", depending on your tastes) would be an improvement over saying "no discernible trend" and moving on.

Oliver said...

-- 6 to 9 -- Richard and Roger

You seem to be agreeing to disagree. Richard is firmly maintaining that the trend figure from Muir-Wood et al makes no difference to the Stern report's conclusions, while Roger seems to think it does. Given the subject of the post, isn't it moderately important if Richard is right, even if it doesn't effect the value that he himself puts on the Stern report as a whole?

Tamara said...

This signal-to-noise argument seems to have become a talking point among certain groups. Soon it will be given its own classification among the well-known logical fallacies.
How many ways can we explain that a trend that is indistinguishable from the variation is NOT a trend? Or, another way, has equal chances of being negative as positive. That's it!! I think I am going to start arguing that the indistinguishable trend shows that climate change decreases the number of hurricanes that make landfall. Wonderful, now we can all rest easy. :)

Roger Pielke, Jr. said...

-16-Oliver

No, I don't think that Richard and I disagree at all. We are making different, but compatible points.

He is explaining how the Stern cost estimates were produced from a technical perspective.

In response, I am pointing out the absurdity of of the fact that costs from extreme events can change by an order of magnitude -- which would seem to be important (i.e., up to 40% of the Stern lower bound estimate) -- and yet this apparently has no effect on the Stern bottom line.

Richard's reply was, well add nonsense to nonsense and what do you get?

Which I thought was a nice summary.

Oliver said...

-18- Roger

On this particular matter I took it differently -- that there was not a change of an order of magnitude in the body of the text and the processes that produced it, but rather there was a typo in the table. Am I missing something?

Roger Pielke, Jr. said...

-19-Oliver

Confusing, I know. Let me summarize:

1. The report (mis)used Muir-Wood's analysis of the past to project that future damages from extremes would be several percent of GDP by the end of the century.

2. The use of MW had no basis -- either in what MW said or anything else.

3. The summary table showed future hurricane impacts alone would be 1.3% of GDP (this was the typo)

4. Since (for developed countries) 1.0% ~ 1.3% everything looked like it fit together, multiple lines of evidence and all that.

5. When the typo was corrected, it becomes obvious that there is a big problem in the report as the individual contributions to damages come nowhere near the 1.0% as 0.13% ~ 1.0% doesn't add up

6. Correcting the typo clearly showed an order of magnitude problem in the table (which is why its quiet correction is problematic)

Apparently what the table says does not really matter to the cost estimates from the technical perspective of how they were created, but logically and common-sensically, if you change the expected losses from extreme events by an order of magnitude (that is, if you remove the flawed reliance on MW and depend instead on Nordhaus and others), it probably should mean something . . . Tol disabused me of such logical and common-sense thinking;-)

Richard Tol said...

-20-Roger
Indeed. Table 5.2 is typical of the Stern Review. It is riddled with error. Any attempt to reconstruct what they did, either in detail or as a broad picture, leads to a logical contradiction.

My initial verdict was that the Stern Review would not pass as a Master's thesis. I have discovered additional errors since.

Post a Comment

Note: Only a member of this blog may post a comment.