11 March 2010

Bias in IPCC AR4 WGIII? A Guest Post by Richard Tol, Part V

This post is Part V of Richard Tol's look at the IPCC AR4 WGIII. The first four posts looked at Chapter 11. Part I is here. Part II is here. Part III is here. Part IV is here. Parts I, II and II looked at Chapter 11 in AR4 WGIII. This installment, like Part IV looks at Chapter 3.

Richard Tol is a research professor at ESRI in Ireland, one of the top 175 economists in the world and a contributor to the work of the Intergovernmental Panel on Climate Change (IPCC), where his work is widely cited. In this guest post, the fourth of a series, Richard takes a look at parts of the IPCC AR4 Working Group III, which has largely escaped scrutiny in recent months. In this Part V he concludes that,
In sum, the IPCC made a mistake in SRES. Instead of admitting and correcting the mistake in AR4, the IPCC distorts the literature review to hide the mistake.
Please have a look at Richard's full discussion below. If you have questions or criticisms of Richard's analysis please submit them in the comments, I am sure that Richard will be happy to engage.
Ignoring the Actual Balance of Views on PPP vs. MER

In parts 1, 2 and 3, I looked at Chapter 11 of the Fourth Assessment Report of Working Group III of the Intergovernmental Panel on Climate Change.

Here, as in part 4, I turn my attention to Chapter 3. The first and second order draft of the chapter and the review comments can be found here.

There was controversy over the IPCC SRES scenarios back in 2003. Ian Castles and David Henderson noted that the IPCC had used market exchange rates (MER) in lieu of purchasing power parity rates (PPP). This is an arcane bit of economics, but easily understood by anyone who was travelled: A dollar goes much further in a poor country.

Economists have long known that international comparisons of living standards using MER are just wrong. But SRES is about emissions. Emission data are derived from energy and agriculture statistics, measured in physical units. So, PPP v MER is irrelevant. Or is it?

The SRES scenarios assume convergence: Poorer countries grow faster than richer countries. In the very long run, everyone is equally rich. If measured in MER, the gap between rich and poor is large; and poor countries grow fast. If measured in PPP, the gap is smaller, and economic growth is slower. Economic growth drives emissions growth. Therefore, PPP v MER matters.

The Summary for Policy Makers of WG3 reads as follows:
Available studies indicate that the choice of exchange rate for GDP (MER or PPP) does not appreciably affect the projected emissions, when used consistently.
In other words, PPP v MER is irrelevant. Castles and Henderson are dismissed as “inconsistent”.

Chapter 3 devotes almost three pages (180-184) to this issue. Here are a few citations. On p. 171 (summary):
In the case of the SRES, the emissions trajectories were the same whether economic activities in the four scenario families were measured in MER or PPP.
On p. 181:
Nordhaus (2005) recommends that economic growth scenarios should be constructed by using regional or national accounting figures (including growth rates) for each region, but using PPP exchange rates for aggregating regions and updating over time by use of a superlative price index. In contrast, Timmer (2005) actually prefers the use of MER data in long-term modelling, as such data are more readily available, and many international relations within the model are based on MER.
At first sight, this paragraph is balanced – but Nordhaus is a leading authority in climate economics and national accounting. Both papers are cited as presentations at a workshop – but Nordhaus’ paper was accepted for publication when the Second Order Draft was reviewed. Timmer’s paper was never submitted.

On p. 183:
Manne and Richels (2003) and McKibbin et al. (2004a, 2004b) find some differences in emission levels between using PPP-based and MER-based estimates.
According to Manne and Richels (2003), carbon dioxide emissions in 2100 drop by 14% from 21 to 18 10^9 tonnes of carbon. They write that “virtually the entire emissions decline occurs in the non-Annex I countries”. Figure 3 suggest a drop of 21% from 14 to 11 10^9 tonnes of carbon. Does that classify as “some” or “not appreciably”?

McKibbin et al. go further: “we show that emission projections based on convergence assumptions defined in MER terms, are 40% higher by 2100 than emissions generated using a PPP comparison of income differentials between economies.” For China, the difference is over 80%; for the Less Developed Countries, almost 110%.

PPP v MER is trivialized on p. 183 of Chapter 3:
To summarize: available evidence indicates that the differences between projected emissions using MER exchange rates and PPP exchange rates are small in comparison to the uncertainties represented by the range of scenarios and the likely impacts of other parameters and assumptions made in developing scenarios, for example, technological change.
The reviewers did not agree at all with the drafts of the chapter. There were 11 votes of protest and 2 votes of support in the First Order Draft; and 16 votes of protest against 1 votes of support in the Second Order Draft.

The reviewers also alert the authors to three further, peer-reviewed papers: Dixon and Rimmer, Tol, and Smith et al. The first two papers matter because the models are specified in a different way but reach the same conclusion as Manne and Richels and McKibbin et al. (there is a substantial difference). The paper of Smith et al. matters because it shows that sulfur emissions would be different as well. These papers were not cited in the published chapter. The chapter did, however, use the phrase “evidence from the limited number of new PPP-based studies” in the summary. The “limited number” probably refers to Manne and Richels and McKibbin – that is, two papers. In fact, there are five papers – and zero papers that use a full-blown model to show the opposite.

There is also a Pielke moment. Comment FOD 3-183:
other modelling teams (IMAGE, IIASA) did not recalibrate but argue on first principles, or as Tol (forthcoming, Climatic Change) argues, on a misinterpretation of first principles (Richard Tol, Hamburg University)
comment is wrong: Tol does not reject the “no change” vision in his paper.
That, the IPCC authors know better what Tol writes about than Tol himself!

In sum, the IPCC made a mistake in SRES. Instead of admitting and correcting the mistake in AR4, the IPCC distorts the literature review to hide the mistake.


  1. Dr. Tol,

    Let me see if I understand this correctly. In round one, there were 11 protests from IPCC peer reviewers regarding a particular point. In round 2, there were 16 protests.

    These were all overlooked/overruled and the IPCC authors carried on as though nothing was amiss, writing what they had intended to write from the outset.

    For a final flourish, an IPCC author responded to a peer review comment authored by you by declaring that your paper says something rather at odds with what you say it says. (All the while referring to you in the third person.)

    How edifying.

  2. -1-TripodGirl
    Correct. There are five peer-reviewed, model-based papers that side with Castles and Henderson. There are zero peer-reviewed, model-based papers that side with IPCC SRES. Yet, IPCC AR4 concludes that IPCC SRES was fine. Despite protests by the reviewers.

    The final flourish would be amusing -- except that a similar thing happened to Roger. As a referee, I reminded the authors of a point I made in a paper. The authors replied that I did not make that point at all. I did so.

  3. It was the whole Castles and Henderson debacle that first prompted me to look closer at climate issues. As an economist, here was something I understood. I understood the issues and I observed the response from the IPCC. And it was not pretty. What was more disconcerting than the error, was the response.

    (The other issue that caught my attention was the Hockey Stick. I read the Nature article, I parsed the verbiage and came to the conclusion that the statistical expertise of physical scientists when dealing with non-experimental data led a lot to be desired. Once again, as an econometrician I could understand the issues and work it out. Once again, the response was more disconcerting than the error.)

  4. The IPCC dismissed the Castles and Henderson critique of its use of 'market' exchange rate converters in its notorious press release in December 2003, and has stubbornly resisted all subsequent advice to the contrary.

    It is risible that the authors of WGIII Chapter 3 should have cited in all seriousness the statement of a modeller that he 'actually prefers the use of MER in long-term modeling, as such data are more readily available...' The IPCC milieu have failed, and still fail, to grasp the fundamental point stressed by Professor Bill Nordhaus at the same IPCC Expert Meeting: "Estimates of output at [market exchange rates] are simply wrong - they are constructed on an economically incorrect basis."

  5. -3-Skippy
    Agreed. We all make errors. We should admit and correct them, and make sure we don't repeat the same mistakes again.

    The IPCC response to Castles & Henderson has set the sorry standard for the current going-ons.

  6. @R. Pielke

    about errors in SRES, you should definitely check out the amount of fossil fuel reserves they are using, which is total nonsense.

    For example all scenarios range between 11ZJ an 50ZJ of oil (with 17ZJ to 30ZJ for median scenarios), when ultimate exploitable reserves are estimated to be at most 14ZJ (best estimate : 11ZJ).
    How can you ever get 50ZJ ?!!

    Same for natural gas : between 15ZJ and 55ZJ in SRES, but only 11ZJ of ultimate reserves.

    And for coal : between 3.3ZJ and 68ZJ in SRES (median scenarios between 13ZJ and 47ZJ) when ultimated reserves are 15ZJ.

    You can see those data graphed for oil, natural gas and coal here starting page 70 against all 40 SRES scenarios here :

    All datas can be found here:

    IPCC SRES scenarios, column "hydrocarbon use":

    Data for oil (p7):
    or this page for proven reserves :

    Data for natural gas (last page) :
    or this page for proven reserves:

    Data for coal :

    Now you just have to find the conversion rates to Joules to find the numbers cited above.

    By the way, this information is also confirmed by one of the climategate e-mails:
    "We ARE NOT supposed to be working with the assumption that these scenarios are realistic."


  7. Jean B said,

    There are 1055 Joules in a BTU. A ton of coal has between 17 and 25 million BTU's.

    Oil and Gas reserves are hard to estimate as we probably don't know how much is under the oceans. So on oil and gas I can buy the IPCC estimates.

    On coal reserves while we probably don't know how much is under the ocean, we can fairly easily conclude that the cost of undersea coal mining will exceed the cost of other forms of energy.

    If one looks at the pace of Chinese Nuclear plant construction, and their efforts to accelerate that pace then the Chinese have already concluded that they are going to be unable to continue to grow their economy efficiently on the back of coal.

    There are only 8 countries in the world where coal is a medium term economic option compared to nuclear and China is the biggest one, followed by the US.

  8. "The IPCC milieu have failed, and still fail, to grasp the fundamental point stressed by Professor Bill Nordhaus at the same IPCC Expert Meeting: 'Estimates of output at [market exchange rates] are simply wrong - they are constructed on an economically incorrect basis.'"

    If Arnold Kling and I are right (and I think we are) then ALL of the IPCC scenarios return world GDP estimates in the year 2100 that are low by somewhere around a factor of 100. That is, the world per-capita GDP in the year 2100 will be over $10,000,000:





  9. I've seen somewhere that we are supposed to assume that the individual scenarios in the SRES have equal probability of occurring. What they don't tell you is, that probability is near zero.

  10. @Harrywr2
    Just ask any expert, or check any data from any agency on oil, the numbers used in the SRES are completely unrealistic.

    What i am saying is that the vast majority (more than 30 out of 40) of them are not remotely feasible if we agree that we only have one planet to get the resources from.

    I really think R.Pielke should look into this... and I am really expecting to see how the new SRES scenarios will look like.


  11. @Jean
    You're mixing up reserves and resources, and conventional and unconventional fuels.

    IPCC SRES is based on the previous assessment of the USGS. The latest USGS has still more fossil fuels, and that was before the shale gas.

    If anything, SRES is too low.

  12. Hi,

    Regarding 6, 7, 9, 10, and 11.

    This is exactly why the IPCC should change its assessment procedures to something scientific. "Scenarios" with no estimates of probability of occurrence*** are simply not scientific.

    The IPCC (if it was interested in science) would switch to probabilistic predictions. That is, there would be a *most* likely set of emissions and resulting atmospheric concentrations for *all* climate forcing agents (e.g., CO2, CH4, black carbon, organic carbon, SO2). And there would be a MOST likely climate sensitivity (temperature change per doubling of CO2 equivalent atmospheric concentration).

    THIS is what Roger should be examining. He even edited a book on such things! :-)

    P.S. In fact, ALL IPCC scenarios have zero probability of occurrence, in the strictest sense, because they are all wrong AT PRESENT with regards to methane atmospheric concentration (they are all too high, because none of them assumed that methane would stop rising like it did). This is one more reason why the MOST probable value should be established for each forcing agent separately. As it is, with each scenario completely different, CO2 could be too high, but SO2 also too high, in Scenario 1, but in Scenario 2, CO2 could be too high, but SO2 too low. It's a complete mess, because it's completely unfalsifiable.

  13. @Richard
    It's pointless to base yourself on USGS resources... they do estimates from the size of underground cavities (so they always greatly overestimate the actual content), and they have a very very bad historical record on actual oil estimates :

    For example, for US-48states (so not the most difficult to estimate)
    *USGS predicted in the 1960's 600Gbilion barels and no forseeable decrease in production
    *Hubbert in 1956 had predicted 200Gbilion barels and a peak around 1970.

    I let you find out the outcome...

    Oil production is in decline in 33 of the 48 largest oil-producing countries, and at least half of the 20 largest oil field are already in decline. Those are just facts.

    If you check out the sources above (ifp for example) you'll see that unconventional sources are already taken into account in my numbers.

  14. "If you check out the sources above (ifp for example) you'll see that unconventional sources are already taken into account in my numbers."

    I don't understand these natural gas numbers at all.

    From ifp Figure 1:


    Looks like ~6 "milliards des metres cubes" (presumably 6 billion cubic meters, i.e., 6,000,000,000 cubic meters?)

    From wikipedia:


    5.977 trillion cubic meters.

    The ifp seems insane...it seems like "milliard" must be 1 x 10^12, not 1 x 10^9. But even the Wikipedia value of 5.977 x 10^12 cubic meters seems too low. As Richard Tol mentioned, I don't see how this could include shale gas (or anything but conventional gas).

  15. Mark,

    6E12 m3 is too low by more than an order of magnitude. EIA (http://www.eia.doe.gov/oiaf/ieo/nat_gas.html) estimates current global reserves at over 6E15 cubic feet. At 35 cubic feet/cubic meter, that's 177E12 m3 or 6.55 ZJ at 37MJ/m3. That puts Jean B definitely in the ballpark.

  16. 3 Skippy
    I also am an economist and followed almost exactly the same path. The Castle and Henderson critique was something I understood, and the IPCC's reaction was shocking. And then the hockey stick,which used techniques which I ought to understand but couldn't until Steve MacIntyre worked out what was going on. But what really shook me, as you, was the response. No proper debate, just abuse. Incidentally Montford's "The Hockey Stick Illusion" gives a superb review of the hockey stick story.

  17. @Mark, @DeWitt
    if you look ifp p7 you see that everything is expressed in Tm^3 (10^12 m^3).

    The chapter "Conclusion sur le gaz naturel" you can read that the max amount is 261 Tm^3.

    With 38MJ/m^3
    300Tm^3 = 11ZJ < SRES median scenarios

    "that's 177E12 m3 "
    In ifp p7, it's estimated 175, so a very close number to yours.

    "or 6.55 ZJ"
    You find 6.55 ZJ, which is 2 times less than the lowest median scenario.
    6.55ZJ means the vast majority of scenarios are out the reality domain.

    I guess in fig1 it's just a typo, in the rest of the paper it's always 10^12 m^3.

  18. @DeWitt
    "That puts Jean B definitely in the ballpark."

    Sorry i had misundertood this idiom, i thought it meant i was wrong but i just found out it meant "close to what is expected.".

    So you do agree with me, scenarios with 55ZJ of NG are just nonsense !

    With 300Tm^3 (so 40Tm^3 more than the current max reserve estimate), you get 11ZJ, which is way less than most of the SRES scenarios.
    So when you OVERestimate reserves you reach the LOWEST scenarios.

    Same thing for oil.
    For coal, the high estimates for reserves are in line with the lower scenarios... but all the carbon-intensive scenarios (more than 30 out of 40) are physically unreachable.

  19. DeWitt,

    Oops, I forgot to explain that the numbers I was getting from ifp and Wikipedia were for the U.S., not the global numbers.

    So Jean B's Wikipedia number for the U.S. has 5.977 trillion cubic meters, or 210 trillion cubic feet. That may be appropriate for *conventional* natural gas, but as Richard Tol mentioned, it's not appropriate when unconventional natural gas is included.

    And I don't know *what's* going on with the ifp number for the U.S.

    P.S. You're EIA site (Figure 41) has 283 trillion cubic feet for North America. Again, that may be appropriate for conventional natural gas, but it's not appropriate when unconventional natural gas is included.

  20. @Mark

    What is your number for global known reserves ?
    Mine is 175e12 m^3 = 6.6ZJ

    Global max reserves (ultimate recoverable : conventional&unconventional, discovered&yet to be discovered) ?
    Mine is 300e12 m^3 = 11.4 ZJ

    Which is far below IPCC SRES median scenarios.

    PS: you probably hadn't seen my message#17 when you posted yours.

  21. JeanB (18):

    "@Mark, @DeWitt
    if you look ifp p7 you see that everything is expressed in Tm^3 (10^12 m^3).

    The chapter "Conclusion sur le gaz naturel" you can read that the max amount is 261 Tm^3."

    I presume that 261 Tm^3 is world-wide. The value for the U.S. appears to be only 6 Tm^3. This matches the Wikipedia value of 5.977 Tm^3.

    The problem is that those numbers translate to 210 trillion cubic feet. That simply does NOT include unconventional natural gas in the United States.

    For example, see this blog which reports a Colorado School of Mines study that comes up with 1836 trillion cubic feet...almost a factor of 10 higher than your Wikipedia or ifp references. (And a factor of 6.5 higher than DeWitt's EIA value of 283 trillion cubic feet for the U.S.).

    Once again, your Wikipedia and ifp references must NOT include unconventional natural gas for the U.S. Or if they do, they're way, way wrong.

    P.S. Regardless of who is right and who is wrong about this particular issue (and if it's natural gas in the U.S., it's currently my job to be right about that! :-)), the more important thing is that the IPCC should come up with a "most likely" future, rather than a bunch of widely ranging scenarios, with no assessment of which scenario is most likely. I'm sure you'll agree that there are at least SOME scenarios that have oil, natural gas, and coal usage that are reasonable.

  22. Just to clarify in my simple mind.

    For any given fossil resource we have 3 piles.

    Exists, Technologically Recoverable, Economically Recoverable.

    For oil and gas 'reserves' have always been described as as the later.

    Hence, in the official 'reserve' numbers we never see more then 10 or 15 years worth of 'reserves'. There might be more out there, but no since looking for it because it will cost to much to extract given current prices.

    Coal has traditionally been reported as how much do we have. Hence we get numbers like the US has 200 years worth of coal 'reserves'. When we apply the term economically recoverable the US has more like 30-40 years worth of coal left at current consumption.

    Coal also has a 'place' limitation. It is expensive to move.

    Roughly 50% of global CO2 emissions come from coal. The Chinese currently pay $107 a ton for steam coal mined in Columbia delivered to a Northern Chinese sea port.

    The price of coal at the mine mouth in Gillette,Wyoming (coal capital of the world) is $10/ton. It's 1,400 miles from Gillette, Wyoming to the nearest seaport.

    The coal in Kazakhstan and the coal in Siberia suffer from the same problem as the coal in Gillette, Wyoming. Even if you give it away, the cost of moving it to those who might want to consume it makes uneconomical compared to other energy alternatives.

    Plugging in 'average' price of coal and the global resource into a Global Economic model doesn't really tell the story accurately.

    In order to get an accurate picture of future coal consumption one has to get a bit more complicated and plug in economically recoverable coal, the distance the coal is from the potential 'growth markets' and the price of transporting that coal to those markets and then compare it against the costs of alternative sources of energy in those markets.

  23. Mark,

    The problem is that they're all over the map. One will have way too much of one and less than what we're actually using right now of one of the other. For oil, if you look at cumulative use to 2100, they're all too high. IIRC, the lowest has oil consumption of over 3 trillion bbl with significant consumption levels continuing in 2100. Deffeyes, who is admittedly at the low end of the Peak Oil prognosticators, has total recoverable oil at 2 trillion bbl. The huge new field off the coast of Brazil is estimated at 40 billion bbl. That's less than 1.5 years consumption at current rates of ~85 million bbl/day. New field discoveries peaked a while ago. A production peak must follow. Converting gas or coal to liquid fuels just means we'll use those resources even faster as the conversion process isn't very efficient.

  24. DeWitt,

    "The problem is that they're all over the map. One will have way too much of one and less than what we're actually using right now of one of the other."

    Yes, exactly. That's what I think Roger should focus on (not that some or even most of them are too high in one fuel or another).

    The most logical, scientific and policy-relevant way to go about predicting future warming is to come up with a "most likely" value for every year in the century, and then try to figure out what the uncertainty is around that most-likely value.

    What the IPCC has done is not logical, scientific, policy-relevant, or even honest. What they've done allows them--decades from now--to choose whatever scenario came closest to reality and say, "We predicted that." But they didn't, because they ALSO had a bunch of other scenarios that weren't even close to reality...and they didn't predict in advance which scenario *would* be closest to reality.

  25. Richard Tol,

    I'd be really interested in how they come up with the scenarios. It looks to me like they first estimate the population growth curve, then the gdp growth based at least partly on the population curve. Then calculate the energy needed for that gdp level and finally apportion the energy needed to different sources by some arbitrary rule. At no time does the reality of the future availability and cost of those energy sources seem to come into play.

  26. -25-DeWitt
    Roughly correct.

    The models differ in their detailts, but in SRES, population and economic activity are independent of the price of energy (and one another). Energy demand is driven by the size and structure of the economy and the price of energy. Energy supply is driven by something akin to cost minimisation. Energy supply and demand jointly determine the price.

  27. 26-Richard,

    Economic activity is independent of the price of energy? That concept seems unintuitive. How much would the price of a Big Mac go up in constant dollars if the cost of energy increased by an order of magnitude? That would look like inflation according to the way the CPI is calculated. But if governments inflate their currency to keep up, the price of energy in nominal currency will go up even faster. The more I think about it, the uglier it looks.

  28. -27-DeWitt
    It's not the best of assumptions, but there are worse things in SRES.