04 January 2012

How Not to do a Technology Assessment

In the future Dom Perignon will cost less than drinking Budweiser! Yes, you read that right. I have performed a sophisticated assessment and determined that in the future, drinking champagne need not cost any more than drinking a mass produced beer.

What methodology did I use to arrive at this fantatstic result?

Well, in my analysis I assumed that you will be drinking 1 milliliter (less than a small thimbleful) of the bubbly, as compared to a pint of beer. Thus, by varying the amounts delivered I arrived at the conclusion that the costs of the Dom Perignon will be just a bit smaller than the Budweiser. Drinkers rejoice!

How might you reply? Apples and oranges? Sleight of hand?

Both allegations would be true. If the point is to compare costs of two quantities, it would be methodologically suspect to rearrange variables seemingly arbitrarily in order to equalize costs. It would be far more transparent to present cost per unit, where the units are equal. In this case, the champagne costs about 1,000% more than the beer. Seems like an obvious enough point, no?

However, in an assessment of the costs of various energy technologies released last week, the British government committed the Dom Perignon error. The assessment looked at 4 scenarios, business-as-usual plus 3 alternatives. But the total energy consumed is different across the scenarios. 

The FT caught this methodological problem right away:
[W]hy [is] the “higher renewables” scenario is combined with “more energy efficiency” when the other options are not to the same degree?

(DECC presumes energy saving of 54 per cent in the renewable scenario; 43 per cent under the fossil fuels/biomass scenario and only 31 per cent in the nuclear scenario).

Why is there a presumption that if we move towards wind/solar power then people will use more insulation? It does not necessarily follow. Presumably if you combined the extra energy efficiency to nuclear or fossil fuels they may no longer come out as the most expensive options.
The Guardian, however, did not catch the problem and thus misreported the conclusions:
Every person in Britain will need to pay about £5,000 a year between now and 2050 on rebuilding and using the nation's entire energy system, according to government figures. But the cost of developing clean and sustainable electricity, heating and transport will be very similar to replacing today's ageing and polluting power stations, the analysis finds. . .

The predictions challenge suggestions that the costs of embracing low-carbon energy and meeting the UK's legally binding commitments to tackle global warming will be higher than the bill would be for using traditional energy sources.
The actual apples-to-apples math is pretty straightforward (under the assumptions of the analysis) -- if the costs of delivering about half the supply renewable energy are about the same as the costs of delivering non-renewable energy, then the renewable energy costs twice as much per unit of energy delivered.

No one is well served by claiming that Dom Perignon costs the same as Budweiser, because it doesn't.


  1. This DECC's spreadsheet by Prof. David Mackay has been getting a lot of attention in the UK over the last few days. To my jaundiced eye, the slight of hand simply means that the conclusion was written prior to the economic analysis and then the analysis was simply adjusted to support the desired outcome....green energy is more economical. It seems a lot of (political) science in energy and climate is presented this way.

  2. .

    Funding the crony capitalists requires some level of mendacity. The Guardian was fooled, and so no doubt were others.

    p.s. The new format looks very nice.


  3. I agree that this is hugely misleading by the UK Government; the fact that you can change the assumptions in the tool yourself is hardly the point.

    The other thing that the Guardian gets wrong is in the reference to the Stern's review £6500 per person for the costs of climate change, which it implies should be set against the do-nothing option but not the others.

    Even if you believe the Stern review is correct (ha ha), given that the UK accounts for just over 1.5% of global emmissions, what the UK does about energy policy will do almost nothing to change the costs of global warming. Let's assume that the most carbon-efficient scenario reduces UK emissions by the targeted 80% relative to do-nothing (again, keep a straight face), that £6500 figure would be reduced by maybe £500.

  4. sorry, Roger, I added a zero. That £500 at the end should read £50.

  5. To be eligible for the full rate of Feed in Tariffs homeowners may have to improve the energy efficiency of their homes under the UK governments proposed changes to the FiT scheme.

    This might be where some of the difference in the scenarios is presumed to come from. I don't know how the numbers would pan out, but there at least is a possible partial justification for the different scenarios. I suspect a closer reading of the DECC scenarios may bear that out (assuming they actually list their assumptions...).

  6. It's kind of like extrapolating advances in renewable technology into the future while holding conventionals constant (even when advances - like turbine efficiency - are applicable to both).

  7. No, the Guardian was not 'fooled.' They can do their sums as well as the FT. They are just happy to do their job passing on the propaganda - its what they do.

  8. The "green" technology has capital costs, operational costs, and, of course, environmental costs. It is the last which few people are will to discuss in polite company. Unless places like China will continue to absorb the environmental and human costs that follow from resource recovery and development, the cost of "green" technology will be even greater in the future.

    Then there is the matter of several hundred thousand birds that are slaughtered annually in America alone as they fail to run the windmill gauntlet. Another environmental cost which is not discussed in polite company.

    Of course, as Christopher noticed, the requisite resources and technology are versatile. Still, it serves no useful purpose to obfuscate the relevant issues associated with each path. If anything, it undermines a critical review of value, and defers a measure of uncertainty to the future.

    They know they are misrepresenting their position. They know that the result will be an uncertain future and its attendant consequences. This is purely deceptive and it is for profit and designed to wrest control of the private and public sectors.

    This is utterly ridiculous. It does not serve the best interests of Americans or humanity when either scientific or technological research is prematurely constrained to fit a special interest's agenda.

    There is a place for "green" energy technology as a supplement to primary sources, including: nuclear, coal, gas, etc., because both its value and viability are situational and circumstantial. There is nothing to be gained in pigeonholing a technology based on perceived value and issues.

    The journalists are doing a disservice to their customers when they do not report objectively or, at minimum, critically. They confirm the value in the old adage: optimal truth from multiple, independent sources. It's hardly surprising that individual reporting is biased by personal perception.

  9. [W]hy [is] the “higher renewables” scenario is combined with “more energy efficiency” when the other options are not to the same degree?

    Because the smart meter that they will have to install to manage the intermittants will be turning off the heat when it's cold and turning off the air conditioning when it's hot.

    Everyone will still be allowed 8000 hours of heating or cooling, the seasons when you are allowed to use them will just be reversed.

  10. Is it Leslie Ann Warren in Victor, Victoria?