30 July 2012

Evidence-Based Policy: Which Side are You On?

We'd all like to think that policy makers consider evidence from experts in how they make decisions. Of course they do, but in case after case, such consideration takes a form far from that which might be considered ideal by most experts. Typically, a decision is made based on considerations that have little or nothing to do with evidence, and only then is evidence sought out to support that decision. Science in all of its glorious bountifulness is almost always compliant.

Consider these two cases from opposite sides of the US political spectrum, the first having to do with abortion and the second climate change (surprise, surprise in both cases). These cases illustrate important dynamics associated with what I call "science arbitration" in The Honest Broker, a process focused on proffering advice on claims that can be resolved empirically, using the tools of science. Most people just call this "science advice." Science arbitration is distinct from Issue Advocacy and Honest Brokering of Policy Alternatives, both of which are focused on policy action, and are typically called "policy advice."

The first case comes from South Dakota where a US Appeals Court has ruled that doctors must inform women seeking an abortion that the procedure leads to an increased risk of suicide.  The Minnesote Star-Tribune reports:
South Dakota can require doctors to warn women seeking abortions that they face an increased risk of suicide if they go through with the procedure, a federal appeals court ruled Tuesday.

The 8th Circuit Court of Appeals upheld the portion of the 2005 South Dakota law dealing with the suicide advisory 7-4.

"On its face, the suicide advisory presents neither an undue burden on abortion rights nor a violation of physicians' free speech rights," the court wrote in its majority opinion.

In September, a three-judge panel upheld U.S. District Judge Karen Schreier's decision to overturn the requirement following a lawsuit brought by Planned Parenthood. The decision Tuesday by the full 11-member court grants judgment to the state and vacates the permanent injunction against enforcing the provision.
What is wrong with such a requirement you might ask?
Well, for one thing, the evidence does not support a relationship between abortion and suicide. Like most such debates, this one hinges on competing scientific studies, interpretations of evidence and scientists with deeply vested political interests.

In the UK in 2011, the government funded an expert assessment by the Academy of Medical Colleges to survey the state of the science of abortions and mental health, which concluded:
The rates of mental health problems for women with an unwanted pregnancy were the same whether they had an abortion or gave birth.
Such "science arbitration" is difficult to do well, as I've often noted, but doing it well is essential to using expertise effectively in decision making. I find the process used by the UK to be trustworthy and its results compelling. (The fact that the most cited research showing a relationship is being considered for retraction should be noted as well.)

Regardless where one comes out on the state of the science, the strongest position that the South Dakota Court could justifiably take is that the evidence is equivocal. By mandating what doctors must tell patients the court is requiring them to lie. Not good.

Now to the other side of the US political spectrum. Here environmental activists are petitioning the US Secretary of Agriculture to lie to farmers and the public about the state of the science of human-caused climate change and the current US drought:
Environmental activists want top federal officials to directly address the possible connections between climate change and the current drought that’s crushing the life out of U.S. heartland, with potential implications for global food supplies.

Specifically, Forecast the Facts and FoodDemocracyNow! want Secretary of Agriculture Tom Vilsack to directly address the massive implications of manmade climate change for our entire farming sector. Scientists are clear that climate change is already leading to more extreme weather, such as longer and more severe droughts, according to Daniel Souweine, campaign director for Forecast the Facts.

Souweine said that, in multiple press appearances last week, Secretary Vilsack dodged questions about what drought-stricken farmers need to know about climate change, saying that he’s “not a scientist,” and that the department is focused on the “near term.”

The groups say they’re trying to pin Vilsack down with a petition that reads, in part:

“Please tell farmers and the American public about the connections between climate change and the current drought, as well as the massive implications that climate change has for the future of American farming.”
Just like in the case of abortion and mental health the link between human-caused climate change and the current drought is tenuous (to be polite). The IPCC, an international expert assessment body, recently released a report on extreme events that is squarely in my area of expertise, and which I have noted overall did an excellent job.

On drought the IPCC concluded:
... in some regions droughts have become less frequent, less intense, or shorter, for example, central North America ...
If this conclusion is solid, then these activists are asking the US government to lie to farmers and the public. Not good. Just like the abortion case described above, the strongest position that the environmental activists can legitimately take is that the science is equivocal and likely to remain that way for a very long time.
So where does this leave us if we'd like to secure effective evidence-based policy?

In a nutshell, for those who care about the integrity of science in decision making it is OK to divide the world into two camps -- Finally, a Manichean debate I can support!

However, doing such a dividing according to macro-political perspectives (left-right) are contributing to the problem, as are those who, at the micro-perspective are do the dividing according to specific policy preferences (e.g., restrict abortion, secure action on climate change).

If political perspectives, whether ideological or policy-specific, could dictate which science was most sound, we wouldn't need science, politics could substitute.

The best way to divide the world into two camps are to segregate those who seek science to confirm political prejudices and those who support effective and trustworthy science arbitration, wherever it may lead. The scientific community has at times lost perspective in all of this complexity and found itself in the former category rather than the latter, siding with political advocates whose interests in the integrity of science come second to whatever issue of the day they are championing.

Securing evidence-based policy with integrity requires a ruthless adherence to the ideals of effective science arbitration, even when uncomfortable and inconvenient. Perhaps most uncomfortable of all is the realization that those who would champion the political causes supported by most scientists themselves would not champion the integrity of science itself when its results do not conform to their prejudices. The championing of scientific integrity is a cause unto itself.


  1. Roger,

    What I don't understand about the climate example, is what good would it do if Vilsack starts blaming climate change? Is it likely that USDA will then go it alone from the rest of the Executive Branch in an even more anti-climate change policy than it currently has (which is aligned with the rest of the Executive Branch)? I just don't get it.

    PS At the House Natural Resources Committee hearing when some tried to shift the topic from "what are we gonna do about problems with wildfires" to "it's all about climate change" (including inviting the vituperative Joe Romm!),
    it sounded as if they weren't listening to local concerns and were willing to overlook people's real problems of today in pursuit of some national or international agenda.

  2. -1-stfriedman

    Thanks, I do believe that you have answered your own question;-)

  3. As soon as science "informs" government policy directly (i.e. to government bureaucrats and politicians), it is no longer science, but propaganda. Politics is the realm of preference, not empirical study.

    Science should provide information, that information should be absorbed by voters, the new information will perhaps change the voters preferences, they will vote for representatives that then takes the information provided by science into consideration. It's only through this indirect route that science should have any effect on policy at all.

  4. "The best way to divide the world into two camps are to segregate those who seek science to confirm political prejudices and those who support effective and trustworthy science arbitration, wherever it may lead."

    Yes, and everybody is already convinced that they and their cause falls into Camp Two, its just that anything that tends lead away from confirming their political prejudices is 'bad science'. [and I'm not completely excluding myself from this tendency;]

    Now what do we do?

  5. The government ignored the evidence when policy was made re: DDT. They knowingly lied about AIDS and transmission risks in heterosexuals. And they knowingly lied about second smoke risks. Why shouldn't left-wing political activists expect the same for drought and global warming?

    Especially since the current administration which has just flat-out lied about fast and furious, obamacare and the black panther case for political reasons. I think their expectations are pretty reasonable under the circumstances.

  6. People are confused.

    They often think in binary ways when the universe doesn't correspond to it. Adam and Eve partook of the tree of the knowledge of good and evil and were ejected from paradise.

    Christians are confused.

    The founder of Christian doctrine, St. Paul was a Manichean, a Gnostic, a whisperer of secret teaching, a borrower from the Greek mysteries and a believer in absolute predestination. In other words he was an outrageous heretic who would have been burned at the stake in most periods of history.

    James Hansen is confused. He thinks he is a liberal, but endorsed a an extreme right wing eco fascist book by Keith Farnish. He also believes carbon trading is a massive scam cooked up by the oil companies and banks yet supports global climate deals based on it. He believes he is a rational scientist who forms his conclusions on evidence, yet uses outrageous emotional language to promote his biases.

    Climate 'scientists' are confused. Many of them have inbuilt environmental biases that lead them to choose that career. They also have political biases that they simply do not understand because they don't have the time to study politics. How many of them wondered why they were being given billions of dollars in research funds by the evil, planet burning regime of GW Bush ?

    The media is not confused. It has consistently promoted the (potential) multi trillion dollar carbon trading industry. It is the toxic combination of this and the bias of scientists that has lead us where we are today.

  7. Why do we need the IPCC when we have Bill McKibben? ;-)

  8. "If this conclusion is solid, then these activists are asking the US government to lie to farmers and the public."

    I heard an NPR interview of someone in government...I think it was the Department of Agriculture, and may have been Tom Vilsack...and the interviewer tried to press him to say the drought was caused by climate change.

    The person being interviewed very correctly and properly said words to the effect of: "Climate change is outside of my area of expertise. I'm just trying to help out farmers during this drought."

    It's completely silly for Tom Vilsack to tell farmers that this drought or any one in the future is caused by greenhouse gas emissions. There's nothing farmers can do with that information to improve their situation.

    So even if they were asking Tom Vilsack to tell the truth to farmers, the truth would do about as much to help their situation as telling the truth about what the weather was like in Moscow, Russia, in the summer of 1856.

  9. #5 Eric144

    Cristians and Scientists are confused..? christians want to make science bow to religion and climate scientists want belief from the masses
    not surprisng then ? I would keep them seperate

    Governments want advisors to do the bidding (politically correct) rather than explain science to them, science on the other hand is not a human construct but a universal language unfortunately presented in many political guises

    The sad truth is science has become all too political and governments have long since needed departments to present apraisals of scientific evidence and politicians who are trained to understand same.

    The media feeds on the failures like a parasite undermining both

  10. Roger

    Do you have any data to compare the 1930's drought(s)? I think they had a drought in the mid west for 3 consecutive years 1934, 35 & 36. I also seem to recall that 1935 was the hottest year on record in the US.

  11. I think the greatest problem is in the ability for the Science Arbitrator to say "I don't know". For both issues you raise, you can't demonstrate a 100% or even a 50% (coin-flip) correlation between the outcome and the potential cause.

    It is a more serious problem that politics is being peddled as "science", but it is perhaps worse where Science is being tortured until it pronounces guilty or innocent, yes or no, or some other binary decision with metaphysical certainty on some issue.

    (I would point out suicide is not the same as "having mental health problems", but the outcome of having or not having an abortion can be subtle - tracing women over years, insuring that other factors weren't involved, etc. so the correct answer is probably "we don't know", but that won't satisfy either side, no more than whether the last 2 months of weather was or was not caused by AGW, a solar maximum, or merely a statistical outlier - we demand you choose a side!).

  12. "Securing evidence-based policy with integrity requires a ruthless adherence to the ideals of effective science arbitration, even when uncomfortable and inconvenient." A great example of evidence-based policy in my field of medicine is the US Preventive Taskforce. They have taken a lot of heat and political abuse when they stated that there is insufficent scientific evidence to support yearly mammograms starting at age 40 and that routine PSA blood test screening for men are not recommended.