04 April 2012

Be Careful What You Wish For

When US President Barack Obama announced a government-wide effort to protect federal science from political interference, the US Department of the Interior (DOI) took an early lead. In 2011, it became the first agency to finalize a new policy on scientific integrity and it has hired ten scientific-integrity officers to work with staff in its various bureaus. But the DOI may also be the first to run into a problem with the way the policies are implemented, as one of those officers claims to have been fired for upholding the guidelines.

“I thought I was doing the job I was hired to do and was doing the right thing. I was stifled,” says Paul Houser, a hydrologist at George Mason University in Fairfax, Virginia, who was appointed as scientific-integrity officer for the DOI’s Bureau of Reclamation in April 2011. Houser was fired on 10 February and filed a complaint under the DOI’s scientific-integrity policy two weeks later.
What is the issue in this case?
Houser says that he was asked by a press officer to check some material that the DOI planned to make public about the probable environmental impact of the dams’ removal. But the material painted an overly rosy picture of the benefit, Houser says. For example, in a summary document, the DOI said that studies had shown that the annual production of Chinook salmon (Oncorhynchus tshawytscha) would rise by 83% a year after the dams were removed. However, it did not include any of the uncertainties about how the population would respond that an expert panel commissioned by the DOI had listed. In the final version of the summary — which is now on a government website — the number was changed to 81.4%. “That number expresses an accuracy that’s ludicrous,” says Houser. The figure comes from an unpublished computer-modelling study and had an uncertainty range of −59.9% to 881.4%, which was not reported in the summary.

Houser says that last September, his supervisor, deputy commissioner for external and intergovernmental affairs Kira Finkler, chided him for documenting his concerns. He says Finkler told him that “the secretary wants to remove those dams”. Finkler did not respond to questions from Nature about the situation, but the scientific-integrity officer who is over­seeing implementation of the department’s policy, Ralph Morgenweck, confirms that Houser’s complaint is being investigated.

Kelly says that the DOI is looking forward to the outcome of the investigation. “We believe all actions will be proven to be fully justified,” she says, adding that the studies the agency is using about the impact of the dam removal are available on the Internet for anyone to see and review. However, members of the expert panel contacted by Nature have said that they, too, felt that the materials flagged by Houser played down the uncertainties in scientific predictions.
While the various allegations are under investigation, Houser's complaint (which you can read in full here in PDF) signals that the implementation of scientific integrity policies is going to be a rocky road for federal agencies.  In effect, the policy empowers all agency employees (and in some cases, like this one, a watchdog) to challenge all agency communications and decisions based on how information is presented.  The Department of the Interior was the first out of the gate with its guidelines (presented at the White House here in PDF), so it is not unexpected that a controversy shows up under that agency.

Because all communication involves discretion in the section of material to include and not to include, the notion of an agency watchdog over the use of science in agency communications is likely to lead to many more such disputes.

For instance, one of Houser's complaints (detailed in the excerpt above) is that a DOI document did not include the full range of uncertainties that could be found in the underlying study. As a summary document there was probably a lot of information from the original study that was not included.

Who decides what information should and should not be included?  Who gets to second guess agency policy makers and the press office on what information should be included or not included? I testified on this issue in 2007 before the House Government Reform Committee during the period when the Bush Administration received similar criticisms (here in PDF). At the time I wrote:
[N]o information management policy can ever hope to eliminate political considerations in the preparation of government reports with scientific content
 The issues remain much the same today under a different administration. Even though the political context has changed, the underlying dynamics have not.

The Houser case will likely prompt some additional thinking about these issues and what it means to try to regulate or otherwise manage the scientific content of agency information. I suspect that eventually agencies will have to accept the reality that in many if not most cases the proper place for debate over agency decisions and communications is simply in the broader political arena as part of ongoing policy debate.

As it stands, the DOI scientific integrity policy may foreshadow ever more disputes over science between career government employees and political appointees, and perhaps even a further politicization of agency science. This is probably not the outcome expected or desired by the Obama Administration when putting forward a call for agency integrity guidelines.

7 comments:

  1. Poor guy, he was probably trained as a scientist when the DOI was looking for an expert witness to support the desired conclusion.

    ReplyDelete
  2. Dropping the huge uncertainties associated with a lot of scientific work, as well as other qualifications, is routine for members of this administration. Lisa Jackson (head of EPA) has been a particularly egregious offender. For example, in front of the Senate Environment and Publics Works committee, she said the recently imposed new standards for power plants would prevent 120,000 cases of childhood asthma.

    Trouble is, the EPA's own Regulatory Impact Analysis estimated that there would be a reduction of between 4,100 and 390,000 exacerbations of existing asthma cases from the new rule. The 120,000 was the most probable estimate, but with those uncertainly limits, the most probable number is a very weak number indeed. And exacerbating an existing asthma case is quite different from causing asthma.

    The problem is, a lot of the science on which regulatory actions is based really has uncertainty limits much larger than the number measured. That's not saying it's not good science; if it honestly states uncertainties, it may well be. One of my papers came up with a number of 20 ±/- 60 for a particular quantity. People without technical training often don't understand why the uncertainly can often be larger than the value itself, and that this doesn't mean it's a useless measurement. If it's 20 +/- 60, it's almost certainly not 200 or -200.

    ReplyDelete
  3. They can dismiss an Inspector General at will. Surely they can also dismiss an "integrity officer" with the same level of justification and scrutiny.

    This is a perfect example of why the people providing oversight have failed in their principal occupation. Their personal interests are certainly not limited to the financial sector. They are subject to oversight until they determine it is inconvenient.

    What a corrupt morass our civil servants have invited. It's not just the motivations of scientists who engender Americans' skepticism.

    ReplyDelete
  4. Poor guy. He didn't realize that he wasn't hired to make sure there was integrity in Government "science" based PR. He as hired to provide a rubber stamp and a fig leaf when those statements end up in court.

    "You see your Honor all of these statements are based on sound science and backed up by our own integrity officer..."

    ReplyDelete
  5. More on scientific integrity in general here: http://www.straightstatistics.org/article/lot-science-just-plain-wrong

    It includes this link to highly relevant comments about the EPA: http://science.house.gov/sites/republicans.science.house.gov/files/documents/hearings/HHRG-112-SY20-WState-SYoung-20120203.pdf

    Extract 'He called for the EPA to make the data underlying key regulations, such as those on air pollution and mortality, available. Without it, he said, those papers are “trust me” science. '

    ReplyDelete
  6. .

    “the secretary wants to remove those dams”


    The guy simply did not understand the Chicago way.


    .

    ReplyDelete
  7. Roger, I don't think that government science could possibly get more politicized anywhere in government, especially at EPA, where you will find no whistleblowers at any time with regard to overstatement of environmental issues, not any more.

    ReplyDelete