04 June 2012

Should Scientific Communications be Privileged?

Writing in the Boston Globe, two scientists at the Woods Hole Oceanographic Institute have expressed concern that they have been required by a court to release email correspondence related to studies of oil flow rates in the aftermath of the Deepwater Horizon oil platform disaster. 

They write:
Because there are insufficient laws and legal precedent to shield independent scientific researchers, BP was able to use the federal courts to gain access to our private information. Although the presiding judge magistrate recognized the need to protect confidential e-mails to avoid deterring future research, she granted BP’s request. . .

BP claimed that it needed to better understand our findings because billions of dollars in fines are potentially at stake. So we produced more than 50,000 pages of documents, raw data, reports, and algorithms used in our research — everything BP would need to analyze and confirm our findings. But BP still demanded access to our private communications. Our concern is not simply invasion of privacy, but the erosion of the scientific deliberative process.
In the lawsuit, the government is suing BP for damages associated with the oil spill and clean-up. The work done by WHOI, under government contracts (e.g., here and here) is presumably a key exhibit in the lawsuit, as the amount of oil spilled and its ultimate fate will be important to determining damages.

Given the many billions of dollars at stake, it should not seem surprising that BP has requested background information on the production of the oil flow estimates by the scientists, including their emails.

for their part, the scientists are worried that the emails may be used against them to impeach their work:
In reviewing our private documents, BP will probably find e-mail correspondence showing that during the course of our analysis, we hit dead-ends; that we remained skeptical and pushed one another to analyze data from various perspectives; that we discovered weaknesses in our methods (if only to find ways to make them stronger); or that we modified our course, especially when we received new information that provided additional insight and caused us to re-examine hypotheses and methods.

In these candid discussions among researchers, constructive criticism and devil’s advocacy are welcomed. Such interchange does not cast doubt on the strengths of our conclusions; rather, it constitutes the typically unvarnished, yet rigorous, deliberative process by which scientists test and refine their conclusions to reduce uncertainty and increase accuracy. To ensure the research’s quality, scientific peers conduct an independent and comprehensive review of the work before it is published.
If their science is sound and has been accurately reported in the peer reviewed literature, then these concerns are misplaced. The messiness of science is a reality and need not undermine solid scientific work. If the peer reviewed work is defensible -- and I presume that it is -- then this case may have to be made, but that would not distinguish this lawsuit from a bazillion others where experts are called to provide evidence.

A second concern raised by these scientists is that their "intellectual property" is at risk. This is a strange concern, as there are multiple avenues for protection of intellectual property that these scientists could take advantage of, protecting themselves from it being stolen by BP. If I were WHOI I'd be more concerned about high value and unprotected IP being shared around via email, regardless of the BP disclosure.

The WHOI scientists conclude:
Our experience highlights that virtually all of scientists’ deliberative communications, including e-mails and attached documents, can be subject to legal proceedings without limitation.
This is correct -- When scientists operate with public support and the resulting work is used in the making of important decisions, whether in a lawsuit or in policy, we should expect that deliberations will be subject to public disclosure. Asking for a special privilege in scientific communication is highly unlikely ever to be a successful strategy under US law. Besides, good science, even when messy, does not need to be hidden from view.

19 comments:

  1. Are the scientists concerns really misplaced? If the lawyers find disagreement due to their ‘playing devil’s advocate’ deliberations, they’ll do everything they can do cast doubt on those very peer-reviewed conclusions. In an ideal, honest world, all would respect the peer-reviewed conclusions, but, as we’ve learned from other science/email collisions, those preconditioned to disagree can advertise the cherry-picked emails very loudly with harmful results.

    But, the bigger story is: what isn't subject to public disclosure? Every word a scientist speaks while at work? Can't there be some degree of conversation that remains confidential? Personal and phone conversations are often being replaced by email. I don’t know if personal and phone conversations are allowed to be confidential, or just conveniently not recorded: could BP have requested all personal meetings be taped? Perhaps email due to its permanence is NOT a good format for casual scientific bantering, and not a good replacement for phone conversation. However, this is how it is used: scientists do not consider their institutionally-owned email account as a public record. These email accounts may not be the solution, but I think it’s fair that scientists should be able to have confidential “working drafts” of their day-to-day process.

    ReplyDelete
  2. Correspondence review, when a matter rests on expert opinion, is the analog for calculations and empirical evidence. There is simply no way to assess the level of uncertainty related to a consensus of expert opinions. Personally, trust disappears the moment I'm told I can only see a consensus summary- especially the all too common practice of un-signed summaries.

    If expert opinion allocates costs, or shapes public policy-- agreement to show all work must be a condition to serving.

    ReplyDelete
  3. Every e-mail sent by executives and employees of businesses are discoverable. Why should scientists doing their business be any different than everyone else doing their business?

    Some scientists obviously think their excrement is somehow less odoriferous. Welcome to the real world.

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. from a fellow scientist:

    "i don't think everything we say and write when at work belongs to the entity that pays us. i see it more as a contract: we agree to produce publications, speak at conferences, advise the public and so on. but there must be a distinction between working on something and speaking out about something publicly. i'd be totally paralyzed if everything i did could potentially become publicized.

    at the same time, of course, we must meet high standards in what we say and write publicly. but they don't need to -- and should not -- reflect the entire weary process we went through to come to those results, which is totally irrelevant in assessing the credibility of them. all we need to do is state our method clearly and correctly to make the results credible and reproducible."

    ReplyDelete
  6. "To ensure the research’s quality, scientific peers conduct an independent and comprehensive review of the work before it is published."
    This sort of claim is also made in climate science, but is ideological or wishful, rather than reality. As all of us know, journal peer review in climate science virtually never includes an examination of data and is thus not "comprehensive" in the sense relevant to litigation. Nor, all too often, is it "independent" as understood in the real world,as associates often review one another's work.

    ReplyDelete
  7. To Emily- if your expert opinion forces me to act in a certain way or ascribes to me costs and or risks then you should be obligated to show the entire "weary process." While the "weary process" may end for you- all too often the costs go on forever for those impacted.

    " In an ideal, honest world, all would respect the peer-reviewed conclusions" I have one word to describe your comment- hubris. We need more Devil's Advocacy- not less- because we do not live in an "ideal honest world" but rather a human one. Human systems to work effectively need to be trustable. I said so and I'm the expert does not work for me.

    You should also read up on the history of science--Louis Agassiz championed the creation of the National Academies of Science where all the experts were to be chosen and controlled by Agassiz and his allies. Agassiz's intended purpose was to use this newly formed government/scientific authority to crush Darwin's supporters. (Why Agassiz failed in his attempt for controlling the NAS had nothing to do with peer review)

    ReplyDelete
  8. OK, I should have worded things more accurately: I did not mean to imply we should respect peer-reviewed conclusions without further investigation (yes i can see that my words did imply that). Rather, I was just echoing Roger's statement that the peer-reviewed conclusions, if they are defensible, will not be undermined by the "messiness of science."

    I'm happy to back up my "expert opinion" with data, code, all methods, etc. in order to defend a conclusion/show that the conclusions are thus reproducible and credible. Yes, that process is important, since, as you say "the costs go on forever" for those impacted by scientists' work.

    However, should the above include "personal" communication? Does personal correspondence allow others to determine whether or not my conclusions are valid? The burden of proof of such conclusions does lie with the scientists; why isn't full transparency and availability of data, code, and methods sufficient?

    An example of a situation that may allow the "messiness of science" to undermine defensible conclusions: Email to peer down the hall: "I don't see any trend in the data." Sent after a preliminary data analysis and before *whoops!* I forgot to actually plot the right variable.

    Uncertainty should be communicated as clearly as possible as an overt part of all work, and even if it's not, data, methods, and further analysis, not personal correspondence, will expose it.

    Legally, if emails are discoverable, then of course the WHOI employees should not get special treatment compared to other businesses. Again, maybe scientists need to realize email is public and start using it that way. I'm just positing that perhaps some degree of day-to-day work be off the public record. But maybe this won't be possible until data, code, methods, etc. are more transparent.

    ReplyDelete
  9. #1

    But, the bigger story is: what isn't subject to public disclosure? Every word a scientist speaks while at work? Can't there be some degree of conversation that remains confidential?

    Whether or not something is 'discoverable' in a court proceeding and whether or not it becomes part of public discourse are two different subjects.

    A judge can simply order a seal on discovery pending an evidentiary hearing which is a fairly common practice.

    ReplyDelete
  10. It's worse than suggested by "email request" When the Lawyer get going they think they can request every thing you touched,from everyone with whom you have had contact. There has to be a private channel where robust views can be expressed which an antagonist cannot pull up in court and use it against you.

    Have a look at the ATI request to UVA:

    "We seek materials that Dr. Michael Mann produced and/or received while working for the University of Virginia and otherwise while using its facilities and resources, as specifically enumerated in the Attachment. We seek these records from a backup server identified already by the University as part of a related search, as detailed, below.
    ...
    15. The scope of this request is to reach any and all data, documents and things in your possession, including those stored or residing on any of the specified or referenced (see FN 1, supra) computers, hard drives, desktops, laptops, file servers, database servers, email servers or other systems where data was transmitted or stored on purpose or as a result of transient use of a system or application in the course of day to day research or product processing work that is owned or contracted for by you or any of your officers, managers, employees, agents, board members, academic departments, divisions, programs, IT department, contractors and other representatives.

    2. As used herein, the words "record", "records", "document" or "documents" mean the original and any copies of any written, printed, typed, electronic, or graphic matter of any kind or nature, however produced or reproduced, any book, pamphlet, brochure, periodical, newspaper, letter, correspondence, memoranda, notice, facsimile, e-mail, manual, press release, telegram, report, study, handwritten note, working paper, chart, paper, graph, index, tape, data sheet, data processing card, or any other written, recorded, transcribed, punched, taped, filmed or graphic matter now in your possession, custody or control."

    ReplyDelete
  11. Emily,
    The problem I have is where and how expert opinion is used. As an example in NJ species are added to the endangered/threatened list using the Delphi process. Every part of the process is confidential except for the summary judgement.
    So is a non breeding bird on the edge of its geographic range declining as the result of natural variation or the loss of habitat? What metrics were used? Was it a an agreement of 100% of the experts or simply more than 50%. What questions were they asked? Who was the facilitator? What were the points of agreement and disagreement? We are not allowed to know because it is all confidential. And more importantly given the fact the summary statement is unsigned- we cannot know whether the summary is in fact representative of the expert opinion. (I'm still awaiting an answer as to whether the names of the experts are also confidential.) The evolving conversation is the science when expert opinion replaces empirical evidence. So who gets to decide what parts of the conversation remain confidential?
    Compare the above frustration with the power's available to government- the arrest of BP's engineer Kurt Mix. It would be interesting to hear the Woods Hole researcher's thoughts on their treatment versus Mr. Mix's.

    ReplyDelete
  12. i don't think everything we say and write when at work belongs to the entity that pays us.

    I think the lawyers would disagree.

    Not only does all we write at work belong to the employer, but many people are quite heavily restricted about what they can do at home. Scientists are not generally allowed to file patents in their own name, for example. Lawyers cannot work outside of their firm. Police cannot do quite a lot of outside jobs.

    People have even been sacked for expressing personal opinions about their employers too vigorously in their free time.

    There has to be a private channel where robust views can be expressed which an antagonist cannot pull up in court and use it against you.

    No there doesn't.

    The current position is not new. There is no lobby seeking to change the rules. They work. They work because transparency, even when ugly, is always better than secrecy.

    That a few people think their individual ruminations are so precious that no-one else should see them has never been a serious issue.

    ReplyDelete
  13. thefordperfect wrote: "There has to be a private channel where robust views can be expressed which an antagonist cannot pull up in court and use it against you."
    Let me ask you whether you think that should be true of the discussions among bankers at JP Morgan about the risks of their derivative trading? Or among scientists at RJ Reynolds about the risks of a new smokeless cigarette? Why should scientists at Woods Hole be any different?
    Now if you said that pre-trial discovery ought to be dramatically cut back in all cases, I'd agree with you. But I'm not seeing any case being made for special treatment for certain classes of scientists.

    ReplyDelete
  14. 1) If you wrote it down or emailed it, it can be discoverable regardless of your expectations at the time of creation.
    2) If you meet informally and discuss and debate and don't take notes, there is nothing to discover although you may be deposed and asked why the secrecy and who said what.
    3) Scientists should resist the conceit that they are too precious and special to ever be judged by others or be held accountable by laymen because of their personal wonderfulness for being scientists.
    4) Scientists should especially resist the conceit that they can deign to come down from Olympus and shape public policy at leisure without having to endure any impertinent questions from the laity. You want public money and to make policy, you play by the same rules as everybody else.

    ReplyDelete
  15. .

    Once "Hide the Decline" became public, there is no reason if there ever was one to take a scientist's word at face value. Pity that.

    .

    ReplyDelete
  16. Abdul Abulbul Amir is right. After the public realized that it had been betrayed by scientists there was good reason not to take scientist's word at face value.

    AGW (alias environmentalism) is well-packaged propaganda designed to redistribute wealth in the fascist, one-world society that nobody intended when this path was chosen in 1945 to "save the world" (and world leaders) from the fate Hiroshima suffered on 6 Aug 1945 - nuclear annihilation.

    The sad tale of deceit and betrayal of the document that established our country on 6 July 1776 – The US Declaration of Independence - is documented in these links:

    http://omanuel.wordpress.com/about/#comment-105

    http://www.ushistory.org/declaration/document/

    With kind regards,
    Oliver K. Manuel
    Former NASA Principal
    Investigator for Apollo
    http://www.omatumr.com

    ReplyDelete
  17. I agree with Stan, everything done in business is available to discovery and often has been tossed into the public domain. The fact of discovery has been part of life for the rest of the world for decades and it is about time it penetrated the secret and corrupt world of academia.

    An above-the-board open process where all data, code, and processes are published is the only way to conduct science. Unfortunately, most of what has passed for "science" outside of some specific fields of physics does not meet this minimum threshold and has complicated the ability to know what is "good" science and what is just meaningless dribble meant for academic advancement.

    ReplyDelete
  18. If the taxpayers funds, then yes, all matters related must be public. It is our right.

    ReplyDelete
  19. This issue is revisited in an article in the current edition of SCIENCE by Eli Kintisch, who writes a piece blatantly sympathetic with the WHOI scientists.
    All governments agencies must archive and produce emails on demand. Scientists that contribute to the work product of such agencies must be subject to the same rules.
    I agree with Jr. entirely -- it is the science that must stand up. If the authors of that science have doubts and disagreements, and later decide to cover them up then they are doing 'climate science science' not real science.

    ReplyDelete