29 August 2009

Scientific Arguments as Tribal Politics

Michael Tobis, a climate scientist who can always be relied on to say what he really thinks, provides another remarkable glimpse inside modern climate science as he explains how he evaluated the arguments in Klotzbach et al., our recent paper that he continues to dismiss as "nonsense":
No scientist really knows everything he or she claims to know from direct experience. Most of what we know as individuals comes from two factors: 1) a network of trust and 2) the test of coherence. . .

Consider how I got mixed up with the Klotzbach paper. Prominent naysayer Joe d'Aleo's blog alleged that most warming on land was due to a bias in the land surface temperature record, referring to Klotzbach's paper. Klotzbach's claim was not consistent with my coherence network. Therefore, I resolved to figure out what the unclear claim really amounted to. The deeper I got into it, the stranger it got. Eventually, one of the authors was compelled to admit that the word "bias" did not actually indicate an error in temperature. Further reading revealed many other flaws in the work, although the key one, which appears to have been reported (somewhat at second hand) by James Annan, is something I still don't entirely understand.

This is where the first principle cuts in. Should I further investigate the key claim, still contested by the authors? Well, I know James to be an extraordinarily careful and precise thinker, and it's already demonstrated that his opposition is not. Since boundary layer meteorology is not my forte, and since the rest of the paper is flawed in many ways, I feel satisfied that it's best to put my attentions elsewhere.

The main point for present purposes is that I immediately questioned the result claimed by d'Aleo on the basis of its incoherence with everything else I know. And my questioning turned out to be justified. The publication, though it passed peer review, probably should not have done so. It looks like science from a distance, but up close it looks like nonsense.
Remarkably, Michael admits to having no expertise in the subject of our paper and not to understand the arguments made in it or about it, but because the paper challenges the beliefs that he and his tribe ("coherence network") firmly hold, he concludes that the scientific arguments in our paper must be wrong. He in fact knew this to be the case before even reading our paper. Michael speaks of an "opposition" -- and presumably these are the people in other tribes who hold views that are different than his own. I guess I am in that "opposition" because someone Michael dislikes happened to blog on our paper (d'Aleo). Michael reminds us that, after all, some other climate scientist in Michael's tribe (James Annan) who also admitted to having no expertise in the area of our paper said it must be wrong.

Will Michael's or James' critiques of our work appear in the peer reviewed literature? Of course not (because their critiques are off target and simply wrong). But their critiques do serve an important sociological function by reinforcing the tribal network and also give some comfort to those who evaluate arguments solely by their degree of conformance with views already held. Had Michael been blogging around the time of Copernicus, he would have explained to his readers that the world is in fact at the center of the universe, and that Copernicus guy must be wrong, because Michael and all of his Ptolemian friends said the world was at the center of the universe, so those saying differently must be wrong because they do not jibe with his "coherence network."

The troubling thing about this is not that people evaluate arguments based on trust -- we all do and it is a necessary part of life. What is troubling is that Michael suggests that he has found our paper "is flawed in many ways" but the basis for this is a felt need for tribal affirmation and not having his firmly held beliefs challenged. His scientific judgment on our paper is not grounded in the logic, data or analysis found in our paper which passed peer review and is published in a leading journal in the field. To the extent that Michael's views of scientific arguments are shared among his climate science peers, that community loses its ability to evaluate arguments based on their merits rather than by their putatuve tribal characteristics. I don't think that all or even most climate scientists think or act this way, but my experience is that enough do to create an unhealthy degree of politicization within the community. Michael is to be applauded for his candor, but what it reveals is not a pretty sight.

So here is an offer to Michael Tobis: Write up a serious critique of our paper's logic, data and analysis and I will publish it prominently here. Prove that our paper is in fact "nonsense" based on science and that your evaluation of it is not simply tribal politics played out on a blog.

23 comments:

Michael Tobis said...

I do not think the point about "tribes" is correct. I admire (and like) James, but I don't to my knowledge share any loyalties with him other than to science itself.

I stand by my points in
http://initforthegold.blogspot.com/2009/08/my-final-word-on-klotzbach.html
including the point that I don't find the paper worth examining in any further detail.

As stated in that blog article, I rely on Parker '04 and Parker '06 as counterevidence. It would not be difficult to replicate the Parker studies as they are based on available observational data. Given that, as well as James' claim that the Pielke Sr. & Matsui mechanism is incorrectly evaluated, it is difficult to see why the paper would reward further research.

In the grand scheme of things, my opinion isn't that important. It's kind of you to promote me to "scientist", but it's a marginal call, really. I do, though, try to think like a scientist.

The unusual part of what I am doing is not in deciding to abandon further interest in Klotzbach, but in asserting publicly that I have done so. It's more typical to drop things like that quietly. I realize this is awkward for both of us, but I thought making this aspect of how scientists direct their attentions explicit would be illustrative of the actual ways in which truth emerges from the scientific process.

The possibility that I could be spectacularly wrong on this matter has not escaped me. However, at this point I find the likelihood of that outcome extremely small. You will have to engage somebody else's attentions if you want to have further debate about this publication.

Roger Pielke, Jr. said...

-1-Michael

Thanks for dropping by and explaining to us how "scientists direct their attentions" and "the actual ways in which truth emerges from the scientific process." You have indeed shed some light on both of these processes -- I can only hope that your rather interesting approach to "science" is not widely followed among climate science practitioners.

Over the long run (and through the peer reviewed literature) we'll see how well Klotzbach et al. stands up. Maybe it will, maybe it won't. Though I do wish I could share your degree of utter certainty in knowing the "truth" ;-)

Your response to my invitation is noted.

Sharon F. said...

You know I am developing a hypothesis that "climate science" is a network of subfields (e.g., boundary layer meteorology) The fact is that no one other than an expert in each subfield is probably knowledgeable enough to debate the points of a given paper. And as you have argued, Roger, the specific findings of each subfield are not necessarily all that relevant to the larger policy issue - what should we do about climate change?
So maybe "climate science" is a network of information that is informed as scientific discourse within subfields but informed socially (or tribally, as you would have it) among subfields.. which could make random groups of scientists summarizing across fields not really science. So there would actually be no scientific field of "climate science." Just a hypothesis..

Roger Pielke, Jr. said...

-3-Sharon F.

I think your observation is correct. "Climate science" (e.g., as represented by the IPCC) really includes many, many disciplines and interdisciplines. And tribalism within academia is really nothing new. But even so, there are plenty good examples of interdisciplinary synthesis, and at its best the IPCC is a good example of this.

But while it is true that no one can be an expert in everything, it should still be possible for people to evaluate arguments based on their merits (even if it means learning something new). Climate scientists such as Tobis do their field(s) a great disservice by explaining that they can evaluate peer-reviewed scientific arguments without resource to data or analysis. They can simply decide based on the blog that happen to read about it on! ;-)

Not Whitey Bulger said...

From the first link above:

"Climatology, being broadly interdisciplinary, may lack rigor but it isn't likely to miss broad brush questions."

The former is quite an admission. The latter? Given the former, hardly a reliable assertion.

Sylvain said...

Mr Tobis, isn't that convenient?

"including the point that I don't find the paper worth examining in any further detail"

That seems typical of climate science, when you don't like something ignore it, and pretend it does not exist.

Yet, I'm not sure what your complain about the paper actually is. It does not dispute GHG theories. It doesn't dispute that warming is observed. It mainly stipulate that land surface show bigger trend than satellite and ocean temp.

The methodology employed is quite simple and compare different observational data set between
each others. The result is that statistically significant trend are observed only between land surface temp and satellite, I don't believe they compare ocean temp to land temp as they did with satellite (which might be interesting), and show no statistically significant trend between the other sources.

I don't believe that the paper goes into detail of why there is such a differences since I can't access GRL.

Is the only thing, that you don't like, is that the believed warming might not be as high as previously thought?

Andrew said...

Buck up chum. As they say in the military:

Illegitimi non carborundum

Which is Dog Latin, but then if I said it correctly, it would be a real insult to those guys, and nobody would get it.

Better to jokingly insult people in a transparent, incorrect way.

andrewt said...

So Michael& James are no longer goign through Haldane's 5 stages of acceptance - now they are are blinded by tribal loyalties to your Copernican revolution?

Anyway if you must compare yourself and coauthors to Copernicus, perhaps you could at least discover why he is famous. Ptolemy thought the earth spherical and had a decent estimate of the eath's radius which let him codify a system of latitude/longitude - unlike Copernicus he thought the earth was at the centre of our solar system.

Roger Pielke, Jr. said...

-8-andrewt

Yes, I believe they are stuck in the "one stage of rejection" ;-)

Your comment about Copernicus reminds me of a message that we received from a classics professor upon issuing our first Center newsletter a while back. See it here:

http://sciencepolicy.colorado.edu/ogmius/archives/issue_2/ogmius_news.html

;-)

itisi69 said...

These people are not scientists, they are advocates who defend their truth, against better knowing. How can one diss a paper without even reading it properly or knowning the underlaying science?

bernie said...

Roger (#9):
Those appear to have been kinder, gentler and more witty times!!

Andrew said...

"It looks like science from a distance, but up close it looks like nonsense."

Compared that to:

"something I still don't entirely understand" and "boundary layer meteorology is not my forte"

Uh, sir, how did you even get close?

Old Dad said...

A layman's take. I'm a Ford man. My neighbor is a Dodge man, and he's got a close circle of buddies who are also Dodge men. I'm quite the expert on all things Ford, and my neighbor and his circle are equally expert on Dodges. I'm lucky enough to get a feature in Hotrod about how I renovated and souped up my 1979 Ford F-150. My neighbor writes a snarky letter to the editor because he disagrees with my choice of paint jobs. One of his Dodge buddies argues that a 36 hour dry time on the prime is the only way to go. My neighbor concludes that my article is nonsense, but then again, he's a Dodge man. He's told me many times that he doesn't know diddly about Fords except that they stink. His buddies told him so.

I hope I'm mistaken in this interpretation.

Sharon F. said...

Following Old Dad’s analogy, if you randomly select Fred and Nancy from the Dodge and Ford factions and ask them about cars, would the results be “car science?” What if Nancy’s personality were overbearing and Fred’s meek? Would we get the same answer if we teamed Fred with Flavia, who is known to be mellow? If the answers from our two teams are different, are they both equally valid “car science?”

Larry said...

Old Dad, no you're not. That's a very good analogy. Ford trucks have decals with Calvin peeing on a bowtie. Why? Why do people root for sports teams or political parties? These are all tribes. Remember "I don't know anyone who voted for Nixon"?

Andrew said...

Larry-I don't "remember" this (way to young) but there were also the bumper stickers "Don't Blame Me, I'm From Massetchusetts"

Roger Pielke, Jr. said...

Michale Tobis has a strange way of "not debating" a topic which includes continued vigorous debate! See him a Lucia's blog one day after calling of debate:

http://rankexploits.com/musings/2009/7003/

Lucia's comments are well worth reading.

Roger Pielke, Jr. said...

In that same thread at Lucia's DeWitt Payne demonstrates that he has a better reading comprehension level than James Annan:

DeWitt Payne (Comment#18880) August 30th, 2009 at 1:45 pm

James Annan’s objection to the Klotzbach, et. al. paper is a classic straw man argument. He objects to something the paper doesn’t say. This is easy to discern whether you understand boundary layer meteorology or not. James Annan objects to the paper because he claims that the authors conclude that ghg’s do not warm the atmosphere and that they claim that the underlying temperature measurements are inaccurate. How he arrives at this conclusion is beyond me. He couldn’t have read the paper carefully. There is no statement to this effect in the paper. You don’t have to even look at the math to realize this.

The fundamental argument of the paper is really quite simple in terms of heat transfer. On a calm, clear night, heat radiates away primarily from the surface. The layer of air closest to the surface then transfers heat to the surface and the classic diffusion layer temperature gradient forms. By the time minimum temperature is reached the diffusion layer is up to about 10 meters so there’s quite a large gradient between the surface and 2 meters. Now impose a trend on the surface layer over time. The claim of the paper is that the gradient at 2 m decreases in a warming trend and increases in a cooling trend regardless of the reason for the warming or cooling. That means the minimum temperature at 2 m warms faster than the surface in a warming trend and cools faster than the surface in a cooling trend, exaggerating the trend in either direction and introducing a bias in the average temperature calculated from (max+min)/2. Changes in CO2 and clouds modify the size of the effect, but it’s still there. Wind reduces the height of the diffusion layer and at some wind speed the thickness of the diffusion layer is less than 2 m so there is no effect on the measured temperature. So even if one assumes the temperature measurements are perfectly accurate, there is still a bias in the trend because of the location of the measurements.

But of course, a lowering of the trend by the authors’ estimated 0.05C/decade would make the models even further off and we can’t have that, so out come the big guns.

Roger Pielke, Jr. said...

Thanks to all 7 people who wrote in noticing my error in science history, now corrected -- chalk it up to jet lag ;-)

leonardeuler said...

You might find a brief exchange I recently had with Mr. Tobis in this thread on his blog a bit amusing. At one point in frustration I jotted down three questions I felt summarized the "important things to resolve" in this whole business of global warming:
(1) are the current changes abnormal?
(2) if so, is the abnormality caused by man's influence?
(3) are the current changes, on balance, harmful?


His reply was as revealing as it was dismissive:
Regarding your three questions, they misframe the issue entirely, as people who focus on the observations and not the theory frequently do.

The three questions I would propose are

1) Is there a strong basis in physics for expecting anthropogenic CO2 increases to affect climate?
2) Is the scale of such changes likely to become very large and very disruptive, based on the best physical understanding?
3) Do the observations accord with the theory?

...

But your questions are a straw man that your allies have set up. They do not properly represent the reasoning behind the concern expressed by the experts. The fact that they are not a slam dunk doesn't mean that concern about carbon accumulation is similarly at issue.


Um, as I understand it, to "focus on the observations and not the theory" can be roughly translated to "interrupt the politically convenient narrative with facts that inconveniently refuse to conform". I also find it revealing that he appears to regard "physics" and the assumptions behind the computer models that supply the bread on his table as synonymous. Apparently he regards these jerry-rigged and famously unreliable, assumption-ridden "models" as the unadulterated offspring of Newton's Principia, reductio ad mathematicorum. How inconvenient that, unlike the case with Principia, nature appears unwilling to conform to this "theory".

Interesting that Mr. T thinks of such questions purely in terms of "allies". The word "tribal" hadn't occurred to me but you certainly have a point here, and I think it nails the attitude nicely. He regards "climate science" so thoroughly in these terms he apparently cannot imagine that someone who disagrees with him actually has questions of their own...no, they must be reading from some "denier" playbook. It's not easy to articulate the disgust that kind of tunnel-vision inspires in me. Does Mr Tobis not understand that I have merely articulated, in my own words, the same questions that will occur to most of the very public he wishes to convince?

Well, apparently Mr T felt my questions so epitomize the way "deniers" try to skirt the "real issues", and his reply so clever, that he chose to feature our exchange of questions prominently in a later post on his blog, where he summarizes as follows: "We let the bafflegab from the opposition obscure our message and confuse our message. Here is a key strategy which both demonstrates and dominates the way nitpicking diverts form the real issues."

Well, my apologies for using technically baffling terms like like "normal", "influence" and "harmful", for my insidious "nitpicking" over little things like "facts" and "observations" and for obscuring the "real issues" by asking about trivialities like whether there is anything abnormal, harmful or anthropogenic about the changes we're seeing.

gt said...

Re: leonardeuler

The answers to MT's questions are no, no, and no.

craigloehl said...

I am afraid that if one does not understand a topic in some detail it is simply not possible to judge it to be nonsense. The expedient approach of relying on your favorite experts is not sufficient when a topic is rapidly changing, highly politicized, and very complex.
Craig Loehle

DeWitt said...

Roger,

It may be a slightly left handed compliment, but I'll take it. Thanks,

DeWitt

Post a Comment

Note: Only a member of this blog may post a comment.