07 December 2011

What Are the Odds?

It is early 2006, and you are asked what the probabilities are of 6 years in a row of US hurricane seasons with no landfalls of intense (Cat 3+) hurricanes. No such event had ever been observed.

What would you say? Here is what I'd have said:

1. From 1900 to 2005 there were 69 of 106 years with intense hurricane landfalls (data in graph above from NOAA).
2. That means that, based on the entire record, the odds of a single year with no such landfalls is 0.349 (i.e., 1 - (69/106)).
3. The odds of six such years in a row is thus (0.349 ^ 6) (to the sixth power)

. . . or 0.18% or 1 in 553.

What would you have said?

26 comments:

Tom said...

"I don't know," would seem a sensible response, but then climate and weather are both well outside my field.

That's a very cavalier assumption of independence you make there. Is it justified?

Hector M. said...

Roger,
it's not the odds you are mentioning, but the probability. If p is the probability, the odds are p/(1-p).

Roger Pielke, Jr. said...

-1-Tom

Well, I've published a paper arguing that we have no skill in 5-year hurricane landfall forecasts. But I think we can say more than "I don't know."

This methodology would be classified as a "naive" expectation based on long-term climatology.

Roger Pielke, Jr. said...

-2-Hector M.

Yes, you are correct! Fixed in the text of the post ... Thanks;-)

Tom said...

Do you know if the data behind that graph is available somewhere? I can only spot the image on the NCDC website.

Roger Pielke, Jr. said...

-5-Tom

Try this:

http://www.aoml.noaa.gov/hrd/hurdat/usland1851-1930&1983-2010-Mar2011.html

Harrywr2 said...

I would have said never bet on a coin toss unless you are the one tossing the coin.

In my misspent youth I managed to toss 27 heads in a row, or just to show off the same number of tails.

Malcolm said...

What happened in the past is not an indication of what will happen in the future.
The odds are somewhere between 0 and 100%

Salamano said...

Could you imagine if an insurance company only paid off if a hurricane managed to landfall at a certain category threashold? Talk about bedlam.

Tom said...

Well, I've just killed an hour and a half trying to get R to read the NCDC data, and the answer is... there is minimal autocorrelation in the duration between Cat3+ landfalls. So independence seems a good assumption.

R frustrates me.

Tom Scharf said...

I would say once again climate change has caused hundred year events to happen much more frequently! ha ha.

Abdul Abulbul Amir said...

.

We should be thankful for these very beneficial effects of global warming.

.

Tom C said...

You need to use the binomial distribution. So, for X = 1 event in N = 6 years with a probablility of 0.65 for the event itself (no landfall), it would be:

N! / X! (N-X)! p ^ X q ^ N-X

6 * 0.65 ^ 1 * 0.35 ^ 5 = 0.02

or 2%

Unknown said...

Is that graphic supposed to be some sort of test? Ya know, like see if anyone notices all the errors in the graphic?

According to the best tracks database.
1900 had three hurricanes, two majors and one of those hit the US. That color should be blue.
1901 had five hurricanes with zero majors.
1902 had three hurricanes with zero majors.
1903 had seven hurricanes with one major that hit Mexico not the US.

On the other hand, I see now what the path is to that graphic. Maybe it isn't a good idea to put up graphics that have research as part of the path.

Tom said...

-14-Unknown

If by 'best' you mean IBTrACS, I don't see any easy way of assessing US landfall (of course you may well have access to somewhat better GIS tools than I).

The graph above comes from the Atlantic Oceanographic and Meteorological Lab (AOML) dataset - http://www.aoml.noaa.gov/hrd/hurdat/ushurrlist18512008.txt - which indeed lists one hurricane, at Cat 4, making mainland US landfall in 1900.

Unknown said...

Roger,
I'm the unknown comment #14. I'll put my name at the bottom of this comment in case it happens again.

When doing your odds calculation you used [total majors/years]. Since some years had more than one major US land fall, you should have used [years without a major/years]. I used the Safir Simpson damage rating and not the peak wind speed to classify the storms since a few storms did not reach category 3 by wind speed, but did according to SS damage. My major storm total is 70 in 52 years.

With 54 years during 1900-2005 period without a major US land fall I came up with much more realistic odds for a six year derth of majors.

54/106 = .509
.509 ^ 6 = .017
(1/.017) -1 = 56.2 to 1 odds

A 7th year would make the odds 111.3 to 1

#15 Tom,
I have the Hurdat Best Tracks database from NHC through 2008. Safir Simpson damage ratings are indicated for all/only US hurricanes. I wrote a tool to reparse it into two files(storm data and track data) to make it suitable for spreadsheet use. I also added a few bells and whistles such as travel distance between observations, current compass direction and distance to nearest land. Haven't done much with it in the past couple years, but I doubt there have been many substantial reanalysis changes to old data in the past three years. Maybe in the spring I'll download the latest one through the current season. If you are interested in seeing (you won't want to work with it) what the original NHC Hurdat file looks like you can download it here. http://www.nhc.noaa.gov/pastall.shtml#hurdat

Over the entire database I don't see any significant change in storm intensity, frequency or location which can't be accounted for by improvements in detection.

Bob K

Roger Pielke, Jr. said...

-16-Bob K

Yes, if you redefine the SS scale according to damage then you will get different results. I also think that the 6 year stretch goes away too ;-)

Unknown said...

Ok. I see what you're doing. You're defining the category as the wind speed at land fall instead of using the peak wind speed for the entire storm. You may be entirely right with your figures.

I see my 2008 database is missing about eight older storms which have been inserted since I last downloaded it. Guess I'll have to do a rebuild to include the updated figures when I get a chance.

I don't know why my google account isn't passing my nickname back to insert in my comments. I just checked and my nickname is still there. Used to work. Haven't logged in using it for a long time though. Oh well. :(

Bob K

Mark B. said...

One could ask for the odds of getting six years in a row with no hurricane landfalls. A more relevant question would be, what are the odds of getting six years in a row STARTING NOW. It's the difference between getting six heads in a row with coinflips, and getting six heads in a row in the next six flips.

This is the error made (deliberately) by climate scientists in recently claiming that the current lack of warming 'is not unexpected.'

Mark said...

3. The odds of six such years in a row is thus (0.349 ^ 6) (to the sixth power)

. . . or 0.18% or 1 in 553.


The odds of any one selected six year stretch. But there have been 100 such possible stretches, so the odds are much closer to 18% than 0.18%. (16.5%)


You need to use the binomial distribution.

Poisson would be better, no? Since it allows for multiples strikes in a year, rather than yes/no.

With that you get lambda = 69/106 per year. So the lambda for a six year stretch is 3.91. The probability of zero strikes in any one selected six year period is then = 2%.

But again we have a 100 opportunities for this to occur. So that means a blank six year stretch is 87% likely. (1-(0.98^100)

Roger Pielke, Jr. said...

-20-Mark

Thanks, but I question your math ...

The question was, at the start of 2006, what would you have handicapped the probabilities of the next five years being free of Cat 3+?

That is a different question than, over 106 years, what are the odds of a Cat 3+ stretch?

I reject the Poisson distribution usage in this case, simply because the units of analysis are year with intense strikes, not average strikes per year.

Thanks!

Cees de Valk said...

I would expect the intensity (expected # of landfalling storms /year) to be correlated due to fluctuations in the climatic background at different time-scales.
So the probability would likely exceed the estimate based on a year-to-year stationary Poisson process. Same remark applies when considering the indicator for no strike in a year; also expected to be correlated.

bernie said...

Roger:
I tend to agree with Mark in the way the question should be addressed, but given the way you have posed it, I would say that the odds are somewhat larger than 0.349 ^ 6. For starters since last year was already a blank and you are asking for the next five years, 0.349 ^ 5 is more accurate. However, you base your estimate on the hurrican record of the last 106 years which assumes that the processes that lead to US landfalls of hurricanes has and will remain constant for the next 5 years. But this assumption is surely unfounded. If we base the estimate on the last 6 years, then of course we get 1.0^ 5. So assuming that we need to chose a period between 6 and 106 in order to calculate the existing frequency, then we know that the probability is somewhere between 0.349 ^ 5 and 1. Matt Briggs would have a field day with this problem, since we are in fact dealing with an ill-posed question and a frequentist approach assumes that we know far more than we do.

Mark said...

Roger, the odds of the stretch starting in 2006 being free of Cat3 or higher is your 0.2% using binomial and my 2% using Poisson.

But why 2006? It could have been 2005 onwards, or 2004 onwards, etc. The odds of the current decade containing a long free stretch are obviously much higher than one specific period.

Cees, surely if the events are correlated, my Poisson distribution will give a likelihood that is too high, not too low, since reality will give double strikes more often than the distribution allows, which means my lambda is too high. I agree that a free year is likely to be correlated to the next year being free, but that's why I don't like a simple binomial.

Roger Pielke, Jr. said...

-24-Mark

2006 is chosen because that is the first year that cat modeling firms started issuing 5-year forecasts, and thus also the first five-year period for which we have full verification data.

The order of magnitude difference between binomial and Poisson is very interesting indeed. Poisson does seem too high, if only because a 6-year stretch has only be observed once in 112 years.

(a follow on question would be, what are the odds of seeing at least 1 6-year stretch over 112 years under binomial and Poisson distributions -- does one use binomial or Poisson (or some other) to calculate the answer?;-)

Mark said...

I'm no statistician - only a High School teacher who can work my war round the various common distributions.

If I was doing this with students, I would run a Monte Carlo simulation. That would allow me to tweak my hurricanes to be (lightly) correlated.

I confess, regardless of the odds, that it amuses me that severe hurricanes are absent precisely as the least sensible alarmists were suggesting they would be much more common.

(With global temperatures at level, Arctic ice at level and sea level rises decreasing it is a good time to be a sceptic!)

Post a Comment

Note: Only a member of this blog may post a comment.