Statistical Significance – The Simple One

Joanne Nova recently branched out of her comfort zone (spreading doubt) to try her hand at statistics claiming that surface temperature data since 2005 shows statistically significant cooling in 4 of 5 datasets.

Her words …

The cooling for the last eight years is statistically significant in 4 of the 5 major air temperature datasets.

Statistically significance is very rarely found on such short timeframes, so how did Nova manage to find it? Simple! She invented her own concept of statistically significance; “The easy one” as she put it.

When shown how much variability existed, and how statistical significance methods showed large variability Nova replied …

So you can come up with a different more complex method but don’t recognise the simple one? How odd. You have referred to one in a paper by Foster and Rahmstorf that is complicated. I used a simple, reasonable, rough and ready method and drew conclusions that fitted (see all the caveats). It is not sophisticated but it is still valid.

Nova has posted about statistical significance before and insists “no half-decent scientist would claim that the world was warming knowing that it was statistically insignificant.

The Simple One?

Nova goes on to explain her unique SS method.

Is the trend greater than the errors in the individual measurements (about 0.05C per data point here)? If the trend is less than the errors of individual instruments it is not really credible. With your preferred measure it looks like a trend could, under some circumstances, be statistically “significant” even though it is less than the individual measurement error. Hmm.

Oh dear. Nova is wrong, for so many reasons, I suspect her misunderstanding stem from mistakenly confusing the term “significant trend” with “a large trend”. When talking about statistical significance we’re after a confidence that the trend is not just by accident, but rather that it represents the true trend. In “Nova world”, she believes the gradient of the trend must be large in order for it to be called significant.

Error 1

Trends are more affected by natural variation, such as ENSO than by instrumental error (let’s assume that Nova’s 0.05°C instrumental error is correct). Each month global surface temps can jump by as much as 0.4°C. This is the natural monthly variation caused as heat transfers into and out of the oceans, distorting the real trend. The month to month variations are NOT caused by instrument errors.

So any method for comparing the trend against the noise (for statistical significance purposes) needs to evaluate against the natural variability (noise). The noise is inherently in the data, not some “instrument error” evaluated externally.

Error 2

By Nova’s definition a neutral trend of less than 0.05°C, neither warming or cooling, would never be statistically significant. If the data was perfectly flat, Nova would say the trend was statistically insignificant because it neither went up or down. Sheesh!

Error 3

Nova’s significance test depends on what scale is used for the trend, in her case, °C/decade. Does this mean if we stated the trend as °C/year it would suddenly become invalid? How on earth did Nova arbitrarily decide that  °C/decade can be compared to instrument error?

Error 4

With Nova’s SS test, we find almost any timeframe we pick passes the Nova SS test and that it passes more easily with less data. According to Nova logic, the following trends are ALL “Nova statistically significant” because the magnitude of the trend is greater than 0.05.

Example 1 – Since 2008, warming with “Nova statistical significance” because the trend is 0.25°C/decade.

2008 trendExample 2 – Since 2010, the planet is cooling with “Nova statistical significance” because the trend is -1.03°C/decade.

Example 3 – Since 2012, warming with “Nova statistical significance” because the trend is 2.45°C/decade!!

Example 4 – Since 2013, plunging into an ice age with “Nova statistical significance” because the trend is -29.76°C/decade!!

2013 trendUsing Nova’s SS test, we can be 95% sure that we are plunging into an ice age this year based on Jan and Feb’s global surface values. How ridiculous! Not only that, the examples show that with less data, the magnitude of the “Nova significance” has increased, which is opposite to statistical theory.


Statistical significance in surface temperature trends usually takes a decade or more to achieve simply because of the large monthly and yearly variations. These variations occur naturally as heat is transported into and out of the oceans, not because of instrumental inaccuracy. More data gives greater statistical confidence, not less. Cherry picking 2005 and announcing that it is cooling with statistical significance is as stupid as cherry picking 2008 and claiming it is warming with statistical significance – something no climate scientist has ever done.

But that’s Nova logic for you. A frightening example of someone who overestimates their own ability.

Tags: ,

4 Responses to “Statistical Significance – The Simple One”

  1. Vince Whirlwind Says:

    I too thought that quite hilarious when she tried to deny she’d misused the term “significant” by trying to imply it had alternative uses.
    So in addition to over-estimating her ability, she is in a constant state of denial over the many errors she makes.

  2. FIN Says:

    Great stuff, I’ve only just recently stumbled across this site. For some time now I’ve been grinding my teeth in frustration at the egregious bullshit “Cherry” Nova regurgitates at her site. Thank you for putting the record straight, much appreciated, well done.

  3. MMM Says:

    Error 2 isn’t quite right… “statistically significant”, as usually used, refers to a statistically significant difference from a null trend. Therefore, a perfectly flat trend would, in fact, never be statistically significant.

    Now, on the other hand, a trend of, say, 0.04 degree/decade could be statistically significant, despite a measurement error of 0.05 degrees, as long as the trend is long enough in comparison to the noise…

    • Mark F Says:

      What if the “null trend” is the linear trend of the past 30 years of temps, then a perfectly flat line is the deviation from that?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: