Author Topic: Standard Deviation Questions  (Read 3305 times)

neko

  • Professor
  • ****
  • Posts: 66
Standard Deviation Questions
« on: February 26, 2016, 12:58:08 PM »
The obscure reference that details how the Cameca software calculates detection limit.

Speaking of this obscure, 30 page long paper, does anyone have a concise explanation of how the calculated st dev wt% is applied to measurements?

"The StdDev Wt% precision corresponds to a confidence of 95% to be in the interval of 3sigma given as a result."

I don't have much statistics training so I'm probably about to make a fool out of myself, but does that mean the reported StdDev is an absolute 3 sigma error? E.g. 50% Si with a .25 StdDev Wt%, 49.875 to 50.125% would be the 3 sigma range the measurement is 95% confident to be inside??

Probeman

  • Emeritus
  • *****
  • Posts: 2839
  • Never sleeps...
    • John Donovan
Re: Standard Deviation Questions
« Reply #1 on: February 28, 2016, 01:25:08 PM »
The obscure reference that details how the Cameca software calculates detection limit.

Speaking of this obscure, 30 page long paper, does anyone have a concise explanation of how the calculated st dev wt% is applied to measurements?

"The StdDev Wt% precision corresponds to a confidence of 95% to be in the interval of 3sigma given as a result."

I don't have much statistics training so I'm probably about to make a fool out of myself, but does that mean the reported StdDev is an absolute 3 sigma error? E.g. 50% Si with a .25 StdDev Wt%, 49.875 to 50.125% would be the 3 sigma range the measurement is 95% confident to be inside??

This is getting a bit off topic, but it's a good question (Edit by John: moved to new topic). 

I also am no expert in statistics, but I do know that the standard deviation is the one sigma variance which describes how many measurements in the data set are predicted to be included within a normal distribution variance.

That is, one can calculate two sigma and three sigma statistics to predict inclusion of 95% and 99%, respectively, of the measurements in a random (normal) distribution of data. But I don't know why the quoted paper (I found it rather dense to be honest), suggests that 3 sigma includes a 95% distribution, unless they are defining things differently (are they using Poisson statistics for low count rates?).  But as I said: I'm no expert in statistics.

What also confounds me is why one sees the standard deviation used to describe the variance of the mean, when it really should only be applied to the individual measurements.  That is, someone will quote an average and the std deviation, for example 0.005 +/- 0.002.

But the variance around the mean is actually better than that (you do actually get some benefit by making more measurements!), and the value that *should* be quoted for the variance of the mean is the standard error.

Probe for EPMA calculates both values as seen here in this set of trace element measurements in zircon:

ELEM:       Th      Hf       U       P       Y   SUM 
    49    .001    .034   -.007    .000   -.001 100.026
    50   -.002    .034    .000    .001   -.002 100.031
    51    .008    .035   -.001    .001    .001 100.044
    52   -.001    .015    .013    .001   -.001 100.028
    53    .005    .018    .002    .001    .003 100.029

AVER:     .002    .027    .002    .001    .000 100.032
SDEV:     .004    .010    .007    .001    .002    .007
SERR:     .002    .004    .003    .000    .001

I once asked an actual statistician why people quote the standard deviation along with the average and he said, "don't know, maybe because it's a more conservative estimate..."

So there it is.  Are there any experts out there that can chime is on these questions?
« Last Edit: February 28, 2016, 04:42:40 PM by Probeman »
The only stupid question is the one not asked!

Jeremy Wykes

  • Professor
  • ****
  • Posts: 42
Re: Standard Deviation Questions
« Reply #2 on: February 29, 2016, 03:24:53 AM »
Australian Synchrotron - XAS

Probeman

  • Emeritus
  • *****
  • Posts: 2839
  • Never sleeps...
    • John Donovan
Re: Standard Deviation Questions
« Reply #3 on: February 29, 2016, 02:46:51 PM »
Nassim Taleb has an opinion: https://edge.org/response-detail/25401

This is also useful: http://www.leeds.ac.uk/educol/documents/00003759.htm


Hi Jeremy,
Excellent links.

Well, I'm just glad to find out I'm not the only person who's thought the standard deviation is not always being applied appropriately!
john
« Last Edit: February 29, 2016, 05:24:20 PM by Probeman »
The only stupid question is the one not asked!