Analytical uncertainty in DTSA-II vs PFE?

[orlandin]

Hello, all!

I have gotten DTSA-II to reproduce a wide variety of geological standards with high accuracy (RDEV<2%), but have gotten confused when trying to explain the precision of the technique to people. I was initially tempted to take PFE analyses of the same standards, propagate the 'analytical error' to 2-sigma, and compare with twice the sigma reported in DTSA-II - but now that I look into how these two uncertainties are calculated, they do not appear the same. Looking at the 'advanced topics' help file for PFE, it does not seem to consider the absorption matrix correction when reporting analytical error. This is a major contributor to the uncertainty reported in DTSA-II.

If the question is 'but how precisely do you know that calculated concentration', should I not be using the 'analytical error' reported by PFE? Is there a similar statistic in the Report of DTSA-II that is directly comparable to PFE's analytical error, setting aside which estimate is more robust for the moment?

Thank you!

Best,

Phil

[Nicholas Ritchie]

It is true that DTSA-II makes a more thorough accounting of uncertainty.   In addition to the contributions to the precision of the measurements (count statistics), it also computes contributions from uncertainties in the mass absorption coefficient and the backscatter coefficient.   DTSA-II tabulates both a effective uncertainty due to all contributions (which it reports as the total uncertainty) but also the individual contributions.  There are four independent components - uncertainty (precision) due to count statistics in the unknown, uncertainty (precision) due to count statistics in the standard, uncertainty (accuracy) due to the absorption correction (the MAC), and uncertainty (accuracy) due to the backscatter coefficient.  Collecting more data can improve the first two but not the later two.  The optimal direct comparison between PfE and DTSA-II would be to consider the sum in quadrature of the two precision terms.  (ie take the square root of the sum of the squares of the count statistics terms).

[orlandin]

Hello, Nicholas!

Ah ha - thank you very much! Reporting these additional sources of uncertainties can make such a big difference that I feel I have been mis-representing the 'precision' of WDS results. I've attached an example because I can't figure out how to get a table to come together nicely in a post. If you don't feel like logging in to see it, a good example is that F goes from 9.40 rel% 1-sigma with the full accounting of uncertainty, to 2.17 with only the 'analytical uncertainty' equivalent calculation. That measurement of 3.52 wt% F was made @ 1 nA with a 30s livetime on an OI 50mm² X-Max.

The same apatite was analyzed on our JEOL 8200 @ 10 nA for 60s on-peak, giving a rel% of 1.371 (de-focused beam and TDI, of course! The EDS analysis was a ~50 micron2 area collected via Aztec).

Best,

Phil

[John Donovan]

Hi Phil,
I wouldn't say you are "mis-representing" WDS uncertainty, because it all depends on what one means by "uncertainty".  Are you including the effect of changes in barometric pressure in your WDS uncertainty calculations?       Probably not...   But I have to give Nicholas credit for trying to include many, many sources of uncertainty in his calculations!

Probe for EPMA includes two calculations for analytical uncertainty which I assume you saw are both taken from Goldstein et al. So I take no credit (or blame!) for them, but they have been found to be "useful".

The first uncertainty calculation method (for single data points only) simply ratios the P-B uncertainty (one sigma) of the standard and unknown counts rates and yes, the expression doesn't include other uncertainties, such as the matrix correction uncertainty (or barometric pressure uncertainty for that matter!).  This expression is documented here in the PFE reference manual:



However, the second uncertainty calculation in Probe for EPMA is the so called "t-test" analytical uncertainty calculation for multiple data points which is shown here:



is often considered a more "predictive" calculation because it depends not only on the number of data points acquired, but also includes the *measured* variance of the sample (which would therefore include uncertainties such as spectrometer reproducibility). So in theory this t-test method does provide some representation for other varying experimental parameters (though not necessarily variations in barometric pressure!).   

As for estimating the "uncertainty" of the various matrix corrections, I usually utilize the Use All Matrix Corrections checkbox in the Analyze! window of Probe for EPMA. This then calculates all 10 matrix corrections and reports the average and variance of these physical models as seen here:



This accuracy uncertainty is certainly not the only way to characterize the matrix correction uncertainty, but it's one option. In any case, I will defer to Nicholas on these topics as he has thought a lot more on these questions than I have.

[orlandin]

Hello, John!

Ah ha, thank you for this! I had no idea that changes in barometric pressure could increase single-line uncertainty by up to five times like the MAC uncertainty apparently can! I guess probing in a thunderstorm is a bad idea for several reasons, really. I agree that both methods of calculation are very useful, and I am grateful to both of you for putting so much time and effort into calculating them on my behalf and now explaining them as well. Emulating PFE-quality analyses with SEM-EDS and DTSA-II has taught me more about how microprobe analyses work than anything since taking Julien's microanalysis course. And as an aside, boy have i learned to appreciate the many small quality of life features

I really, really like the t-test capability of PFE, but I find that researchers tend to obsess over single-line uncertainties. It is also one of the statistics that people reach for to beat me over the head with about why quantifying EDS is obviously such a waste of time and the probe is the best solution to all problems. Both of your help in sorting out that the reported 'uncertainty' in DTSA-II is larger than in PFE for comparable analyses is much more due to a more comprehensive treatment of error rather than a fundamental deficiency of the technique is going to be valuable in building trust in DTSA-II's results. I need to learn to use the tabulate command that Nicholas described in an earlier post and put together a summary that I will post to the forum.

Nicholas, is DTSA-II doing what John is describing with the variance of all published matrix corrections when it reports the mu/rho term, or is it more based on what you discuss in Ritchie 2020 and 2021 in 'Embracing Uncertainty: Modeling the Standard Uncertainty in Electron Probe Microanalysis' parts I and II (but mostly part II) with the Chantler MAC datasets?

Best,

Phil

[John Donovan]

Ah ha, thank you for this! I had no idea that changes in barometric pressure could increase single-line uncertainty by up to five times like the MAC uncertainty apparently can! I guess probing in a thunderstorm is a bad idea for several reasons, really.

MAC uncertainty is another parameter that I did not address, but in PFE (and CalcZAF) one can model using up to 6 different MAC table values (not to mention tabulations of empirical MAC determinations) to get an idea of the MAC uncertainty. However, be aware that especially for low energy emitters one sometimes sees extrapolations across absorption edges which never gives a useful result!   

And I hope you do realize that I was making a funny about barometric pressure. But I guess it all depends on the time scale, or the particular super storm hurricane eye you happen to be entering!      Don't worry, someday we'll get solid state detectors for our WDS spectrometers...

I really, really like the t-test capability of PFE, but I find that researchers tend to obsess over single-line uncertainties. It is also one of the statistics that people reach for to beat me over the head with about why quantifying EDS is obviously such a waste of time and the probe is the best solution to all problems.

I think you will find that it depends on the problem. Accuracy of major elements in simple matrices is quite similar for both WDS and EDS as many reports have shown. That is *if* the analyst treats both measurement types with the same care. Unfortunately most EDS users just punch the button for standardless analysis and so WDS often wins on accuracy in these (99%?) cases. So maybe it's a good thing that the WDS response is so non-linear that it generally requires the use of standards!

On sensitivity, because the P/B of WDS is often much better than for EDS, people often turn to WDS for trace elements for good reason.  But really I think it boils down to the care that the analyst takes in making their measurements whether the method be WDS or EDS.

In my mind, the main advantage of EDS (for many labs are aren't well controlled in temperature) is that EDS tends to be more stable over time because the WDS is much more dependent on mechanical stability.

As is often the case in science, the answer is "it depends".

[Nicholas Ritchie]

It is worth remembering there are two kinds of uncertainty.
Type A: The kind that is improved by making more or longer measurements - can be thought of like `precision` or `reproducibility`. 
  - This is what count statistics calculations and t-tests can illuminate
Type B: The kind that results from inaccuracies in the measurement model (ie matrix correction) - can be thought of like`accuracy`.
  - This is harder to understand as it can only be evaluated by comparing measurements with the correct value established through other means.  You can estimate a range of values into which the true value is likely to fall but no amount of repeating measurements will help (although you could imagine ways to use multi-keV measurements to estimate the influence of the MAC since MACs become less significant at lower overvoltages.)

As are learning from studies of the transition metal L-lines,  the concept that we can always compute material MACs from elemental MACs is flawed - a model failure.  File under: "All models are wrong, some models are useful."  It is a very useful model for almost all K-lines and many L- and M-lines but not all. This means that estimates of the uncertainty in a material MAC must go beyond simply asking how elemental MACs vary between tabulations.  We need to develop an understanding of where material MACs are simply weighed sums and where this model fails.  This remains a subject of study.

The uncertainty calculation in DTSA-II is based on a less sophisticated model than discussed in my recent Embracing Uncertainty - Part I & II articles.  I continue to believe the DTSA-II model (from a 2012 M&M article) gives answers that are basically correct and I don't expect to update to the full model any time soon.  However, the full model can compute uncertainties due to other sources like surface quality.

[John Donovan]

Nicholas is of course exactly correct.

I tend to think of uncertainty as consisting of sources of random uncertainty (which as he says can be improved by simply making further measurements), and systematic uncertainties, which require more thoughtful efforts to correct for.

The rub is that the same source can contribute to one type of uncertainty or the other, or even contribute both types of uncertainty in our measurements. For example, stage reproducibility issues can produce random uncertainties in moving back to the standard calibration position Z position. But the stage could also produce systematic uncertainties if the stage is drifting mechanically in one direction.

Hey, if it was easy, anyone could do it!