Wish List For CalcImage

[JonF]

Hi,

   Is there any way of importing the JEOL EDS maps in to CalcImage for quantification? Everything works for the point analysis, but CalcImage comes back with the following message when I try to import a spectrum image: "Only Thermo NSS and Pathfinder spectrum image files are supported for integrated WDS and EDS map quantification at this time."

It'd be really great to get the EDS working alongside everything else for quantification 

[John Donovan]

   Is there any way of importing the JEOL EDS maps in to CalcImage for quantification? Everything works for the point analysis, but CalcImage comes back with the following message when I try to import a spectrum image: "Only Thermo NSS and Pathfinder spectrum image files are supported for integrated WDS and EDS map quantification at this time."

It'd be really great to get the EDS working alongside everything else for quantification 

Hi Jon,
I wish for this also!   

Originally we've asked JEOL Japan (on the older 8230/8530 instruments) for the ability to start a synchronized WDS and EDS map, but that never happened.  And now they've moved on to their new MEC (Microscope External Control) interface with the new iSP100/iHP200F instruments.  So we could eventually implement that in the new interface, but we are currently testing the single spectrum acquisition interface for these new instruments. It's a very dense interface and minimal documentation.

But after all this time I think there are other ways forward on the spectrum map front. One way is to figure out the structure of the JEOL EDS spectrum image files, and write code to read the intensities. But that means performing the spectrum deconvolution ourselves.  And normalizing to the live time of each pixel.  That is non-trivial to say the least.

Instead, we now think the best way forward is to ask JEOL for a DLL interface call to read the JEOL acquired EDS spectrum image, and obtain the net intensity maps corrected for the live time of each pixel. There is already an MEC call to obtain spectrum map results, so we are looking at that.  It is not clear if these net intensity maps are normalized to the pixel live times. As I said, the MEC documentation is minimal.

Currently we are in communications with JEOL on this last point.  So don't hold your breath, but maybe something will happen (at least on the new iSP100/iHP200F instruments).

[sem-geologist]

But after all this time I think there are other ways forward on the spectrum map front. One way is to figure out the structure of the JEOL EDS spectrum image files, and write code to read the intensities. But that means performing the spectrum deconvolution ourselves.  And normalizing to the live time of each pixel.  That is non-trivial to say the least.

Whats so wrong in doing deconvolution ourselves (yourself)? Or converting counts to cps? I am pretty sure it is not possible to do deconvolution anyhow worse than OEM implements it or treat the raw data anyhow worse than OEM software does. That is the most bottom line where any attempt to do better will result in better. That is why we had went for such lengths in HyperSpy, where b.t.w. the reader/writter of file formats were recently split into separate python library RosettaSciIO. There are spectral map of jeol and bruker implemented (reverse engineered, but blows away the native Bruker Esprit reader in speed order of magnitude) - thus at least it can be figured out how to read those files. As for live time correction - that depends from implementation of vendor, how it tracks that. At least oxford and Bruker has this nice 0keV peak which tracks the live time, so every spectra in the map has its own encoded live time, other vendors probably contain live time raster.

[John Donovan]

But after all this time I think there are other ways forward on the spectrum map front. One way is to figure out the structure of the JEOL EDS spectrum image files, and write code to read the intensities. But that means performing the spectrum deconvolution ourselves.  And normalizing to the live time of each pixel.  That is non-trivial to say the least.

Whats so wrong in doing deconvolution ourselves (yourself)? Or converting counts to cps? I am pretty sure it is not possible to do deconvolution anyhow worse than OEM implements it or treat the raw data anyhow worse than OEM software does. That is the most bottom line where any attempt to do better will result in better. That is why we had went for such lengths in HyperSpy, where b.t.w. the reader/writter of file formats were recently split into separate python library RosettaSciIO. There are spectral map of jeol and bruker implemented (reverse engineered, but blows away the native Bruker Esprit reader in speed order of magnitude) - thus at least it can be figured out how to read those files. As for live time correction - that depends from implementation of vendor, how it tracks that. At least oxford and Bruker has this nice 0keV peak which tracks the live time, so every spectra in the map has its own encoded live time, other vendors probably contain live time raster.

Nothing is wrong with doing the deconvolution ourselves. But as I said, it's non-trivial.

There are many, many factors to take into account (e.g., detector characteristics) and it is my experience that the various EDS vendors know better than most, in how to treat their own spectra.  We actually hired a mathematical physicist several years ago, who originally claimed: "oh that should be easy", but after several months of effort, they said it was a lot harder than they thought and gave up.

That doesn't mean that it can't also be done by others, but the various EDS vendors already have much of the work done, so it makes sense to leverage that work.

[Philipp Poeml]

Hi John
And while we have you as a captive audience at the other side of the room, have you considered using shared backgrounds for complex situations for mapping? At the moment we can use either MAN or off-peak.......
Cheers
Malc.

Shared backgrounds in calcimage would be really useful, because it would minimize the number of off peak maps to be measured. In PfE I use for 6 elements on the same spec the same 2 high and 2 low backgrounds with an exponential fit (sharing the backgrounds and then multiple point backgrounds, I skip the acquisition of all the other backgrounds using the Nth point feature). In CI, I would have to acquire 12 background maps instead of only 4. I will now trick the system and take the average of the 2 high and 2 low background maps, respectively, and apply them to all 6 elements. But it seems the only way. The acquisition of this 500 x 3000 pixel map took several weeks, so I am going to use it, whatever the cost. 

[John Donovan]

The shared/multi-point bgd method code is very complex, but we will look into seeing if this is possible.

[JonF]

Shared backgrounds in calcimage would be really useful, because it would minimize the number of off peak maps to be measured. In PfE I use for 6 elements on the same spec the same 2 high and 2 low backgrounds with an exponential fit (sharing the backgrounds and then multiple point backgrounds, I skip the acquisition of all the other backgrounds using the Nth point feature). In CI, I would have to acquire 12 background maps instead of only 4. I will now trick the system and take the average of the 2 high and 2 low background maps, respectively, and apply them to all 6 elements. But it seems the only way. The acquisition of this 500 x 3000 pixel map took several weeks, so I am going to use it, whatever the cost. 

You would need to write the code, but you could open the 4 background maps (as .GRD is easiest) as a 3 dimensional array, loop through each XY pixel to fit an exponential through the four background positions (ie Z in the array), and then you could create off-peak background maps for each element at arbitrary points along the fitted curve.
All you have to do then is tell CalcImage to use the appropriate off-peak map for each element (with the off peak and exponential fit).

[Philipp Poeml]

Hi Jon, thanks, yes, that was what I was thinking about. But luckily John has implemented the background sharing in CI in the meantime... 

[John Donovan]

Hi Jon, thanks, yes, that was what I was thinking about. But luckily John has implemented the background sharing in CI in the meantime... 

Note this is only implemented for testing with our beta testers as of now.  I will let everyone else know when it's been tested sufficiently and released.

[John Donovan]

Hi John
And while we have you as a captive audience at the other side of the room, have you considered using shared backgrounds for complex situations for mapping? At the moment we can use either MAN or off-peak.......
Cheers
Malc.

Shared backgrounds in calcimage would be really useful, because it would minimize the number of off peak maps to be measured. In PfE I use for 6 elements on the same spec the same 2 high and 2 low backgrounds with an exponential fit (sharing the backgrounds and then multiple point backgrounds, I skip the acquisition of all the other backgrounds using the Nth point feature). In CI, I would have to acquire 12 background maps instead of only 4. I will now trick the system and take the average of the 2 high and 2 low background maps, respectively, and apply them to all 6 elements. But it seems the only way. The acquisition of this 500 x 3000 pixel map took several weeks, so I am going to use it, whatever the cost. 

Your wish is granted!   

This wasn't easy to implement but I think we have something that will work.

Some history first: when we first designed Probe Image (for map acquisition) and CalcImage (for quantification of maps) we knew that we would need to support both off-peak and MAN background methods, but we never imagined that anyone would ask for a multi-point background (MPB) method for mapping:

https://probesoftware.com/smf/index.php?topic=701.msg4283#msg4283

That means that in Probe Image one can decide whether to acquire both high and low off-peaks, no off-peaks (when using the MAN background method), or just one either high or low off-peak (for the high only, slope-high, low only or slope-low off-peak background fit models). 

Now to be clear there are two different versions of multi-point backgrounds. First there is the original MPB method where the user specifies multiple (two or more, usually 4 or so) background positions for each analyzed element as originally suggested by a number of people: Mike Jercinovic, Julien Allaz, Mike Williams and Philipp Poeml. Later we implemented a "shared" off-peak method as suggested by Karsten Goemann:

https://probesoftware.com/smf/index.php?topic=9.msg1579#msg1579

This "shared" off-peak method is based on two off-peak point backgrounds per element, *but* where at least two elements share the same spectrometer and crystal. The idea being to convert these multiple off-peak backgrounds into multi-point backgrounds so that they can be used by all the elements on that spectrometer/crystal.  Of course we want to avoid interpolating (or extrapolating) across absorption edges, but this method turns out to be very useful for situations where the background is curved and one doesn't want to bother acquiring MPBs in the first place.

A discussion comparing the advantages and disadvantages of these various background methods is here:

https://probesoftware.com/smf/index.php?topic=1378.msg9999#msg9999

However, since the MPB method is really a trace element method, we never imagined that we would get requests for an MPB mapping method in CalcImage.  In any case, we decided to not implement the option to acquire multi-point background maps in Probe Image and that decision seems to have worked out well for the purposes of this new capability in CalcImage in order to quantify X-ray maps using (shared) multi-point backgrounds, because it does make sense for the "shared" MPB method, since one merely utilizes the off-peak map acquisition options selected in Probe Image.

Starting with v. 13.8.5 of the latest Probe for EPMA software release one can now acquire off-peak maps in Probe Image, and if the "shared" background method is specified in Probe for EPMA for the sample basis in CalcImage (by using the Search for Shared Backgrounds button in the Analyze! window as shown here):



one can apply a (shared) MPB method to their x-ray map quantifications!  How cool is that?   

Please note that there is one important difference between the MPB method in Probe for EPMA and how the MPB method is applied in CalcImage: in Probe for EPMA, when the user specifies the fitting parameters that the (shared) MPB off-peak method utilizes (by using the Iterate To parameter or the manual "Always Use" or "Never Use" manual flags), one can be assured that a non-zero intensity will be found because Probe for EPMA always acquires all off-peak points that are specified for the sample.

However, because in Probe Image one can specify to either acquire both off-peak maps, one off-peak map or no off-peak maps, CalcImage cannot be assured that all off-peak maps are available.

Therefore CalcImage will, if it cannot find a specified off-peak map, set the manual flag for that point (actually pixel), as "Never Use", so a zero intensity will not be applied in the routine for obtaining the MPB regression.

[Philipp Poeml]

Thanks for implementing this! I know it is a bit of a very special feature for special use cases, but at least I was not the only one asking for it for a while! 

In irradiated nuclear fuel we have so many elements to measure that even a setup of 24 elements is still a selection of the most important ones! There are just too many of them and they all behave in a different way during irradiation.

For example on my SP2 QTZ, I am measuring U, Np, Pu, am, Cm, and Xe (Xe is there bcause it fits nowhere else and I had to fill up the channel to 6 elements as well). Those funny actinides are all overlapping each other and it is quite impossible to find any background positions. MAN could be an option but nobody has yet come up with a good method because we do not have standards for Am, Cm, Bk, Cf, Es, ....

What I do is I get two backgrounds to the far left and two backgrounds to the far right and do an exponential fit, I think this is very close to as good as it can get. These elements are all no trace, we are talking about 1 wt % and above up to 20 wt %. So using this MPB approach in PI + CI as well (am using it in PfE for the point analysis since a while) saves me quite some mapping, because I need to do only 4 background maps instead of 12. I can even go up with the dwelling time for those 4 maps a bit to "improve" the statistics a bit.

In any case: Great work! Thanks again.

[John Donovan]

What I do is I get two backgrounds to the far left and two backgrounds to the far right and do an exponential fit, I think this is very close to as good as it can get. These elements are all no trace, we are talking about 1 wt % and above up to 20 wt %. So using this MPB approach in PI + CI as well (am using it in PfE for the point analysis since a while) saves me quite some mapping, because I need to do only 4 background maps instead of 12. I can even go up with the dwelling time for those 4 maps a bit to "improve" the statistics a bit.

This is a really interesting idea to acquire only some off-peak maps and then use the MPB shared method to get backgrounds for all the map elements.

That brings up another idea which is that one could also only acquire a high or low background for the map elements and then use this shared MPB method to still get at least a linear background fit on all the elements...