Probe Software Users Forum

Software => CalcImage => Topic started by: Malcolm Roberts on April 30, 2014, 07:50:59 PM

Title: Wish List For CalcImage
Post by: Malcolm Roberts on April 30, 2014, 07:50:59 PM
Hi John
One of my wishes is to be able of change some of the defaults in the output surfer script, e.g., I would like one map per page as a default and a wait time of only 1 second. At the moment I do a bit of editing and by the time I have done this, I might as well have run the default 8 secs. Any hot tips?
Cheers,
malc.
Title: Re: Wish List For CalcImage
Post by: John Donovan on April 30, 2014, 08:00:24 PM
One of my wishes is to be able of change some of the defaults in the output surfer script, e.g., I would like one map per page as a default and a wait time of only 1 second. At the moment I do a bit of editing and by the time I have done this, I might as well have run the default 8 secs. Any hot tips?

Yes!

Just edit this keyword in the Probewin.ini file:

[software]
SurferPlotsPerPage=1
Title: Re: Wish List For CalcImage
Post by: Malcolm Roberts on February 13, 2015, 05:30:58 PM
Hi John
And while we have you as a captive audience at the other side of the room, have you considered using shared backgrounds for complex situations for mapping? At the moment we can use either MAN or off-peak.......
Cheers
Malc.
Title: Re: Wish List For CalcImage
Post by: John Donovan on February 14, 2015, 04:22:35 AM
Hi Malcolm,
I doubt it would be worth it due to the reduced precision for x-ray mapping. 

That is also to say, you'd have to acquire multiple off peak maps. It's an interesting idea since the multi-point bgd could handle any variation in composition automatically (that's one of its best aspects).  But although it could be done, would it be worth it?

For one thing, I doubt it would be an improvement given the limited precision of a say, 200 msec per pixel acquisition.  Instead I'll just mention here that if I wanted to do accurate mapping of REE minerals I would probably (surely) use MAN backgrounds.  I don't think one could do better for the time invested.
Title: Re: Wish List For CalcImage
Post by: Malcolm Roberts on February 14, 2015, 05:18:28 PM
Hi John
I am glad you said that. I am in complete agreement with your comments on MAN and mapping. In fact, even for regular REE measurement in such things as monazite, I would be tempted to use MAN and resort to off peaks for dealing with things like U, Th and Pb. I have yet to try this out, but will give it a go for mapping runs, where age maps can be generated? Watch this space.....
Cheers
Malc.
Title: Re: Wish List For CalcImage
Post by: John Donovan on February 15, 2015, 01:33:22 PM
I am glad you said that. I am in complete agreement with your comments on MAN and mapping. In fact, even for regular REE measurement in such things as monazite, I would be tempted to use MAN and resort to off peaks for dealing with things like U, Th and Pb. I have yet to try this out, but will give it a go for mapping runs, where age maps can be generated?

Hi Malcolm,
I'm not convinced that one can obtain sufficient precision in mapping mode for useful monazite age measurements, but yes, this can certainly be done in CalcImage as seen here:

http://probesoftware.com/smf/index.php?topic=100.0

Instead I would probably utilize the rectangular grid or polygon grid method in Probe for EPMA to acquire an array of single point analyses with sufficient precision and accuracy and then output those to Surfer for presentation similar to how these maps were created:

http://probesoftware.com/smf/index.php?topic=60.msg227#msg227

Title: Re: Wish List For CalcImage
Post by: Malcolm Roberts on February 15, 2015, 04:12:51 PM
Hi John
That makes abundant sense/ While, I'm on a roll here and not being mysteriously logged out all the time, any tips on how to change that default 8 secs to 1 sec permanently? See first post........
Also, I briefly chatted with Paul C about this plotsperpage thing in probewin.ini. To save time, I usually spit out the detection limit maps at 4 plots per page, then do the quant maps at 1 plot per page. This means that I have to modify the script for either 1 or 4 depending on what I am doing. Would it be possible to set default scenarios for each plot type? e.g., 1 plot per page for quant maps and 4 for detection limits?
Cheers,
Malc.
Title: Re: Wish List For CalcImage
Post by: John Donovan on February 16, 2015, 02:51:08 AM
That makes abundant sense/ While, I'm on a roll here and not being mysteriously logged out all the time, any tips on how to change that default 8 secs to 1 sec permanently? See first post........
Also, I briefly chatted with Paul C about this plotsperpage thing in probewin.ini. To save time, I usually spit out the detection limit maps at 4 plots per page, then do the quant maps at 1 plot per page. This means that I have to modify the script for either 1 or 4 depending on what I am doing. Would it be possible to set default scenarios for each plot type? e.g., 1 plot per page for quant maps and 4 for detection limits?

Hi Malcolm,
These are good ideas and I will implement them as soon as I get back.  I'll probably just add some "checkable" menus items for the plots per page and the Scripter page delay parameters.

Thanks and nice meeting you in person!
john
Title: Re: Wish List For CalcImage
Post by: Probeman on February 18, 2015, 05:33:54 PM
That makes abundant sense/ While, I'm on a roll here and not being mysteriously logged out all the time, any tips on how to change that default 8 secs to 1 sec permanently? See first post........
Also, I briefly chatted with Paul C about this plotsperpage thing in probewin.ini. To save time, I usually spit out the detection limit maps at 4 plots per page, then do the quant maps at 1 plot per page. This means that I have to modify the script for either 1 or 4 depending on what I am doing. Would it be possible to set default scenarios for each plot type? e.g., 1 plot per page for quant maps and 4 for detection limits?

Hi Malcolm,
These are good ideas and I will implement them as soon as I get back.  I'll probably just add some "checkable" menus items for the plots per page and the Scripter page delay parameters.

Thanks and nice meeting you in person!
john

Hi Malcolm,
So awesome to meet you finally in Hobart.

I've implemented your ideas in the latest CalcImage release.  The "plots per page" script keyword (for both normal output and also polygon output) is ready to download in v. 10.7.1. Below are some screen shots:

(http://probesoftware.com/smf/oldpics/i57.tinypic.com/2s16d0i.jpg)

(http://probesoftware.com/smf/oldpics/i57.tinypic.com/9fz8eo.jpg)

(http://probesoftware.com/smf/oldpics/i60.tinypic.com/126g3y8.jpg)
Title: Re: Wish List For CalcImage
Post by: Malcolm Roberts on February 18, 2015, 06:30:54 PM
Excellent........
Many thanks John. I'll give it a go......
Cheers,
Malc.
Title: Re: Wish List For CalcImage
Post by: Ben Buse on February 26, 2015, 11:05:38 AM
How do people manage there files? Would it be useful to be able to select the folder in which the processed maps are saved into?

What we do at the moment is collect quant point data in MDB file, then run a map - with raw maps in a subfile. I like to keep my maps and quant points seperate. Then when I run the calczaf quant map I call up the recipe from the MDB file and it outputs the processed maps to the MDB location rather than the raw map location.

Thanks

Ben
Title: Re: Wish List For CalcImage
Post by: John Donovan on February 26, 2015, 11:36:04 AM
How do people manage there files? Would it be useful to be able to select the folder in which the processed maps are saved into?

What we do at the moment is collect quant point data in MDB file, then run a map - with raw maps in a subfile. I like to keep my maps and quant points seperate. Then when I run the calczaf quant map I call up the recipe from the MDB file and it outputs the processed maps to the MDB location rather than the raw map location.

Hi Ben,
You could modify one of the "custom" output scripts and set the output folder location in Scripter automatically at run time. See here:

http://probesoftware.com/smf/index.php?topic=55.0

and also check this recent script issue that you should be aware of:

http://probesoftware.com/smf/index.php?topic=55.msg2352#msg2352

john
Title: Re: Wish List For CalcImage
Post by: Philipp Poeml on March 08, 2016, 08:13:17 AM
Hi John,

right now I make myself a nice custom xy script for the surfer output. What could be cool, in calcimage, under sufer scripts, like a text box where we could set some variables that would be included in the script template, i.e. handed over to scripter.
For example I could set
SetXYorigintoZero%=1 or
SetXYorigintoZero%=0
then I could activate this subroutine in the script from within calcimage. I could use then the same script template for slightly different purposes.
Or I could add
UseColorscale$ = "blacktored" or something.

Stupid idea?

Cheers
Philipp
Title: Re: Wish List For CalcImage
Post by: Probeman on March 08, 2016, 09:59:45 AM
Right now I make myself a nice custom xy script for the surfer output. What could be cool, in calcimage, under sufer scripts, like a text box where we could set some variables that would be included in the script template, i.e. handed over to scripter.
For example I could set
SetXYorigintoZero%=1 or
SetXYorigintoZero%=0
then I could activate this subroutine in the script from within calcimage. I could use then the same script template for slightly different purposes.
Or I could add
UseColorscale$ = "blacktored" or something.

Stupid idea?

It's a good idea.  But let me ask, is your script a modification of one of the existing Surfer scripts?  That is, are you already utilizing one of the "custom" script menu positions in CalcImage to call your modified scripts? 

That is, are you saying you only need a text box in CalcImage where you can enter some additional script parameters as described above, that are automatically added to the custom script output section?  I assume we'll have to declare them as globals so your modified script can find them. Can you post some script code with the parameters you want to use?

Also, do you need these custom parameters saved along with the project?  I suppose it would be nice?   :)
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on November 15, 2016, 09:41:50 AM
Hi, would it be possible to add raw count maps to the traverse (slice) function output function in CalcImage? Currently it has only the processed maps as options for slice/polygon/multiple strip. 

Thanks!
Title: Re: Wish List For CalcImage
Post by: Probeman on November 15, 2016, 11:33:28 AM
Hi, would it be possible to add raw count maps to the traverse (slice) function output function in CalcImage? Currently it has only the processed maps as options for slice/polygon/multiple strip. 

It is possible.  Can you wait until later this week?
john
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on November 15, 2016, 12:31:05 PM
Sure. It is not that urgent, I just was showing CalcImage to a user and realized that doing traverse on his raw maps would not be possible (I would just frankenstein an mdb together if it was urgent).

Thanks in advance.
Title: Re: Wish List For CalcImage
Post by: John Donovan on November 16, 2016, 10:08:32 PM
Sure. It is not that urgent, I just was showing CalcImage to a user and realized that doing traverse on his raw maps would not be possible (I would just frankenstein an mdb together if it was urgent).

Hi Anette,
I took a look at your request to add slice, polygon and strip output of the raw data maps to CalcImage tonight, and I found that there is an issue: I cannot add any more menu items to the CalcImage app! It's a limitation of the compiler! Now of course I could re-write things to use a dialog and checkboxes for presentation output, but that will take a considerable amount of work... so in the meantime time here is an alternative that is not so "frankenstein".  ;)

First of all, yes there is currently no slice, polygon or strip output of the raw intensity maps (just for the quant, oxide, atomic, detection limits, etc. maps), but there is slice, polygon and strip output for net intensity maps... maybe that is good enough?  Why the heck do you want to look at raw data anyway!   :o

But here is my less frankenstein suggestion: take your quant slice script, for example open any .bas script output by CalcImage and here is what you will see near the top of the file:

Dim iMax As Integer
iMax% = 6

ReDim FileArray(1 to iMax%) as String
FileArray$(1) = "Mg diffusion-2_00491_SP1_O_PC1__Quant"
FileArray$(2) = "Mg diffusion-2_00491_SP2_Al_LTAP__Quant"
FileArray$(3) = "Mg diffusion-2_00491_SP3_Gd_LLIF__Quant"
FileArray$(4) = "Mg diffusion-2_00491_SP4_Mg_TAP__Quant"
FileArray$(5) = "Mg diffusion-2_00491_SP5_Sn_PET__Quant"
FileArray$(6) = "Mg,Al,Gd,Sn,O_09-13-2012_Mg diffusion-2_00491__Total_Quant"

ReDim ZLabel(1 to iMax%) as String

ZLabel$(1) = "O Wt%"
ZLabel$(2) = "Al Wt%"
ZLabel$(3) = "Gd Wt%"
ZLabel$(4) = "Mg Wt%"
ZLabel$(5) = "Sn Wt%"
ZLabel$(6) = "Total Wt%"

All you need to do is edit the script, like this:

Dim iMax As Integer
iMax% = 5

ReDim FileArray(1 to iMax%) as String
FileArray$(1) = "Mg diffusion-2_00491_SP1_O_PC1_"
FileArray$(2) = "Mg diffusion-2_00491_SP2_Al_LTAP_"
FileArray$(3) = "Mg diffusion-2_00491_SP3_Gd_LLIF_"
FileArray$(4) = "Mg diffusion-2_00491_SP4_Mg_TAP_"
FileArray$(5) = "Mg diffusion-2_00491_SP5_Sn_PET_"

ReDim ZLabel(1 to iMax%) as String

ZLabel$(1) = "O cps/nA"
ZLabel$(2) = "Al cps/nA"
ZLabel$(3) = "Gd cps/nA"
ZLabel$(4) = "Mg cps/nA"
ZLabel$(5) = "Sn cps/nA"

Note that I changed the iMax parameter from 6 to 5 because there is no "Total" map for the raw intensities!  I also edited the file names for the raw intensity files (just an underscore after the crystal name), and the labels from Wt% to cps/nA.

When this edited script is run in the Surfer Scripter app, this is what you get (see attached images below).  Hopefully this is a reasonable method for you.
john
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on November 17, 2016, 07:12:53 AM
Great! I give this a try.

Thanks!
Anette
Title: Re: Wish List For CalcImage
Post by: Probeman on November 17, 2016, 08:41:20 AM
Great! I give this a try.

Thanks!
Anette

It occurs to me just now, that if one wants to perform slice/polygon/strip output of raw intensity maps, an even easier edit, is to utilize the existing net intensity slice/polygon/strip output script.  Rename the output script .bas file to something else, e.g., Mg,Al,Gd,Sn,O_09-13-2012_Mg diffusion-2_00491__Raw_Slice.bas, then just edit the grd filenames since the net intensity script won't have the total file to deal with and the z data labels are *already* "cps/nA"...
john
Title: Re: Wish List For CalcImage
Post by: Dan Ruscitto on February 27, 2017, 06:18:19 AM
Two requests (I am currently using 11.7.3):

1. For maps containing multiple WDS passes, please add an option to align map passes using BSE or SE images BEFORE doing mapping calculations. This would result in cutting off some of the pixels, but it should greatly help reduce artifacts around phase boundaries. For example, carbides surrounded by a metal matrix...

2. During CalcImage processing, please allow the user to specify a different output directory from the MDB database file used for quant/backgrounds. This would make file management easier and save disk space... currently if I want to have a separate folder for each quant map, I need to copy in the original MDB data base file.

Thanks!
Dan
Title: Re: Wish List For CalcImage
Post by: John Donovan on February 27, 2017, 12:56:43 PM
Two requests (I am currently using 11.7.3):
Hi Dan,
Please update to v. 11.8.3!   Lots of cool new stuff and improved behaviors:

http://probesoftware.com/smf/index.php?topic=40.0

1. For maps containing multiple WDS passes, please add an option to align map passes using BSE or SE images BEFORE doing mapping calculations. This would result in cutting off some of the pixels, but it should greatly help reduce artifacts around phase boundaries. For example, carbides surrounded by a metal matrix...

This is a Probe Image request I think?

If I understand what you are asking for, the good news is that Brian and I working with Paul Carpenter and Gareth Seward have a solution coming soon.  I will send you our spec document we are working on if you like.

2. During CalcImage processing, please allow the user to specify a different output directory from the MDB database file used for quant/backgrounds. This would make file management easier and save disk space... currently if I want to have a separate folder for each quant map, I need to copy in the original MDB data base file.

Are you asking that instead of using Windows Explorer to copy the necessary files for different output types to separate sub folders, you'd like something along the lines of a "clone" menu in CalcImage to make a sub folder and copy the required files for additional/subsequent output of images and maps?
john
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 02, 2017, 06:21:11 PM

1. For maps containing multiple WDS passes, please add an option to align map passes using BSE or SE images BEFORE doing mapping calculations. This would result in cutting off some of the pixels, but it should greatly help reduce artifacts around phase boundaries. For example, carbides surrounded by a metal matrix...

2. During CalcImage processing, please allow the user to specify a different output directory from the MDB database file used for quant/backgrounds. This would make file management easier and save disk space... currently if I want to have a separate folder for each quant map, I need to copy in the original MDB data base file.

Hi Dan,
If you update PFE and open the CalcImage Log Window (from the Window menu), you will see this new menu for cloning the current project to another folder. Say for performing multiple polygon extractions from the same map.

(http://probesoftware.com/smf/gallery/1_02_03_17_6_11_37.png)

The only thing I should mention about this feature is that in the Browse folder dialog, first right click where you want the new folder to be, then click the New | Folder sub menu and name the folder, select it and click OK.  Or just select an existing folder.

I'm still thinking about your idea for a visual align maps and crop feature...  it's a cool idea but want to think about it a bit.
john
Title: Re: Wish List For CalcImage
Post by: John Donovan on June 01, 2017, 12:37:49 PM
Hi Dan,
...
I'm still thinking about your idea for a visual align maps and crop feature...  it's a cool idea but want to think about it a bit.
john

In case any one missed it, the Align and Crop feature for aligning multiple spectrometer passes for stage or sample drift is described here:

https://probesoftware.com/smf/index.php?topic=1153.0

It's pretty cool if I do say so myself!    :D
john
Title: Re: Wish List For CalcImage
Post by: John Donovan on June 24, 2017, 08:57:57 AM
Dave Wark recently asked if I could modify the analytical sensitivity calculation for x-ray maps in CalcImage.  This analytical sensitivity is from Goldstein et al. as described here:

http://probesoftware.com/smf/index.php?topic=93.msg341#msg341

Originally during the saving of the map GRD files I would "filter" the analytical sensitivity calculation so that pixels that represent less than 1 wt.% concentrations would be "blanked" so they would be ignored when plotting them in the Surfer app as shown here:

(http://probesoftware.com/smf/gallery/1_24_06_17_8_45_20.jpeg)

The reason for this is simple: the analytical sensitivity values (in relative percent) start to "blow up" as the concentrations approach zero, making visualization of the data difficult. If one did not blank these pixels, one would obtain this result when the data is plotted in Surfer:

(http://probesoftware.com/smf/gallery/1_24_06_17_8_46_02.jpeg)

Notice that many pixels (those less than 1 % concentrations) show analytical sensitivity values over +/- 10^6 percent!  But apparently Dave has some calculations he wants to utilize these values for, so I added some new checkboxes in the quantitative parameters dialog as shown here:

(http://probesoftware.com/smf/gallery/1_24_06_17_8_26_23.png)

The default (unchecked) is to blank pixels which represent concentrations under 1 wt.%. This results in the original behavior shown in the first plot above. But if you check the box as shown here:

(http://probesoftware.com/smf/gallery/1_24_06_17_8_25_38.png)

You get the analytical sensitivity data unfiltered for calculation purposes. We'll discuss the effect of the additional "skip blanking" flag on detection limit calculations next.
Title: Re: Wish List For CalcImage
Post by: John Donovan on June 24, 2017, 09:06:15 AM
For the detection limit calculation I originally did not blank any pixels, but I wanted the calculation to be symmetrical with the analytical sensitivity calculation, and besides I realized there is a benefit for blanking the detection limit calculation pixels also.

That is, we really don't care about the detection limits for pixels whose concentrations are very large, do we? What we care about for detection limits are low concentration pixels. So I modified the code to blank pixels (by default) if the pixel concentrations are larger than 10 wt.% as seen here:

(http://probesoftware.com/smf/gallery/1_24_06_17_8_45_04.jpeg)

The value of this method is that the eye can know instantly if the pixel is a low concentration or not for that element, so we can know which areas of the map to focus our attention on.  And of course if one really wants to see the detection limits for all pixels, including high concentration pixels, one can simply check the "skip blanking" flag and get this output for all pixels:

(http://probesoftware.com/smf/gallery/1_24_06_17_8_45_41.jpeg)

Please let me know what you all think.
john
Title: Re: Wish List For CalcImage
Post by: John Donovan on September 12, 2018, 10:11:17 AM
Andrew Mott at Texas A&M noticed recently that although CalcImage was correctly calculating and outputting a specified element by difference for the elemental weight percent and atomic percent .DAT files, the program was only outputting quant x-ray maps for elemental weight percents for the element by difference, but not quant x-ray maps for atomic percent element by difference.

And sure enough when we looked in the code and it's wasn't exporting the .GRD file properly. So we modified the code and now it properly outputs .GRD files for export to Surfer for elemental, atomic and oxide weight percents for the element by difference as seen here for atomic percents:

(https://probesoftware.com/smf/gallery/1_12_09_18_8_54_21.jpeg)

This modified code now also outputs maps for the totals, oxygen by stoichiometry, excess oxygen and mentioned already, the element specified by difference.

However, when exporting the maps to Surfer, the code automatically suppresses the total concentrations maps for atomic and element by difference output since those maps are always going to be pixels all at 100%.

One caveat: because the new code now outputs separately named .GRD files for the totals, oxygen by stoichiometry, excess oxygen and the element specified by difference elements, for both the atomic and oxide weight percent output, if you have previously calculated runs containing a totals and/or oxygen by stoichiometry quant maps and you are using atomic or oxide output, you will need to re-calculate the quant to create the newly named atomic and oxide totals and/or oxygen by stoichiometry quant maps.  That is if you want to re-export the output to Surfer. 

Sorry for any trouble this causes anyone.  Of course, new quant map calculations in CalcImage will proceed automatically, as will any previously calculated map outputs that just used elemental weight percent output.

We think (hope) this is good for everyone but let us know what you think.
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on March 03, 2020, 04:21:28 PM
Hi John,

at one point, CalcImage allowed map math/RGB for the raw intensity maps (or any map that was open in the window). After an update, it now requires the classification file which I only get when quantifying the maps (afaik). Would it be possible to implement the old option in CalcImage again (in addition to the current function that I like too)? It was quite useful at times for initial ad-hoc processing.
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 04, 2020, 11:21:07 AM
Hi John,

at one point, CalcImage allowed map math/RGB for the raw intensity maps (or any map that was open in the window). After an update, it now requires the classification file which I only get when quantifying the maps (afaik). Would it be possible to implement the old option in CalcImage again (in addition to the current function that I like too)? It was quite useful at times for initial ad-hoc processing.

Yeah, there were a lot of problems with the old method of just using various images already loaded into the CalcImage main window.    By using the .DAT file output, there is a lot more flexibility for quant calculations, but as you say, one needs to quant the maps first:

https://probesoftware.com/smf/index.php?topic=519.msg8023#msg8023

If you only want to RGB some random images, you can easily utilize ImageJ and probably more than a few other commercial apps.
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on March 04, 2020, 02:33:49 PM
It was more the "Map math" that I am missing, for combining multiple spectrometers or doing quick ratio-ing of some elements.
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 04, 2020, 05:09:50 PM
It was more the "Map math" that I am missing, for combining multiple spectrometers or doing quick ratio-ing of some elements.

Ah, I see.

Well, if you already have your data in GRD files, you can use the Grid | Math menu in Surfer to perform arbitrary math operations on any number of files.

In fact, not only do you get many more mathematical operators in Surfer, but you can also write scripts to automate the image processing if you want.

But now that I think about it, I'm sure one can also perform all sorts of math operations on image data in ImageJ.
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on March 05, 2020, 10:33:02 AM
Good point. I need to get more proficient in Surfer really.
Title: Re: Wish List For CalcImage
Post by: JonF on March 09, 2020, 03:31:58 AM
We've been playing around with the quant mapping features quite a lot here and they're going down a treat. The increase in analysis time is a fair trade off to be able to post-process and discriminate the data set as opposed to the single point, fire away and hope for the best approach of old.

The align maps and TDI map features are both really good - but I'm a bit confused over the order they should be used in cases where both a slight realignment of a mapped area with volatile elements is required.

If we align the maps first (the correct way?), we're required to start up a project and import all .prbImg files to be converted to .GRD. We can then align these .GRD files, but then I can't use the TDI feature as this requires the original .prbImg (which haven't been realigned?).

If we use the TDI correction first on the raw .prbImg files, this creates the e.g. Na_TAP_TDI_1, _2 etc .prbImg files and renames them all to the same run number. I then import them in to CalcImage, where .GRD files of all the .prbImg files are created, including an extra .GRD file (e.g. Na_TAP_.grd) which is the map listed in the Image File for (e.g.) Na in the Specify Quantitative Parameters! menu. If I run the alignment procedure now, is this .grd file regenerated with the new pixel alignments? 
I could rename all the _TDI_x.grd files back to their original run numbers, run the alignment feature, and then rename them back to the first run number, which would (hopefully) make all the _TDI_x.grd files aligned, but would the created _Na_TAP_.grd file be correct (or is it even used when TDI is enabled?). This all has my head spinning!


Would it be possible to alter the align feature to work with the .prbImg files instead of/as well as the .GRD files? 
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 09, 2020, 08:39:07 AM
Hi Jon,
You're making my head spin!   :-[    I'm not quite sure how the alignment cropping would work on a set of TDI replicate files. 

I think you want to do the replicate renaming first on the PrbImg files, then do the alignment procedure on the GRD files.   But I don't have a set of suitable files to test this on.  Maybe you could put together a test set of PrbImg files for me to try this on?

But it's worth mentioning that the conversion of replicate PrbImg files to TDI GRD in CalcImage is intended to be a temporary thing, and that the next Probe Image version will have a new "replicate" acquisition feature that will automatically create the _TDI_1, etc. file names during the image acquisition.
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 09, 2020, 02:52:53 PM
If we use the TDI correction first on the raw .prbImg files, this creates the e.g. Na_TAP_TDI_1, _2 etc .prbImg files and renames them all to the same run number. I then import them in to CalcImage, where .GRD files of all the .prbImg files are created, including an extra .GRD file (e.g. Na_TAP_.grd) which is the map listed in the Image File for (e.g.) Na in the Specify Quantitative Parameters! menu. If I run the alignment procedure now, is this .grd file regenerated with the new pixel alignments? 
I could rename all the _TDI_x.grd files back to their original run numbers, run the alignment feature, and then rename them back to the first run number, which would (hopefully) make all the _TDI_x.grd files aligned, but would the created _Na_TAP_.grd file be correct (or is it even used when TDI is enabled?). This all has my head spinning!

Hi Jon,
So I took a quick look at the code and the extra (non-TDI named) x-ray map (e.g., _Na_TAP_.grd) is simply the *sum* of all the replicate measurements.  Why does this file get created?  It's for when one turns off the TDI correction in the Standard Assignments dialog), and then the program ignores the _TDI maps and calculates the map quantification based only on the summed maps (without the TDI extrapolation to zero time).

Would it be possible to alter the align feature to work with the .prbImg files instead of/as well as the .GRD files?

That is a possibility, but can you explain in more detail what exactly your setup is?  For instance, are you trying to perform an align and crop on the replicate images (because there was stage drift between the replicate acquisitions), or are you performing an align and crop on additional maps for another elements (because you needed to acquire more than 5 elements for your map quantification)?

Maybe we should discuss this by phone or Skype because it gets complicated real quick!
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 10, 2020, 12:44:48 PM
Hi Jon,
OK, I've slept on it and I think I see what the problem is.

So normally when one wants to perform the align and crop method on multiple passes on more than a single set of x-ray maps, it's because one acquired more elements than they have WDS spectrometers, as originally described in this example from a few years ago:

https://probesoftware.com/smf/index.php?topic=1152.0

So if one needed to acquire their x-ray maps in more than a single pass, e.g., Si, Fe, Ca, Al, Mg, Na in the first x-ray map acquisition, and then say, K, Mn, Ti, Ba and Ni in the second x-ray map acquisition. And if the area of interest was very small and/or the stage was drifting, the first map acquisition might not be aligned perfectly with the second map acquisition.

Of course this example assumes that one is utilizing MAN background corrections for the background correction, so no off-peak maps need to be acquired! However whether or not off-peak background maps were acquired, the maps from the first pass are named with one acquisition number (including any off-peak maps), while the maps from the second pass are named with a subsequent acquisition number (including any off-peak maps) as seen here:

(https://probesoftware.com/smf/gallery/1_10_03_20_11_46_20.png)

Where acquisition 00654 was the first pass elements and 00655 was the second pass elements (all MAN backgrounds).

Now in this "normal" align and crop case, one acquires the multiple pass x-ray map acquisitions, then one imports the PrbImg files into CalcImage using the Create New Project wizard menu, and then one has a set of GRD files with the same "base names" as with the original PrbImg files, which one can then (if necessary), apply the align and crop method to these (now GRD formatted) files, prior to the map quantification.

As a quick aside, any off-peak maps acquired will have the same acquisition number as the on-peak maps, and so would be included in the align and crop of the respective on-peak map acquisitions, so there is no way to apply an align and crop between the on and off-peak map acquisitions. Just one more reason to utilize MAN background corrections for your x-ray map quantification I guess!    :)

So far, so good.

Now, in the case of TDI scanning, where one acquires multiple on-peak map acquisitions to account for ion migration and/or beam damage effects on beam sensitive sample, each "replicate" (TDI) acquisition receives a sequential acquisition number, because in the current version of Probe Image, to acquire a set of maps for TDI correction, one acquires multiple on-peak maps each with its own acquisition number as described here:

https://probesoftware.com/smf/index.php?topic=912.0

Now as you have described, currently these separate replicate (on-peak) PrbImg files are first converted to _TDI replicate files to all have the same acquisition number, using the Convert | Convert Replicate PrbImg Files to TDI files menu in CalcImage. Now these replicate PrbImg files have the same acquisition number, but are numbered _TDI_1, _TDI_2, _TDI_3, etc. as seen here:

(https://probesoftware.com/smf/gallery/1_10_03_20_12_38_07.png)

Then one can import these renamed PrbImg files into a new CalcImage project for subsequent map quantification, just as one would any other map acquisition and the _TDI files are automatically handled for TDI scanning corrections.

As another quick aside, as mentioned above, soon we hope to have a new version of Probe Image where instead of acquiring a number of separate on-peak acquisitions (each with a different acquisition number), for each TDI "pass", one would simply specify the number of "replicate" acquisitions for the on-peak map, and the program will automatically acquire a replicate set with the same acquisition number and automatically named with the _TDI number ready to import into CalcImage without the convert menu step.

But the issue when there is stage drift not only between multiple element map acquisitions (when one has more elements than spectrometers), but also between replicate on-peak TDI acquisitions of the *same* element, is that once the _TDI files all have the acquisition number, there is currently no way for the align and crop method to be applied.

In fact that is why you asked if there could be an "align and crop" method that could be applied to PrbImg files, *before* they were converted to _TDI files, with the same acquisition number...  I'm thinking, thinking...  because right now CalcImage currently only loads PrbImg files, by converting them to a GRD file, so that is a bit of work, but not overwhelming.

So I guess we could apply the align and crop method to either GRD or PrbImg files, but it just occurred to me that maybe there is another way to do this, since the _TDI GRD files are all numbered even though they have the same acquisition number.  That is, instead of the app looking for different base names as selected by the user, one could look for different _TDI numbers to apply the align and crop to. That might be doable.

But here's another "wrinkle".  Once we implement the new "replicate" TDI acquisition method in Probe Image, these replicate PrbImg files will all have the same acquisition number, and so a _TDI align and crop method, would be the only way this stage drift correction could be applied.

On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.  Whew!

My, my, this gets complicated fast.  My head is spinning again.   :-\
Title: Re: Wish List For CalcImage
Post by: JonF on March 11, 2020, 06:19:54 AM
Hi John,

  Thanks for looking in to this for me.

Yes, it does get very complicated very quickly!

Quote
In fact that is why you asked if there could be an "align and crop" method that could be applied to PrbImg files, *before* they were converted to _TDI files, with the same acquisition number...

Yes, exactly.

I've found two slightly different ways of doing TDI for maps (Xstals/elements for example!).
(i) Where all the elements are replicated on all spectrometers with a short count time being used (so the sum of all count times is used for the quantification ie 100ms rather than 20ms)
RunSP1SP2SP3SP4SP5Time
LLIFPETLPETTAPTAP(ms)
1aFeTiCaMgNa20
1bFeTiCaMgNa20
1cFeTiCaMgNa20
1dFeTiCaMgNa20
1eFeTiCaMgNa20
2aMnVScAlSi100
And so on...

Or

(ii) Where you have one spectrometer for one particular volatile element (looking at you, Na)
RunSP1SP2SP3SP4SP5Time
LLIFPETLPETTAPTAP(ms)
1FeTiCaMgNa100
2MnVScAlNa100
3NiCrKSiNa100

The TDI points are further apart (er, in time) but should hopefully still display a decent time zero intercept.

The second method currently breaks the TDI scripts anyway, so a bit of moving files and renaming is required to get everything to work.

From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Unless, that is, you could implement a TDI flag in the header file of the prbImg format? Say, TDI=0 for single collection and TDI=1 (2, 3 etc) for TDI maps? Reading that in CalcImage is one thing, figuring out how to implement that at setup in Probe Image is very much another!
Title: Re: Wish List For CalcImage
Post by: John Donovan on March 11, 2020, 03:28:10 PM
From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Hi Jon,
OK, we're working on implementing a method to load and/or output (to Surfer) the TDI correction percent maps, as requested by Julien Allaz, so once that is done, we'll look at implementing a method to align and crop PrbImg files.
Title: Re: Wish List For CalcImage
Post by: John Donovan on April 16, 2020, 10:45:32 AM
From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Hi Jon,
OK, we're working on implementing a method to load and/or output (to Surfer) the TDI correction percent maps, as requested by Julien Allaz, so once that is done, we'll look at implementing a method to align and crop PrbImg files.

Hi Jon,
Just wanted to let you know that since we finished the output of TDI percent correction maps feature (as requested by Julien Allaz),

https://probesoftware.com/smf/index.php?topic=912.msg9110#msg9110

we're now working on the Align and Crop method for multiple pass (TDI) PrbImg files as you requested.  It's turning out to be a bit more work compared to GRD files, but hopefully we'll get it done by next week some time.
john
Title: Re: Wish List For CalcImage
Post by: John Donovan on April 19, 2020, 07:22:00 PM
From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Hi Jon,
I think this might work for your TDI replicate sets:

https://probesoftware.com/smf/index.php?topic=1152.msg9177#msg9177

Sorry it took so long, it was a bit more work than we initially thought.
john
Title: Re: Wish List For CalcImage
Post by: JonF on April 20, 2020, 07:34:55 AM
That's brilliant, cheers John! I'll get to having a play with the update.

Sorry it took so long, it was a bit more work than we initially thought.

I think a month turn around time for a new software feature, especially at the minute, is phenomenal!
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on November 12, 2021, 04:10:47 PM
Hi,

I really like the "Extract Shapes, Profiles and Polygon Areas" Feature and it would be even more awesome if it would be possible to zoom into the maps to help with setting polygon or line profile points. Currently, it shows them scales to their original pixel size and some of our quant maps are quite small (strip) maps to account for the increased counting times. So a zoom feature would be are a really big help.

Thanks
Title: Re: Wish List For CalcImage
Post by: John Donovan on November 12, 2021, 04:45:40 PM
Hi,

I really like the "Extract Shapes, Profiles and Polygon Areas" Feature and it would be even more awesome if it would be possible to zoom into the maps to help with setting polygon or line profile points. Currently, it shows them scales to their original pixel size and some of our quant maps are quite small (strip) maps to account for the increased counting times. So a zoom feature would be are a really big help.

Thanks

Yeah, this would be difficult to implement, but we'll look into it.
Title: Re: Wish List For CalcImage
Post by: Anette von der Handt on November 15, 2021, 10:19:12 AM
Would it be easier if - instead of seamless zooming in and out - one could define a zoom factor (e.g. 200x) and then one can digitize in the new view or pop-up window?
Title: Re: Wish List For CalcImage
Post by: John Donovan on November 24, 2021, 12:01:48 PM
I really like the "Extract Shapes, Profiles and Polygon Areas" Feature and it would be even more awesome if it would be possible to zoom into the maps to help with setting polygon or line profile points. Currently, it shows them scales to their original pixel size and some of our quant maps are quite small (strip) maps to account for the increased counting times. So a zoom feature would be are a really big help.

Your wish is granted (I think)!

We didn't want to mess with the scaling issues of graphics "zooming" so we modified the Classify data file loading to allow the user to specify a "pixel expansion" parameter. The default is 1x which means there is no pixel expansion as seen here:

(https://probesoftware.com/smf/gallery/1_24_11_21_11_49_33.png)

By selecting other items from this new dropdown list one can select for example a 2x expansion:

(https://probesoftware.com/smf/gallery/1_24_11_21_11_50_02.png)

Or a 3x expansion:

(https://probesoftware.com/smf/gallery/1_24_11_21_11_50_37.png)

Or, looking at a different dataset, even a 4x expansion:

(https://probesoftware.com/smf/gallery/1_24_11_21_11_51_22.png)

We attempted to keep the pixel selection statistics consistent for all of these "expanded" datasets, but please let us know if you see anything amiss.

Update to v. 13.0.2 of PFE using the help | Update Probe for EPMA menu as usual.
Title: Re: Wish List For CalcImage
Post by: JonF on September 16, 2022, 07:54:27 AM
Hi,

   Is there any way of importing the JEOL EDS maps in to CalcImage for quantification? Everything works for the point analysis, but CalcImage comes back with the following message when I try to import a spectrum image: "Only Thermo NSS and Pathfinder spectrum image files are supported for integrated WDS and EDS map quantification at this time."

It'd be really great to get the EDS working alongside everything else for quantification  ;D
Title: Re: Wish List For CalcImage
Post by: John Donovan on September 16, 2022, 08:50:48 AM
   Is there any way of importing the JEOL EDS maps in to CalcImage for quantification? Everything works for the point analysis, but CalcImage comes back with the following message when I try to import a spectrum image: "Only Thermo NSS and Pathfinder spectrum image files are supported for integrated WDS and EDS map quantification at this time."

It'd be really great to get the EDS working alongside everything else for quantification  ;D

Hi Jon,
I wish for this also!    :)

Originally we've asked JEOL Japan (on the older 8230/8530 instruments) for the ability to start a synchronized WDS and EDS map, but that never happened.  And now they've moved on to their new MEC (Microscope External Control) interface with the new iSP100/iHP200F instruments.  So we could eventually implement that in the new interface, but we are currently testing the single spectrum acquisition interface for these new instruments. It's a very dense interface and minimal documentation.

But after all this time I think there are other ways forward on the spectrum map front. One way is to figure out the structure of the JEOL EDS spectrum image files, and write code to read the intensities. But that means performing the spectrum deconvolution ourselves.  And normalizing to the live time of each pixel.  That is non-trivial to say the least.

Instead, we now think the best way forward is to ask JEOL for a DLL interface call to read the JEOL acquired EDS spectrum image, and obtain the net intensity maps corrected for the live time of each pixel. There is already an MEC call to obtain spectrum map results, so we are looking at that.  It is not clear if these net intensity maps are normalized to the pixel live times. As I said, the MEC documentation is minimal.

Currently we are in communications with JEOL on this last point.  So don't hold your breath, but maybe something will happen (at least on the new iSP100/iHP200F instruments).
Title: Re: Wish List For CalcImage
Post by: sem-geologist on September 16, 2022, 11:55:04 AM
But after all this time I think there are other ways forward on the spectrum map front. One way is to figure out the structure of the JEOL EDS spectrum image files, and write code to read the intensities. But that means performing the spectrum deconvolution ourselves.  And normalizing to the live time of each pixel.  That is non-trivial to say the least.

Whats so wrong in doing deconvolution ourselves (yourself)? Or converting counts to cps? I am pretty sure it is not possible to do deconvolution anyhow worse than OEM implements it or treat the raw data anyhow worse than OEM software does. That is the most bottom line where any attempt to do better will result in better. That is why we had went for such lengths in HyperSpy, where b.t.w. the reader/writter of file formats were recently split into separate python library RosettaSciIO. There are spectral map of jeol and bruker implemented (reverse engineered, but blows away the native Bruker Esprit reader in speed order of magnitude) - thus at least it can be figured out how to read those files. As for live time correction - that depends from implementation of vendor, how it tracks that. At least oxford and Bruker has this nice 0keV peak which tracks the live time, so every spectra in the map has its own encoded live time, other vendors probably contain live time raster.
Title: Re: Wish List For CalcImage
Post by: John Donovan on September 16, 2022, 03:00:21 PM
But after all this time I think there are other ways forward on the spectrum map front. One way is to figure out the structure of the JEOL EDS spectrum image files, and write code to read the intensities. But that means performing the spectrum deconvolution ourselves.  And normalizing to the live time of each pixel.  That is non-trivial to say the least.

Whats so wrong in doing deconvolution ourselves (yourself)? Or converting counts to cps? I am pretty sure it is not possible to do deconvolution anyhow worse than OEM implements it or treat the raw data anyhow worse than OEM software does. That is the most bottom line where any attempt to do better will result in better. That is why we had went for such lengths in HyperSpy, where b.t.w. the reader/writter of file formats were recently split into separate python library RosettaSciIO. There are spectral map of jeol and bruker implemented (reverse engineered, but blows away the native Bruker Esprit reader in speed order of magnitude) - thus at least it can be figured out how to read those files. As for live time correction - that depends from implementation of vendor, how it tracks that. At least oxford and Bruker has this nice 0keV peak which tracks the live time, so every spectra in the map has its own encoded live time, other vendors probably contain live time raster.

Nothing is wrong with doing the deconvolution ourselves. But as I said, it's non-trivial.

There are many, many factors to take into account (e.g., detector characteristics) and it is my experience that the various EDS vendors know better than most, in how to treat their own spectra.  We actually hired a mathematical physicist several years ago, who originally claimed: "oh that should be easy", but after several months of effort, they said it was a lot harder than they thought and gave up.

That doesn't mean that it can't also be done by others, but the various EDS vendors already have much of the work done, so it makes sense to leverage that work.