Author Topic: Wish List For CalcImage  (Read 15540 times)

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #30 on: March 04, 2020, 05:09:50 pm »
It was more the "Map math" that I am missing, for combining multiple spectrometers or doing quick ratio-ing of some elements.

Ah, I see.

Well, if you already have your data in GRD files, you can use the Grid | Math menu in Surfer to perform arbitrary math operations on any number of files.

In fact, not only do you get many more mathematical operators in Surfer, but you can also write scripts to automate the image processing if you want.

But now that I think about it, I'm sure one can also perform all sorts of math operations on image data in ImageJ.
« Last Edit: March 04, 2020, 05:17:39 pm by John Donovan »
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

Anette von der Handt

  • Global Moderator
  • Professor
  • *****
  • Posts: 263
    • UMN Probelab
Re: Wish List For CalcImage
« Reply #31 on: March 05, 2020, 10:33:02 am »
Good point. I need to get more proficient in Surfer really.
Against the dark, a tall white fountain played.

JonF

  • Professor
  • ****
  • Posts: 55
Re: Wish List For CalcImage
« Reply #32 on: March 09, 2020, 03:31:58 am »
We've been playing around with the quant mapping features quite a lot here and they're going down a treat. The increase in analysis time is a fair trade off to be able to post-process and discriminate the data set as opposed to the single point, fire away and hope for the best approach of old.

The align maps and TDI map features are both really good - but I'm a bit confused over the order they should be used in cases where both a slight realignment of a mapped area with volatile elements is required.

If we align the maps first (the correct way?), we're required to start up a project and import all .prbImg files to be converted to .GRD. We can then align these .GRD files, but then I can't use the TDI feature as this requires the original .prbImg (which haven't been realigned?).

If we use the TDI correction first on the raw .prbImg files, this creates the e.g. Na_TAP_TDI_1, _2 etc .prbImg files and renames them all to the same run number. I then import them in to CalcImage, where .GRD files of all the .prbImg files are created, including an extra .GRD file (e.g. Na_TAP_.grd) which is the map listed in the Image File for (e.g.) Na in the Specify Quantitative Parameters! menu. If I run the alignment procedure now, is this .grd file regenerated with the new pixel alignments? 
I could rename all the _TDI_x.grd files back to their original run numbers, run the alignment feature, and then rename them back to the first run number, which would (hopefully) make all the _TDI_x.grd files aligned, but would the created _Na_TAP_.grd file be correct (or is it even used when TDI is enabled?). This all has my head spinning!


Would it be possible to alter the align feature to work with the .prbImg files instead of/as well as the .GRD files? 

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #33 on: March 09, 2020, 08:39:07 am »
Hi Jon,
You're making my head spin!   :-[    I'm not quite sure how the alignment cropping would work on a set of TDI replicate files. 

I think you want to do the replicate renaming first on the PrbImg files, then do the alignment procedure on the GRD files.   But I don't have a set of suitable files to test this on.  Maybe you could put together a test set of PrbImg files for me to try this on?

But it's worth mentioning that the conversion of replicate PrbImg files to TDI GRD in CalcImage is intended to be a temporary thing, and that the next Probe Image version will have a new "replicate" acquisition feature that will automatically create the _TDI_1, etc. file names during the image acquisition.
« Last Edit: March 09, 2020, 08:50:24 am by John Donovan »
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #34 on: March 09, 2020, 02:52:53 pm »
If we use the TDI correction first on the raw .prbImg files, this creates the e.g. Na_TAP_TDI_1, _2 etc .prbImg files and renames them all to the same run number. I then import them in to CalcImage, where .GRD files of all the .prbImg files are created, including an extra .GRD file (e.g. Na_TAP_.grd) which is the map listed in the Image File for (e.g.) Na in the Specify Quantitative Parameters! menu. If I run the alignment procedure now, is this .grd file regenerated with the new pixel alignments? 
I could rename all the _TDI_x.grd files back to their original run numbers, run the alignment feature, and then rename them back to the first run number, which would (hopefully) make all the _TDI_x.grd files aligned, but would the created _Na_TAP_.grd file be correct (or is it even used when TDI is enabled?). This all has my head spinning!

Hi Jon,
So I took a quick look at the code and the extra (non-TDI named) x-ray map (e.g., _Na_TAP_.grd) is simply the *sum* of all the replicate measurements.  Why does this file get created?  It's for when one turns off the TDI correction in the Standard Assignments dialog), and then the program ignores the _TDI maps and calculates the map quantification based only on the summed maps (without the TDI extrapolation to zero time).

Would it be possible to alter the align feature to work with the .prbImg files instead of/as well as the .GRD files?

That is a possibility, but can you explain in more detail what exactly your setup is?  For instance, are you trying to perform an align and crop on the replicate images (because there was stage drift between the replicate acquisitions), or are you performing an align and crop on additional maps for another elements (because you needed to acquire more than 5 elements for your map quantification)?

Maybe we should discuss this by phone or Skype because it gets complicated real quick!
« Last Edit: March 09, 2020, 05:32:41 pm by John Donovan »
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #35 on: March 10, 2020, 12:44:48 pm »
Hi Jon,
OK, I've slept on it and I think I see what the problem is.

So normally when one wants to perform the align and crop method on multiple passes on more than a single set of x-ray maps, it's because one acquired more elements than they have WDS spectrometers, as originally described in this example from a few years ago:

https://probesoftware.com/smf/index.php?topic=1152.0

So if one needed to acquire their x-ray maps in more than a single pass, e.g., Si, Fe, Ca, Al, Mg, Na in the first x-ray map acquisition, and then say, K, Mn, Ti, Ba and Ni in the second x-ray map acquisition. And if the area of interest was very small and/or the stage was drifting, the first map acquisition might not be aligned perfectly with the second map acquisition.

Of course this example assumes that one is utilizing MAN background corrections for the background correction, so no off-peak maps need to be acquired! However whether or not off-peak background maps were acquired, the maps from the first pass are named with one acquisition number (including any off-peak maps), while the maps from the second pass are named with a subsequent acquisition number (including any off-peak maps) as seen here:



Where acquisition 00654 was the first pass elements and 00655 was the second pass elements (all MAN backgrounds).

Now in this "normal" align and crop case, one acquires the multiple pass x-ray map acquisitions, then one imports the PrbImg files into CalcImage using the Create New Project wizard menu, and then one has a set of GRD files with the same "base names" as with the original PrbImg files, which one can then (if necessary), apply the align and crop method to these (now GRD formatted) files, prior to the map quantification.

As a quick aside, any off-peak maps acquired will have the same acquisition number as the on-peak maps, and so would be included in the align and crop of the respective on-peak map acquisitions, so there is no way to apply an align and crop between the on and off-peak map acquisitions. Just one more reason to utilize MAN background corrections for your x-ray map quantification I guess!    :)

So far, so good.

Now, in the case of TDI scanning, where one acquires multiple on-peak map acquisitions to account for ion migration and/or beam damage effects on beam sensitive sample, each "replicate" (TDI) acquisition receives a sequential acquisition number, because in the current version of Probe Image, to acquire a set of maps for TDI correction, one acquires multiple on-peak maps each with its own acquisition number as described here:

https://probesoftware.com/smf/index.php?topic=912.0

Now as you have described, currently these separate replicate (on-peak) PrbImg files are first converted to _TDI replicate files to all have the same acquisition number, using the Convert | Convert Replicate PrbImg Files to TDI files menu in CalcImage. Now these replicate PrbImg files have the same acquisition number, but are numbered _TDI_1, _TDI_2, _TDI_3, etc. as seen here:



Then one can import these renamed PrbImg files into a new CalcImage project for subsequent map quantification, just as one would any other map acquisition and the _TDI files are automatically handled for TDI scanning corrections.

As another quick aside, as mentioned above, soon we hope to have a new version of Probe Image where instead of acquiring a number of separate on-peak acquisitions (each with a different acquisition number), for each TDI "pass", one would simply specify the number of "replicate" acquisitions for the on-peak map, and the program will automatically acquire a replicate set with the same acquisition number and automatically named with the _TDI number ready to import into CalcImage without the convert menu step.

But the issue when there is stage drift not only between multiple element map acquisitions (when one has more elements than spectrometers), but also between replicate on-peak TDI acquisitions of the *same* element, is that once the _TDI files all have the acquisition number, there is currently no way for the align and crop method to be applied.

In fact that is why you asked if there could be an "align and crop" method that could be applied to PrbImg files, *before* they were converted to _TDI files, with the same acquisition number...  I'm thinking, thinking...  because right now CalcImage currently only loads PrbImg files, by converting them to a GRD file, so that is a bit of work, but not overwhelming.

So I guess we could apply the align and crop method to either GRD or PrbImg files, but it just occurred to me that maybe there is another way to do this, since the _TDI GRD files are all numbered even though they have the same acquisition number.  That is, instead of the app looking for different base names as selected by the user, one could look for different _TDI numbers to apply the align and crop to. That might be doable.

But here's another "wrinkle".  Once we implement the new "replicate" TDI acquisition method in Probe Image, these replicate PrbImg files will all have the same acquisition number, and so a _TDI align and crop method, would be the only way this stage drift correction could be applied.

On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.  Whew!

My, my, this gets complicated fast.  My head is spinning again.   :-\
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

JonF

  • Professor
  • ****
  • Posts: 55
Re: Wish List For CalcImage
« Reply #36 on: March 11, 2020, 06:19:54 am »
Hi John,

  Thanks for looking in to this for me.

Yes, it does get very complicated very quickly!

Quote
In fact that is why you asked if there could be an "align and crop" method that could be applied to PrbImg files, *before* they were converted to _TDI files, with the same acquisition number...

Yes, exactly.

I've found two slightly different ways of doing TDI for maps (Xstals/elements for example!).
(i) Where all the elements are replicated on all spectrometers with a short count time being used (so the sum of all count times is used for the quantification ie 100ms rather than 20ms)
RunSP1SP2SP3SP4SP5Time
LLIFPETLPETTAPTAP(ms)
1aFeTiCaMgNa20
1bFeTiCaMgNa20
1cFeTiCaMgNa20
1dFeTiCaMgNa20
1eFeTiCaMgNa20
2aMnVScAlSi100
And so on...

Or

(ii) Where you have one spectrometer for one particular volatile element (looking at you, Na)
RunSP1SP2SP3SP4SP5Time
LLIFPETLPETTAPTAP(ms)
1FeTiCaMgNa100
2MnVScAlNa100
3NiCrKSiNa100

The TDI points are further apart (er, in time) but should hopefully still display a decent time zero intercept.

The second method currently breaks the TDI scripts anyway, so a bit of moving files and renaming is required to get everything to work.

From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Unless, that is, you could implement a TDI flag in the header file of the prbImg format? Say, TDI=0 for single collection and TDI=1 (2, 3 etc) for TDI maps? Reading that in CalcImage is one thing, figuring out how to implement that at setup in Probe Image is very much another!

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #37 on: March 11, 2020, 03:28:10 pm »
From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Hi Jon,
OK, we're working on implementing a method to load and/or output (to Surfer) the TDI correction percent maps, as requested by Julien Allaz, so once that is done, we'll look at implementing a method to align and crop PrbImg files.
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #38 on: April 16, 2020, 10:45:32 am »
From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Hi Jon,
OK, we're working on implementing a method to load and/or output (to Surfer) the TDI correction percent maps, as requested by Julien Allaz, so once that is done, we'll look at implementing a method to align and crop PrbImg files.

Hi Jon,
Just wanted to let you know that since we finished the output of TDI percent correction maps feature (as requested by Julien Allaz),

https://probesoftware.com/smf/index.php?topic=912.msg9110#msg9110

we're now working on the Align and Crop method for multiple pass (TDI) PrbImg files as you requested.  It's turning out to be a bit more work compared to GRD files, but hopefully we'll get it done by next week some time.
john
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

John Donovan

  • Administrator
  • Emeritus
  • *****
  • Posts: 2632
  • Other duties as assigned...
    • Probe Software
Re: Wish List For CalcImage
« Reply #39 on: April 19, 2020, 07:22:00 pm »
From what you've described, I'm not sure that the new way of replicate maps would be able to do the second option, so:
Quote
On the other hand, if we did implement the PrbImg align and crop method, one could still acquire separate TDI replicates as we currently do, and then apply the align and crop on the PrbImg files.  Prior to converting the replicate acquisitions to _TDI files using the CalcImage Convert menu, and then importing them into CalcImage for map quantification.
would be my preferred way.

Hi Jon,
I think this might work for your TDI replicate sets:

https://probesoftware.com/smf/index.php?topic=1152.msg9177#msg9177

Sorry it took so long, it was a bit more work than we initially thought.
john
John J. Donovan, Pres. 
(541) 343-3400

"Not Absolutely Certain, Yet Reliable"

JonF

  • Professor
  • ****
  • Posts: 55
Re: Wish List For CalcImage
« Reply #40 on: April 20, 2020, 07:34:55 am »
That's brilliant, cheers John! I'll get to having a play with the update.

Sorry it took so long, it was a bit more work than we initially thought.

I think a month turn around time for a new software feature, especially at the minute, is phenomenal!