Managing data lines in Analyze window

[Philipp Poeml]

Hi John,

* Analyze Window:  Is it possible to select multiple lines to delete/undelete  certain lines that are not that are not adjacent? (I can select using Shift key but not Ctrl).

* Is there some sort of filter that would let me delete let's say all totals under 98 %? Or delete every line with U under 70 %?

* Delete line does not work when showing "combine analysis lines from selected samples". Is that correct? Would it be possible to delete lines also when viewing combined analysis lines?

Thanks
Philipp

[Gareth D Hatton]

* Is there some sort of filter that would let me delete let's say all totals under 98 %? Or delete every line with U under 70 %?

I think that there is a way.  Perhaps John could clarify but:

Go to CalcImage
Processing -> Classify points (from Probe for EPMA quantification)

Load .dat file (your data, you may have to export it in calcimage cluster format...)
Set min max
Extract range based on total.

I think this should do it for the totals.

No idea about doing it for a specific element though.

[John Donovan]

* Analyze Window:  Is it possible to select multiple lines to delete/undelete  certain lines that are not that are not adjacent? (I can select using Shift key but not Ctrl).

Hi Philippe,
Not that I know of- it's just like the Excel spreadsheet in that you can select ranges of rows but not arbitrary rows. But of course you can select different ranges of rows (lines) in the Analyze! grid and then delete or undelete each different range...

* Delete line does not work when showing "combine analysis lines from selected samples". Is that correct? Would it be possible to delete lines also when viewing combined analysis lines?

That's right, because the "combined sample" is not actually stored in your probe data file and is only present as a "view".  If you want to permanently delete some lines from a number of samples that you will be subsequently "combining", just delete the corresponding lines in each of the samples that will be "combined".

Or... even easier, just use the "Combine Selected Samples into a New Sample" button to create a new samples from the selected samples. This means that you now have a new permanent sample from the selected samples that you can apply all data processing options to. For example, deleting and undeleting analysis lines as seen here:

[Philipp Poeml]

Not that I know of- it's just like the Excel spreadsheet in that you can select ranges of rows but not arbitrary rows. But of course you can select different ranges of rows (lines) in the Analyze! grid and then delete or undelete each different range...

Hm, I am not sure what you mean. If I go to excel, I can use the control key to select as many indivitual lines as I like.



Would there be a way to include this into the analyze window?

[Philipp Poeml]

* Is there some sort of filter that would let me delete let's say all totals under 98 %? Or delete every line with U under 70 %?

I think that there is a way.  Perhaps John could clarify but:

Go to CalcImage
Processing -> Classify points (from Probe for EPMA quantification)

Load .dat file (your data, you may have to export it in calcimage cluster format...)
Set min max
Extract range based on total.

I think this should do it for the totals.

No idea about doing it for a specific element though.

Hi Gareth, thanks for the reply. It sounds rather complicated to me... And it would be fantastic to do it for specific elements too.

Maybe John could fill us in here?

Cheers
Ph

Edit by John: see my next post- I think that will help you.

[John Donovan]

Hm, I am not sure what you mean. If I go to excel, I can use the control key to select as many indivitual lines as I like. Would there be a way to include this into the analyze window?

Ha, I did not know that about Excel, unfortunately the spreadsheet control my app uses is not that sophisticated.  You will have to use the Analyze Selected button more than once, or (and maybe this is the best answer) just delete all the points you *don't* want to analyze, and then just use the regular Analyze button to analyze them.  You can always undelete those points later if you want. What I'm saying is that if your goal is to get the average of points only with totals over a certain number this method will propagate through the program.  In other words, once you delete lines with low totals, PFE will ignore all these deleted lines for output or plotting.

Is that what you are after?

[John Donovan]

Hi Philipp,
I think I have a solution for you.

I have modified the Plot! window output to offer the option to export to Excel based on the Minimum Total value. So just select your samples as shown here, select the Send Data To ASCII File option, and enter the Minimum Total (e.g., 80%) and then click Output. The program will calculate all, but only output the ones that have a minimum total as specified.



Then PFE will ask if you want to export to Excel and you will get output that has all the low totals eliminated as seen here:



Will that work for you?  I will upload this modification tonight so you can download it tomorrow.

[John Donovan]

Hi Philipp,
Ok, I uploaded the Plot! fix I mentioned above so update PFE when you are ready...

The program now handles using both the average only and minimum total options together on multiple samples. Here's the plot output from a bunch of samples (average only) and an 80% minimum total:



But do also consider using the Send Data To ASCII File option in the Plot! window as that also allows one to import the same data into Excel with a mouse click.

[Philipp Poeml]

Thanks John. It is working now!

Then, take the points from Post #1 as feature requests. I can do those things in excel, it would just be handy to do it in PfE as well, or better before exporting the data.

Cheers
Ph

[Probeman]

Hi Philipp,
The fix I made this weekend takes care of your point #2, and Reply #2 above takes care of your point #3, so that just leaves your point #1, right? 

[Philipp Poeml]

1) yes, this would be nice... Very handy.

2) the fix you made is great. It would be wonderful if this would work not only for totals, but also for any element. Like U concentrations about XX%...

3) yes, I Agree and understand, the combine analysis line thing is only virtual. Your solution does not work for me, because I would loose the labels of the individual samples when combining them separately. So I do not want to permanently merge them. In this case, I have 100 samples, with 6 points each. If I want to delete all "bad" analysis points from each sample that means A LOT of clicking. And I just don't like mouse clicks.  that's why I asked to have all 600 points listed together, select with the ctrl key pressed (like in excel) and then just delete the bad points...

[John Donovan]

3) yes, I Agree and understand, the combine analysis line thing is only virtual. Your solution does not work for me, because I would loose the labels of the individual samples when combining them separately. So I do not want to permanently merge them. In this case, I have 100 samples, with 6 points each. If I want to delete all "bad" analysis points from each sample that means A LOT of clicking. And I just don't like mouse clicks.  that's why I asked to have all 600 points listed together, select with the ctrl key pressed (like in excel) and then just delete the bad points...

I see. But you could eliminate the "bad" analyses using the Output Data To ACSII (text) File option in Plot! and then they would be exported to Excel without the "bad analyses", right?

[Philipp Poeml]

Yes, kind of. You mean using this function in the Plot! Window and choosing bad analysis by a low total. This might work. But, can I use then the user defined output, especially the one you included for me (with the space before and after and so on)? Of course I would like to use that feature.

In our case it is also a bit difficult. There are points that certainly have a low total. These are easily recognised. But then there are those points where you see from experience that you don't want to include them. It is a bit like acquiring standards. You do 6 points and you will see by experience which ones are statistics and which ones are just bad... That's why I check every point of my analysis.

[John Donovan]

Hi Philipp,
Please go ahead and add to the "wishlist" topic here:

http://probesoftware.com/smf/index.php?topic=71.0

that you wish we could implement a way to "filter" the User Specified Output using a minimum value for the total or perhaps any element?

The latter gets more complicated, for example: what if the specified "minimum" element is missing from the sample to be output? Remember every sample in PFE can be different! What should the "behavior" of this new feature be then?

[John Donovan]

I wonder if it makes any sense to "combine" EDS spectra when combining WDS elements from multiple samples in the Analyze! window when using the Combine Selected Samples button as seen here:



I'm thinking this might be worthwhile to do (not too hard to code) and besides it will improve precision for the EDS elements.  Of course this improvement in precision is slightly different from the improvement in precision from increasing the geometric efficiency as is the case with the "aggregate intensity" feature, where the same WDS element is measured on more then one spectrometer as discussed here:

http://probesoftware.com/smf/index.php?topic=155.msg646#msg646

Instead this improvement in EDS precision from the "combine samples" button is essentially from increasing the count time time (2x, 3x, etc) by combining EDS spectra from multiple samples...  so I will need to normalize the EDS spectra intensities properly depending on the total "combined" EDS acquisition time.