EPMA - Method Development Tool

[John Donovan]

The format of the CSV-file that I import into the tool is very simple at this stage.



The first column has the x-values and any consecutive column has y-values of the wavescans.  The first row contains the titles of the scans that will appear in the legend of the graph. That is all. However, I am wondering, if we should export wavescan conditions as well (in a separate file?). That might be useful some time down the track...

Hi Sandrin,
It appears that you only have the first column for the spectrometer positions, but multiple columns for the intensities. So this assumes that every sample is run exactly the same way on the same spectrometer? Is that the case? 

So can the data file simply be one sample per file?  That is the first column with the spectrometer positions and the second column the spectrometer intensities?  I assume in cps/nA?

I could add a separate file for the condition details, but wouldn't it be better to have the first column header be the spectro number and crystal type?  Just having "SP UNITS" doesn't provide much information...

How about the first column header like this:

SP1, PET, 15, 50, 5

where the 15, 50 and 5 refer to keV, nA and beam size respectively?
john

[Mike Matthews]

Thanks Sandrin.

I'll see if I can send you some spectra of my 'special' metal.

[Sandrin Feig]

The format of the CSV-file that I import into the tool is very simple at this stage.



The first column has the x-values and any consecutive column has y-values of the wavescans.  The first row contains the titles of the scans that will appear in the legend of the graph. That is all. However, I am wondering, if we should export wavescan conditions as well (in a separate file?). That might be useful some time down the track...

Hi Sandrin,
It appears that you only have the first column for the spectrometer positions, but multiple columns for the intensities. So this assumes that every sample is run exactly the same way on the same spectrometer? Is that the case? 

So can the data file simply be one sample per file?  That is the first column with the spectrometer positions and the second column the spectrometer intensities?  I assume in cps/nA?

I could add a separate file for the condition details, but wouldn't it be better to have the first column header be the spectro number and crystal type?  Just having "SP UNITS" doesn't provide much information...

How about the first column header like this:

SP1, PET, 15, 50, 5

where the 15, 50 and 5 refer to keV, nA and beam size respectively?
john

Hi John

At the beginning every scan had to be done the same way. Than I added the annotations which in most cases don't exactly match the position of the scan where a data point was collected. So what I am doing now is temporarily merging two files with different x values into one. In other words, I think it is ok if the x values of scans that people produce is not exactly the same as mine.

The amount of samples per file:
If we assume that I create an option where people can upload scans of an unknown sample than yes, one sample per file, because most of the time, people are producing scans using multiple crystals. Therefore, I think ideally the file would have the x values in column A and than the columns B, C, D, E, ... are filed with scans of the same sample but with a different crystal used.
Column A: x values
Column B: PC3
Column C: PC2
Column D: PC1
Column E: TAP
...

like this we could import everything from the unknown sample by importing just one file and I could fairly easily add the scans to the appropriate crystal.

Yes, first column spec units and second intensity in cps/nA.

Good idea to replace "SP units" with conditions. If we have crystals in dedicated columns than we don't need them here. So we could just do 15 50 5. Do you have information about the instrument? Could we add the model? Or even the instrument number?
For us that would be: CamecaSX100 SX846 15 50 5

Also, what we would need is the sample name in the first row of all the y values (e,g, Column B, C, D, ...) These names will end up in the legend of the graph later on.
Cheers
Sandrin

[John Donovan]

Yes, first column spec units and second intensity in cps/nA.

Good idea to replace "SP units" with conditions. If we have crystals in dedicated columns than we don't need them here. So we could just do 15 50 5. Do you have information about the instrument? Could we add the model? Or even the instrument number? For us that would be: CamecaSX100 SX846 15 50 5

Hi Sandrin,
I'm trying to avoid making you change your code too much (I know how much of a hassle that can be!), but it seems that requiring all spectrometers to have the same range and step size is very limiting.  If we insist on having this file import limitation, I would have to check at PFE export time that that the acquisition was performed with all spectrometers using the same ranges and step sizes...  causing much consternation and gnashing of teeth if they hadn't already done that!

On the Cameca maybe not so much because the Cameca spectrometers generally all have the same range (more or less except for the 180 mm focal circle spectrometer which I have never seen in the wild). However JEOL instruments often have different focal circle spectrometers with different ranges. Normally from 60 to 260 or so for the 140 mm focal circle spectrometers, but those with a 100 mm focal circle only go from 85 to 250 or so. 

In addition it makes no sense to require the same step size on an LIF crystal as an LDE crystal.  It would make more sense to utilize larger steps but longer integration times for the LDE crystals.

So if we require all spectrometers to have the same range and step size, this will force JEOL users with 100 mm focal circle spectrometers to "throw away" some of their 140 mm spectrometer scanning range. We should think hard about this and see if we really want to limit the import format like this.  I suggested a while back that we provide a spectro position column for each spectrometer and I think this is still a good idea, but I understand if it would be too much work to implement- I feel that way all the time!   

On the sample conditions and spectrometer parameters front, I wonder if we should include the spectro number and crystal info in the x-axis column (if you decide to implement separate spectrometer position columns for each spectrometer), and for the conditions maybe just a separate line as I think we can assume that all spectrometers in an import file will have been scanned at the same beam conditions.

With that in mind here is one possible import format for a single sample:

First line:   sample name, e.g., "Actinolite, Cazadero"

Second line:   beam conditions, e.g., "40 degrees, 15 keV, 40 nA, 5 um"

Third line:   scan data labels with 2 columns for each spectrometer, that is a column for the spectro position and a column for the intensities, e.g., "SP1 LPET", "Ca Ka 4 sec", "SP2 LTAP", "Si Ka 4 sec", etc. where the element-xray pair indicates the emission peak the spectrometer was calibrated to and the integration time for each spectrometer step.

Fourth line:   start of scan data with 2 columns for each spectrometer, that is a column for the spectro position (in instrument units) and a column for the intensities (in cps/nA), e.g., "23345", "0.5657", "23543", "0.4345", etc.

I realize that this is asking a lot, but I have learned *the hard way* that we have to write code not for ease of writing, but for ease of use! 

What do you think?
john

[Probeman]

Hi Sandrin,
I've attached a 16 hour full wavescan (.DAT and .XLXS) on a natural (Toba) zircon using LIF, PET, TAP and PC1 crystals.   The only interesting thing I saw was a possible trace Yb in the LiF scan as seen here:



This is the normal PFE output format for wavescan samples, so I hope you would be willing to consider allowing this to be imported directly into your method development tool.  Or specify a new import format that can handle multiple spectrometers with different x-axis values.  I would be pleased to work with you on this.
john

[Sandrin Feig]

Yes, first column spec units and second intensity in cps/nA.

Good idea to replace "SP units" with conditions. If we have crystals in dedicated columns than we don't need them here. So we could just do 15 50 5. Do you have information about the instrument? Could we add the model? Or even the instrument number? For us that would be: CamecaSX100 SX846 15 50 5

Hi Sandrin,
I'm trying to avoid making you change your code too much (I know how much of a hassle that can be!), but it seems that requiring all spectrometers to have the same range and step size is very limiting.  If we insist on having this file import limitation, I would have to check at PFE export time that that the acquisition was performed with all spectrometers using the same ranges and step sizes...  causing much consternation and gnashing of teeth if they hadn't already done that!

On the Cameca maybe not so much because the Cameca spectrometers generally all have the same range (more or less except for the 180 mm focal circle spectrometer which I have never seen in the wild). However JEOL instruments often have different focal circle spectrometers with different ranges. Normally from 60 to 260 or so for the 140 mm focal circle spectrometers, but those with a 100 mm focal circle only go from 85 to 250 or so. 

In addition it makes no sense to require the same step size on an LIF crystal as an LDE crystal.  It would make more sense to utilize larger steps but longer integration times for the LDE crystals.

So if we require all spectrometers to have the same range and step size, this will force JEOL users with 100 mm focal circle spectrometers to "throw away" some of their 140 mm spectrometer scanning range. We should think hard about this and see if we really want to limit the import format like this.  I suggested a while back that we provide a spectro position column for each spectrometer and I think this is still a good idea, but I understand if it would be too much work to implement- I feel that way all the time!   

On the sample conditions and spectrometer parameters front, I wonder if we should include the spectro number and crystal info in the x-axis column (if you decide to implement separate spectrometer position columns for each spectrometer), and for the conditions maybe just a separate line as I think we can assume that all spectrometers in an import file will have been scanned at the same beam conditions.

With that in mind here is one possible import format for a single sample:

First line:   sample name, e.g., "Actinolite, Cazadero"

Second line:   beam conditions, e.g., "40 degrees, 15 keV, 40 nA, 5 um"

Third line:   scan data labels with 2 columns for each spectrometer, that is a column for the spectro position and a column for the intensities, e.g., "SP1 LPET", "Ca Ka 4 sec", "SP2 LTAP", "Si Ka 4 sec", etc. where the element-xray pair indicates the emission peak the spectrometer was calibrated to and the integration time for each spectrometer step.

Fourth line:   start of scan data with 2 columns for each spectrometer, that is a column for the spectro position (in instrument units) and a column for the intensities (in cps/nA), e.g., "23345", "0.5657", "23543", "0.4345", etc.

I realize that this is asking a lot, but I have learned *the hard way* that we have to write code not for ease of writing, but for ease of use! 

What do you think?
john

Hi John

sorry for the late reply!!!
I have now modified the tool so that it can hopefully take any x-axis value.
I guess we have to split the output file into header and data part.
Header:
Maybe I can show the header information when you hover over the name of the scan in the tool? If I do it as a tool tip, than we could add a bit more text.
I think what would be nice is, if we have some general information as well (Instrument type and number and facility information e.g., Cameca SX100  SX846 UTAS). Do you have these information in your software and could include it into the output? Like this we would know where the scans came from.

Data part:
The data part has to start with #DATA and end with #END. Like this I can easily identify the part of the file that contains the data and extract them. Assuming we have (starting from the second row of the data part) two columns for each scan with first column to be spec position and second column to be intensity than we have to add information in the first row that tells us which crystal was used. The second cell of the first row has to be the sample name (whatever is in this cell will appear as legend entry). Therefore I suggest to add the scan conditions to the header.

So that the output would ideally look something like this:

Sample Name: Actinolite, Cazadero
Date collected: 28 Oct 2016
Instrument / Facility: Cameca SX100,  SX846, UTAS
General conditions: 40 degrees, 15 keV, 40 nA, 5 um
Scan conditions:
SP1 LPET, Ca Ka, 4 sec, step size
SP2 TAP, Si Ka, 4 sec, step size
etc.
#DATA
LPET, Actinolite Cazadero, TAP, ...
23345, 0.5657, 23254, ...
23543, 0.4345, ...
...
#END

What do you think? Would that be possible? Does it make sense?

I have to finish off a few other things with the tool, but soon I will also add an option, where the user can upload a scan that they have collected on their instrument of an unknown material and compare with the scans in the tool. Also for that, the output format would be very useful.

Cheers
Sandrin

[John Donovan]

So that the output would ideally look something like this:

Sample Name: Actinolite, Cazadero
Date collected: 28 Oct 2016
Instrument / Facility: Cameca SX100,  SX846, UTAS
General conditions: 40 degrees, 15 keV, 40 nA, 5 um
Scan conditions:
SP1 LPET, Ca Ka, 4 sec, step size
SP2 TAP, Si Ka, 4 sec, step size
etc.
#DATA
LPET, Actinolite Cazadero, TAP, ...
23345, 0.5657, 23254, ...
23543, 0.4345, ...
...
#END

What do you think? Would that be possible? Does it make sense?

Yes, that doesn't sound too hard to implement.  Give me a few days, maybe next week.
john

[John Donovan]

So that the output would ideally look something like this:

Sample Name: Actinolite, Cazadero
Date collected: 28 Oct 2016
Instrument / Facility: Cameca SX100,  SX846, UTAS
General conditions: 40 degrees, 15 keV, 40 nA, 5 um
Scan conditions:
SP1 LPET, Ca Ka, 4 sec, step size
SP2 TAP, Si Ka, 4 sec, step size
etc.
#DATA
LPET, Actinolite Cazadero, TAP, ...
23345, 0.5657, 23254, ...
23543, 0.4345, ...
...
#END

Sandrin,
On the Scan Conditions section are you assuming there are always 5 spectrometers, or will your import code handle if there are only 3 or 4 spectrometers?  I assume you will count the number of SP# lines and use that value to parse the #DATA section?

Also, in the #DATA section you have duplicated the crystals and the sample name?  Why?  Shouldn't it just be pos1, intensity1, pos2, intensity2, etc. as the column headers?

Finally are you assuming the intensity data is always cps/nA?
john

[John Donovan]

Hi Sandrin,
Another question: in PFE each wavescan sample can have up to 72 elements (ROIs), but I assume that you always want to limit the output to 1 scan per spectrometer?

Also I assume that you always want the spectrometers output in spectrometer order (1,2,3... etc.)?

Finally, if the user acquires scans on fewer spectrometers than they actually have, should I just output zeros for those columns, or will your code skip the missing columns based on the missing parameters in the "Scan Conditions" lines?

john

[Sandrin Feig]

So that the output would ideally look something like this:

Sample Name: Actinolite, Cazadero
Date collected: 28 Oct 2016
Instrument / Facility: Cameca SX100,  SX846, UTAS
General conditions: 40 degrees, 15 keV, 40 nA, 5 um
Scan conditions:
SP1 LPET, Ca Ka, 4 sec, step size
SP2 TAP, Si Ka, 4 sec, step size
etc.
#DATA
LPET, Actinolite Cazadero, TAP, ...
23345, 0.5657, 23254, ...
23543, 0.4345, ...
...
#END

Sandrin,
On the Scan Conditions section are you assuming there are always 5 spectrometers, or will your import code handle if there are only 3 or 4 spectrometers?  I assume you will count the number of SP# lines and use that value to parse the #DATA section?

Also, in the #DATA section you have duplicated the crystals and the sample name?  Why?  Shouldn't it just be pos1, intensity1, pos2, intensity2, etc. as the column headers?

Finally are you assuming the intensity data is always cps/nA?
john

Hi John

The import code should be fine handling 4 or even just 3 spectrometers. Everything in the header will most likely be used for a tooltip or something like that.

In the #DATA section the crystal and the sample name are repeated, that is correct. The first, third, fifth, etc. cell of the first row will be used to identify which crystal was used. That is important so that I can match the scan to the corresponding scans that are already in the tool. The content of the second, fourth, sixth, etc. cell of the first row will be shown in the legend of the graph. Therefore I think repeating the sample name here would be best.
If we use pos1 and intensity1, etc. than I cannot extract which crystal was used and the legend would say "intensity1".

Yes, intensity data should be cps/nA. At least everything else in the tool is cps/nA.

Cheers
Sandrin

[Sandrin Feig]

Hi Sandrin,
Another question: in PFE each wavescan sample can have up to 72 elements (ROIs), but I assume that you always want to limit the output to 1 scan per spectrometer?

Also I assume that you always want the spectrometers output in spectrometer order (1,2,3... etc.)?

Finally, if the user acquires scans on fewer spectrometers than they actually have, should I just output zeros for those columns, or will your code skip the missing columns based on the missing parameters in the "Scan Conditions" lines?

john

Hi John

I think, if the user programs more than one scan per sample than the output should go into separate files. Otherwise it becomes very messy. Also, if you want to upload a scan, than one is enough. Or should we have an average option, where you can export the average of multiple scans collected on the sample sample? The spec units would be the same, just the intensity would be a bit smoother...

The spectrometer order is probably not too important since everybody is using a customized instruments. Therefore we have to use the first cell, first row of the #DATA section to identify which crystal was used. But yes, it is probably good to be consistent, so let's do 1,2,3,etc.

With fewer spectrometers I am not sure. I think it might be best to leave the unused ones blank. But I have to test and see what happens.   

Cheers
Sandrin

[John Donovan]

With fewer spectrometers I am not sure. I think it might be best to leave the unused ones blank. But I have to test and see what happens.   

By "blank" do you missing or present with two columns of zeros?
john

[Sandrin Feig]

With fewer spectrometers I am not sure. I think it might be best to leave the unused ones blank. But I have to test and see what happens.   

By "blank" do you missing or present with two columns of zeros?
john

I think we should go with "missing"
Cheers
Sandrin

[John Donovan]

Hi Sandrin,
I made a first attempt at an output format for your EPMA Development Tool import. Please let me know if you want further changes. I've assumed you don't want a tab delimited file? The complete output file is attached below if you want to play with it.
john

#SAMPLE NAME: Toba Zircon (full scan)
#DATE COLLECTED: 10/25/2016 05:42:43 PM
#INSTRUMENT/FACILITY: Cameca SX100, UofO
#GENERAL CONDITIONS: 20 keV, 100 nA, 10 nA
#SCAN CONDITIONS SP1 : TAP, Hf Ma, 40 sec, 51.1475
#SCAN CONDITIONS SP2 : LPET, Zr La, 40 sec, 51.39684
#SCAN CONDITIONS SP3 : LLIF, Ni Ka, 40 sec, 51.06467
#SCAN CONDITIONS SP4 : PC1, O Ka, 40 sec, 51.555
#SCAN CONDITIONS SP5 : PET, Si Ka, 40 sec, 51.48008
TAP, Toba Zircon (full scan), LPET, Toba Zircon (full scan), LLIF, Toba Zircon (full scan), PC1, Toba Zircon (full scan), PET, Toba Zircon (full scan),
#DATA
 84139.0,  1.46615 84138.0,  8.62505 83739.0,  9.26814 83728.0,  2.21885 83738.0,  2.58736
 84087.0,  1.59160 84087.0,  9.74642 83687.0,  9.29166 83676.0,  2.39134 83687.0,  2.36782
 84036.0,  1.70137 84035.0,  10.3502 83636.0,  9.15051 83625.0,  2.36782 83635.0,  2.88531
 83985.0,  1.57592 83984.0,  10.5385 83585.0,  8.90741 83573.0,  2.34430 83584.0,  4.79065
 83934.0,  1.62296 83932.0,  10.6482 83534.0,  9.72297 83522.0,  2.39134 83532.0,  4.64167
 83883.0,  1.65433 83881.0,  10.8678 83483.0,  9.15835 83470.0,  2.32862 83480.0,  3.72427
 83831.0,  1.67001 83830.0,  11.0874 83432.0,  9.29166 83418.0,  2.26589 83429.0,  2.90099


Edit by John: I removed the attached file and attached a newer format example file in the next post.

[Sandrin Feig]

Hi Sandrin,
I made a first attempt at an output format for your EPMA Development Tool import. Please let me know if you want further changes. I've assumed you don't want a tab delimited file? The complete output file is attached below if you want to play with it.
john

#SAMPLE NAME: Toba Zircon (full scan)
#DATE COLLECTED: 10/25/2016 05:42:43 PM
#INSTRUMENT/FACILITY: Cameca SX100, UofO
#GENERAL CONDITIONS: 20 keV, 100 nA, 10 nA
#SCAN CONDITIONS SP1 : TAP, Hf Ma, 40 sec, 51.1475
#SCAN CONDITIONS SP2 : LPET, Zr La, 40 sec, 51.39684
#SCAN CONDITIONS SP3 : LLIF, Ni Ka, 40 sec, 51.06467
#SCAN CONDITIONS SP4 : PC1, O Ka, 40 sec, 51.555
#SCAN CONDITIONS SP5 : PET, Si Ka, 40 sec, 51.48008
TAP, Toba Zircon (full scan), LPET, Toba Zircon (full scan), LLIF, Toba Zircon (full scan), PC1, Toba Zircon (full scan), PET, Toba Zircon (full scan),
#DATA
 84139.0,  1.46615 84138.0,  8.62505 83739.0,  9.26814 83728.0,  2.21885 83738.0,  2.58736
 84087.0,  1.59160 84087.0,  9.74642 83687.0,  9.29166 83676.0,  2.39134 83687.0,  2.36782
 84036.0,  1.70137 84035.0,  10.3502 83636.0,  9.15051 83625.0,  2.36782 83635.0,  2.88531
 83985.0,  1.57592 83984.0,  10.5385 83585.0,  8.90741 83573.0,  2.34430 83584.0,  4.79065
 83934.0,  1.62296 83932.0,  10.6482 83534.0,  9.72297 83522.0,  2.39134 83532.0,  4.64167
 83883.0,  1.65433 83881.0,  10.8678 83483.0,  9.15835 83470.0,  2.32862 83480.0,  3.72427
 83831.0,  1.67001 83830.0,  11.0874 83432.0,  9.29166 83418.0,  2.26589 83429.0,  2.90099

Hi John

Excellent, well done!
I will download the file and play with it.
One thing that I noticed is, that the #DATA has to be moved a little bit up.
Like this:

...
#SCAN CONDITIONS SP5 : PET, Si Ka, 40 sec, 51.48008
#DATA
TAP, Toba Zircon (full scan), LPET, Toba Zircon (full scan), LLIF, Toba Zircon (full scan), PC1, Toba Zircon (full scan), PET, Toba Zircon (full scan),
84139.0,  1.46615 84138.0,  8.62505 83739.0,  9.26814 83728.0,  2.21885 83738.0,  2.58736
...
#END

And can you please add "," behind every entry in the #DATA section?
84139.0,  1.46615, 84138.0,  8.62505, 83739.0,  9.26814, 83728.0,  2.21885, 83738.0,  2.58736,
otherwise it looks very good. I also noticed that I have been running the scans the other way, something that I haven't thought about. But I am pretty sure that doesn't matter.

Thanks
Cheers
Sandrin