Author Topic: Output file column headings: observations  (Read 313 times)

GarnetSnowball

  • Post Doc
  • ***
  • Posts: 10
Output file column headings: observations
« on: September 07, 2023, 06:34:41 PM »
Hi everyone,

I'm new to this forum and EPMA, having just done some analyses this year as part of my Bsc(Hons) thesis project.

I've written a python script to take PFE output files and separate them into separate files for each sample and mineral that I analyse per run, and then plot up Oxide Wt%, Element Mol % etc. etc. for each mineral.

The naming of columns in the output file has a couple little things that might be worth pointing out.

1) the %ERR column names have a space after %ERR! It took me a while to de-bug this, so if anyone else is writing a data processing script this might be something worth being aware of.

2) I found the column naming system for Element W%, Oxide Wt%, Element mol% and Oxide mol% a little inconsistent.
Element W% is X % (where X is the element)
Oxide Wt% is XO % (so far so good)
Element mol% is X AT%
Oxide mol% is X OXMOL%

Particularly in my case where the user inputs the the oxides they wish to plot, the situation where the column title for Oxide Mol% actually uses the element, not the oxide, just complicates things a little because I needed to write a dictionary to map all possible oxides to their elements (apparently the namers of elements didn't have future script-writers in mind either when they chose single and double letter names for elements). The other option would be to write a function where it converts the oxide to the element by taking all letters before with a number or O.

Anyway, its probably worth keeping these in the output files for backwards compatibility with any other scripts that might be out there, but I just thought I'd share my experiences.

Probeman

  • Emeritus
  • *****
  • Posts: 2858
  • Never sleeps...
    • John Donovan
Re: Output file column headings: observations
« Reply #1 on: September 08, 2023, 08:00:35 AM »
1) the %ERR column names have a space after %ERR! It took me a while to de-bug this, so if anyone else is writing a data processing script this might be something worth being aware of.

ASCII output files from Probe for EPMA utilizes tab delimited output, in which all text strings are enclosed in quotations for easy input into Excel. I don't know where the csv file you attached came from, but why don't you just import the PFE export file into Excel and plot it there?  Or use any other scientific plotting  program?

Particularly in my case where the user inputs the the oxides they wish to plot, the situation where the column title for Oxide Mol% actually uses the element, not the oxide, just complicates things a little because I needed to write a dictionary to map all possible oxides to their elements (apparently the namers of elements didn't have future script-writers in mind either when they chose single and double letter names for elements). The other option would be to write a function where it converts the oxide to the element by taking all letters before with a number or O.

I apologize for those early investigators that didn't consider your needs when naming the chemical elements.   ;)
The only stupid question is the one not asked!

GarnetSnowball

  • Post Doc
  • ***
  • Posts: 10
Re: Output file column headings: observations
« Reply #2 on: September 10, 2023, 01:12:50 AM »
Hi Probeman,

thanks for your reply.

My script saves the .dat files that PFE outputs as .csv just for ease of use with pandas dataframe in Python. So that's why you're seeing a .csv.

It is much better to write a script, with which I can make plots in ~20 seconds. With all the modern stats packages available for python and matplotlib being so easy and customisable, I don't see any reason ever to touch Excel ever again! (yipee)

Probeman

  • Emeritus
  • *****
  • Posts: 2858
  • Never sleeps...
    • John Donovan
Re: Output file column headings: observations
« Reply #3 on: September 10, 2023, 08:50:41 AM »
Hi Probeman,

thanks for your reply.

My script saves the .dat files that PFE outputs as .csv just for ease of use with pandas dataframe in Python. So that's why you're seeing a .csv.

It is much better to write a script, with which I can make plots in ~20 seconds. With all the modern stats packages available for python and matplotlib being so easy and customisable, I don't see any reason ever to touch Excel ever again! (yipee)

Yeah, I don't use Excel for plotting either but a lot of people do.  I much prefer Grapher because it support scripting:

https://www.goldensoftware.com/products/grapher/

https://support.goldensoftware.com/hc/en-us/articles/115004546908-Creating-a-Script-using-Grapher-s-Script-Recorder

And students can get Grapher for $50/year:

https://probesoftware.com/smf/index.php?topic=707.msg8220;topicseen#msg8220
The only stupid question is the one not asked!

GarnetSnowball

  • Post Doc
  • ***
  • Posts: 10
Re: Output file column headings: observations
« Reply #4 on: September 10, 2023, 07:35:01 PM »
Wicked, thanks for bringing this to my attention!