### Author Topic: K-ratio extrapolation and saving bug  (Read 2228 times)

#### Walter

• Posts: 8
##### K-ratio extrapolation and saving bug
« on: March 21, 2017, 06:36:43 am »
Hello,

I was wondering if it was possible to know more deeply how DTSA-II extracts the k-ratios for the quantification using the sample spectrum and standards. Because i'd like to calculate k-ratios by hand for some spectra but I can't obtain the same results.

I've also another issue: I've wrote a script to simulate a batch of spectra, I'd like to save them automatically but the command "saveSpectra" doesn't seem to work, maybe there is a problem with ":". There is an alternative way to save and to avoid having more than 50 spectra unsaved open?

Thank you
« Last Edit: March 21, 2017, 04:40:35 pm by John Donovan »

#### Ben Buse

• Professor
• Posts: 414
##### Re: K-ratio extrapolation and saving bug
« Reply #1 on: March 21, 2017, 02:03:15 pm »
Hi Walter,

Below is an example of a script that batch simulates films on substrates and saves the spectra. For K-ratios think it uses complex peak deconvolution and peak shape fitting so it would be hard to replicate.

The save command is

Code: [Select]
`         resF[fil].save("%s/%s.msa" % ( pathloc, resF[fil] ))   `where pathloc is given a few lines up

Ben

Code: [Select]
`import dtsa2 as dtsa2import dtsa2.mcSimulate3 as mc3test = {}volt = 15 # accelerating voltagedet = findDetector("Si(Li)") # Replace with your detector's namenE = 1000 # Number of electrons simulatede0 = volt # kV using volt loop - 1st 15kV, then 10 kVrhoAg = 10.49# create materialssAg = epq.Material(epq.Composition(map(dtsa2.element,["Ag"],),[1],"Silver"),epq.ToSI.gPerCC(10.5)) #ToSi converts density g/cc to kg/m3, composition as mass fractions316H = epq.Material(epq.Composition(map(dtsa2.element,["Mn","Cr","Mo","Ni","Fe"],),[0.016,0.169,0.022,0.119,0.667],"316H steel"),epq.ToSI.gPerCC(7.8))# calculate electron range for one of the materials - determining range of x-ray emission images and phi-rho-zrange = dtsa2.electronRange(s316H,e0,density=None)# set xray transitionstrs = [epq.XRayTransition(epq.Element.Ag, epq.XRayTransition.LB1),epq.XRayTransition(epq.Element.Fe, epq.XRayTransition.KB1)]# create samples consisting of silver film on steel (316H) substrate film = {}film[1] = [sAg, 0.000000020],[s316H, 0.000010] # 0.000005000 = 5 um 0.000010 = 10um Configuring multi-layers composition and thicknessfilm[2] = [sAg, 0.000000015],[s316H, 0.000010] # 0.000000050 = 0.05 umfilm[3] = [sAg, 0.000000010],[s316H, 0.000010]xtraP = {}xtraP = {"Characteristic Accumulator":True, "Char Fluor Accumulator":True, "Brem Fluor Accumulator":True}xtraP.update(mc3.configureOutput(DefaultOutput))xtraP.update(mc3.configurePhiRhoZ(1.5*range))xtraP.update(mc3.configureEmissionImages(trs, 1.5*range, size = 512))xtraP.update(mc3.configureTrajectoryImage(1.5*range, size = 512))resF = {}for fil in film: extension = str(film[fil][0]) extension = extension.replace("[","") extension = extension.replace("]","") extension = extension.replace(",","") pathloc = 'O:\Documents\PFE_Data\Users\Charles_Younes\\041115_CarbonInSteel\\dtsa2repeat7_' + extension # Change to output folder xtraP.update(mc3.configureOutput(pathloc)) print xtraP resF[fil] = mc3.multiFilm(film[fil], det,e0=e0, nTraj=nE, dose=500.0, sf=True, bf=True,xtraParams=xtraP) # run simulations tmp = str(resF[fil]) tmpx = extension + tmp resF[fil].rename(tmpx) resF[fil].save("%s/%s.msa" % ( pathloc, resF[fil] )) # change pathloc above to output folder ( location, name) it adds extension resF[fil].display()`
« Last Edit: March 23, 2017, 01:36:24 pm by Ben Buse »

#### Walter

• Posts: 8
##### Re: K-ratio extrapolation and saving bug
« Reply #2 on: March 22, 2017, 01:54:10 pm »

Your code is much more complicated than mine, i don't even know a lot of commands that you have used, I'll study on it for a while. Here I've reported one of my typical scripts.

Code: [Select]
`Ta=material('Ta',16.65)Au=material('Au',19.32)SiO2=material('SiO2',2.5)import dtsa2.mcSimulate3 as mc3thk1=[20e-9,70e-9,120e-9,170e-9]thk2=[100e-9,150e-9,200e-9,250e-9,300e-9,350e-9]for i in range(len(thk1)):    for j in range(len(thk2)):        lyr=[(Au,thk1[i]),(Ta,thk2[j]),(SiO2,1e-5)]        mc3.multiFilm(lyr,d2,e0=25,withPoisson=False,nTraj=1000,dose=60.0,sf=True,bf=True,xtraParams={}).display()        print('%d of %d' % (len(thk2)*i+j+1,len(thk1)*len(thk2)))print('end')`
Anyway I've tried to use the command  spectrum.save(path) but the results don't change: if I use

Code: [Select]
`s1.save("C:\s1.msa")`
or

Code: [Select]
`saveSpectra(s1,'C:\s1.msa')`
turns out the same error

Code: [Select]
`Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "C:\Users\Walter\AppData\Local\NIST\NIST DTSA-II Jupiter 2016-05-24\Lib\dtsa2\__init__.py", line 339, in save    os = jio.FileOutputStream(filename.replace(":","")) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(Unknown Source) at java.io.FileOutputStream.<init>(Unknown Source) at java.io.FileOutputStream.<init>(Unknown Source) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at org.python.core.PyReflectedConstructor.constructProxy(PyReflectedConstructor.java:211)java.io.FileNotFoundException: java.io.FileNotFoundException: C\s1.msa (Impossibile trovare il percorso specificato)`
maybe there is an issue in the version that I'm using (the latest), on which version do you run the script?

For the K-ratio basically should be the ratio between the integral with the background stripped between the sample and the standard, right? I'd like to extract in the K-ratio in Origin for some spectra simulated with pyPENELOPE (to make some comparison between the softwares) that are spike like and the quantification of DTSA-II doesn't recognise as spectra.

Walter

#### Ben Buse

• Professor
• Posts: 414
##### Re: K-ratio extrapolation and saving bug
« Reply #3 on: March 23, 2017, 09:29:57 am »
Hi Walter,

I don't known. In Iona resF[fil].save('.msa') works. As does saveSpectra()

Regarding k-ratios why don't you calculate both dtsa-2 and pypenelope spectra by hand using same method so that comparable. Its a ratio to standards so as long as you do the same on unknown and standard you'd expect to get the same value.

There are several methods of k-ratios

On WDS its peak height - background height

EDS I don't know much about but on EDS can by integral under peak minus background. But can also use peak devolution and peak shape fitting. - But this should just remove contributions from overlapping elements.

Do you have PFE? John has now included spectra simulation based on PENEPMA which pypenelope uses - but then does some form of peak broadening to make them realistic. You could then import into dtsa-2.

Ben
« Last Edit: March 23, 2017, 09:36:42 am by John Donovan »

#### Walter

• Posts: 8
##### Re: K-ratio extrapolation and saving bug
« Reply #4 on: March 23, 2017, 10:33:16 am »

I'll try the method that you suggest for k-ratio calculation, I hope to obtain the same values and then, after ZAF correction, the same quantification.

I'm a little abashed, What PFE stands for?
Is something correlated to CalcZAF? or maybe is the program that Philippe Pinard (pyPENELOPE developer) told me about "There is a Fortran program distributed with Penelope That can apply a convolution over the x-ray spectra." but then never reply to me again? or even something else?

Anyway, I really appreciated your help, thank you a lot!

#### Ben Buse

• Professor
• Posts: 414
##### Re: K-ratio extrapolation and saving bug
« Reply #5 on: March 23, 2017, 12:29:35 pm »
Hi Walter,

The peak broadening program is convolg and it is shipped with the version of Penepma that comes with CalcZAF.

PFE is Probe for EPMA - software to drive a microprobe - it is distributed by Probe Software - of which CalcZAF is a free EPMA utility package. See here for PFE spectra simulations:

http://probesoftware.com/smf/index.php?topic=837.0

Ben
« Last Edit: March 23, 2017, 02:22:34 pm by John Donovan »

#### Ben Buse

• Professor
• Posts: 414
##### Re: K-ratio extrapolation and saving bug
« Reply #6 on: March 23, 2017, 12:36:32 pm »
regarding convolg

http://probesoftware.com/smf/index.php?topic=837.15

And paper

PENEPMA: a Monte Carlo programme for the simulation of X-ray emission in EPMA
X Llovet1 and F Salvat2

"A simulated X-ray spectrum corresponds to an ideal detector, and hence, characteristic lines are
very narrow, i.e., each line is fully contained in a single channel of the output histogram. In order to
obtain a realistic spectrum, which can be compared, e.g., to an experimental spectrum, the effect of the
finite energy resolution of the detector must be introduced. The PENEPMA package contains a Fortran
programme called CONVOLG that performs this calculation by assuming that the energy-resolution
function of the detector is a Gaussian with energy-dependent full-width-at-half-maximum."

#### Ben Buse

• Professor
• Posts: 414
##### Re: K-ratio extrapolation and saving bug
« Reply #7 on: March 23, 2017, 01:24:20 pm »
dtsa-2 iona is here

https://www.dropbox.com/s/sswgxrgv7e2i897/dtsa2_Iona.exe?dl=0

comments updated on script posted earlier

#### jrminter

• Professor
• Posts: 52
##### Re: K-ratio extrapolation and saving bug
« Reply #8 on: May 09, 2017, 08:27:02 am »
Walter, below is a reproducible example that works with the current version (Jupiter). The simulation uses the wrapper class provided by Nicholas Ritchie. Take a look at the file "mcSimulate3.py" in the DTSA_Dir/Lib/dtsa2 directory. It has lots of useful functions. The last two lines show how to correctly write a .msa file.

Best regards,
John

# minimal repo example
# Answer for walter on Probe forum

import dtsa2.mcSimulate3 as mc3
import gov.nist.microanalysis.EPQTools as ept
import java.io as jio
import os
import glob
import shutil

e0 = 20.
nTraj = 1000
dose = 60.
outDir ="C:/Temp/"

ag = material('Ag', 10.49)

spc = mc3.simulate(ag, d1, e0, dose, addNoise, nTraj, sf=True, bf=True, xtraParams={})

sName = "Bulk-Ag-%g-kV-%g-Traj" % (e0, nTraj)
spc.rename(sName)
spc.display()

fos = jio.FileOutputStream("%s/%s.msa" % (outDir, sName))
ept.WriteSpectrumAsEMSA1_0.write(spc,fos,ept.WriteSpectrumAsEMSA1_0.Mode.COMPATIBLE)

#### NicholasRitchie

• Moderator
• Professor
• Posts: 26
##### Re: K-ratio extrapolation and saving bug
« Reply #9 on: June 21, 2017, 06:38:46 am »
The save bug was introduced in an early version of DTSA-II Jupiter and has been fixed in more recent versions.

The problem is in the Python class ScritableSpectrum function save(...) in the file "lib\dtsa2\__init__.py" which is executed every time DTSA-II starts up.
The corrected code is as follows:

def save(self, filename):
"""s.save(filename)
Write this spectrum to an EMSA 1.0 format text file."""
os = jio.FileOutputStream(filename[0:2] + filename[2:].replace(":", ""))
ept.WriteSpectrumAsEMSA1_0.write(self.wrapped, os, ept.WriteSpectrumAsEMSA1_0.Mode.COMPATIBLE)
os.close()

The 2017-Feb-17 version on the NIST web site has this fix.