PI data transmission and/or logging timelag?

[orlandin]

Hello, all! This is likely related to my earlier post about using very fast dwell times when mapping, but I was curious if anyone had an insight or workarounds. I have been noticing that my ability to collect data at fast rates is significantly exceeding my ability to get that data to PI during the mapping process. For instance, I will run a 1024x1024 pixel map at 4 ms, and the stage motion will cease (it's pretty loud at 4ms) after more or less the hour that PI guessed it would. However, I look at PI and see that I still have 600 lines remaining, with pixels being dutifully populated on the map image and in the data log. This continues for the next 1-2 hours until all of the data is in PI, at which point it saves everything perfectly normally and moves on to the next map in the queue (with my beam just sitting on the sample the whole time).

Is this an artefact of my fast dwell times and the 8200's data transmission rates? Or, now that I write this, could it be from the verbose logging mode that I turned on after my many crashes using 2-point mapping in PI? If a map weren't slowly populating itself right now and aborting maps didn't crash all three systems, I would try turning off the logging to test that. The data (and the log) are being written directly to a local SSD drive on a computer with an i5-4570@3.2GHz and ~8gb RAM... but maybe I should get in the habit of closing all other programs? Would that help PI get the resources it needs?

[John Donovan]

Hi Phil,
I think I mentioned previously that 4 ms per pixel is much too short a dwell time for stage mapping.  The Cameca won't even allow stage mapping below 18 ms per pixel. 

As for driver logging, you absolutely *do not* want to leave that running in verbose mode, as you will generate a gigabyte file overnight!!!

And yes, that will slow things down just a wee bit.
john

PS: I should mention that due to some latencies in the driver, when you map this fast there will be some data that is still being buffered after the scan has actually finished. So rather than waste time I would collect more photons by changing the pixel dwell time to at least 20 ms per pixel

[orlandin]

Hi John! That's a great tip about the logging and I'll turn it off as soon as these are done. Given what you said about buffering, I'm actually a little surprised that haven't managed to trigger a more serious crash from that avenue. I definitely think that you're right about slower speeds when it comes to quantified mapping or fine gradations in counts, but the maps that come out of these settings are far from useless. I just posted an example on the other thread related to this that illustrates the niche this style of map can fill since it's way faster than the EDS options here (and I gotta fill those niches man, how else am I ever gonna be able to afford Surfer!): https://probesoftware.com/smf/index.php?topic=1262.0

You would know much more about the differences between JEOL and Cameca than I would, but the JEOL stage speed limit of 15 mm/sec is right there in all of the official documentation. I am re-running the same pair of maps now without any logging to compare the total run times, and will report back!

[John Donovan]

Hi Phil,
I see. So you're not trying to quantify these maps, just using them for sample navigation?  That makes sense.

This is very similar to what Mike Jercinovic does to locate monazite grains in his rocks.  They need to find *every* monazite grain due to the complex fine scale textures in their metamorphic rocks. So they use one of their EPMA instruments just to run stage scans on thin sections all day and night.

[orlandin]

Well, the thought of quantifying them did cross my mind since it would be so easy and you can clearly see concentration differences in the Ca, maybe in the P too. Even that travesty of counting statistics would still be more useful than a purely qualitative map!

Yes, exactly like the UMass monazite maps! I learned about this from Julien Allaz while we were trying to find monazites to date in deformed frictional melt veins from lower-crustal earthquakes. What I found here at UT was that folks were spending 8-12 hours collecting full thin section EDS maps, and then just overlaying the P, Zr, and Ti maps. The rest of the gigabytes of data seemed to be entirely beside the point to them, so I proposed this as a much more efficient method for locating their in-situ accessories minerals which could be differentiated with just five elements. Also, the FEI XL30 hosting the OI EDS system died and left them without much choice. So enthusiasm for these UMass-style rapid scans has been growing here too!

Also, you were 100% right about the logging - disabling that has resulted in maps almost as fast as they are supposed to be. Now if only I could double my speed yet again by using the JEOL bidirectional scanning in PI...

Edit: total run time as recorded in .prbimg files went from 3:25 hrs (11.7 ms/pixel) to 1:43 hrs (5.9 ms/pixel) for a map predicted to take 1:02 hrs (3.5 ms/pixel [but entered as 3]) in PI - difference in time on the 'fast' version from predicted because 'carriage returns' at max stage speed take up 41 minutes in a 1024 row map ~1 cm tall?

[John Donovan]

OK, cool. I agree that Julien and Mike are two of the best to learn from.

On bi-directional scanning, yeah, we've thought about implementing that off and on, but the trouble is many stages aren't reproducible enough for this. Of course for low resolution scanning such as you are doing on whole thin sections, it would make some sense.