Post processing and data analysis

[Probeman]

Hi John,

My main point was when averaging do you use mean or median. (For the actual project I don't use calcimage, as i'm dealing with subareas in a map that are non orthogonal)

Thanks

Ben

Hi Ben,
We're just using mean averaging in our Surfer strip scripts, but they could be easily modified for median averaging.  Very fancy that you are dealing with non orthogonal directions with your strip averaging!    Cool!
john

[Ben Buse]

Here are the results from the cartoon example show above, you can see the difference between mean and median. (Although I realised that for R extract function for non-parallel line there is a slight distortion of distance scale, which must be corrected)

[Ben Buse]

Last week at the UK JEOL Users group meeting we were chatting about scripts in ImageJ. Now with the convert to ASCII function in CalcImage its very easy to import maps into ImageJ. I attach a simple script which opens all the '.TXT' files in a folder (i.e. the grd files converted to ASCII), and adds a false color table, an intensity scale and a color scale, and saves the files. It is currently using 16_colors LUT, but this can easily be changed. If you wanted to specify scale width, and number of decimal places on intensity scale these could all be added as user prompts. If you don't want to close all images at end - remove closeall.

Code: [Select]

input = getDirectory("Input directory");
output = getDirectory("Output directory");

suffix = ".TXT";  //you only want to apply to tiff images, no need to ask

pixelsize = getNumber("pixel size in microns", 1);

processFolder(input);

function processFolder(input) {
    list = getFileList(input);
    for (i = 0; i < list.length; i++) {
        if(File.isDirectory(input + list[i]))   //if it's a directory, go to subfolder
            processFolder("" + input + list[i]);
        if(endsWith(list[i], suffix))   //if it's a tiff image, process it
            processFile(input, output, list[i], pixelsize);
        //if it's neither a tiff nor a directory, do nothing
    }
}

function processFile(input, output, file,pixelsize) {

//here, you put the real processingprocessFolder(input);
run("Text Image... ", "open="+"'"+input+file+"'");
run("Set Scale...", "distance=1 known="+pixelsize+" pixel=1 unit=um");
w = getWidth();
h = getHeight();
w1 = w+100;
h1 = h+50;
run("16_colors");
run("Canvas Size...", "width="+w1+" height="+h1+" position=Top-Left");
setForegroundColor(253, 253, 253);
makeRectangle(w, 0, 100, h1);
run("Fill","slice");
makeRectangle(0, h, w1, h1);
run("Fill","slice");
run("Calibration Bar...", "location=[Upper Right] fill=White label=Black number=5 decimal=3 font=12 zoom=1");
scalewidth = d2s((w/2)*pixelsize,0);
run("Scale Bar...", "width="+scalewidth+" height=4 font=14 color=Black background=None location=[Lower Left]");
name = getTitle;
saveAs("Jpeg", output+name);
}
run ("Close All");

It is based on the scripts described in these webpages

https://imagej.net/Batch_Processing

https://forum.image.sc/t/batch-processing-several-folder/6636

Ben

[Ben Buse]

I've just noticed on ubuntu the lut 16 colors is

Code: [Select]

run("16 Colors");
Ben

[glennpoirier]

Thanks  Ben,
I understand this a lot better than the R script I posted!

Cheers
Glenn

[JonF]

Last week at the UK JEOL Users group meeting we were chatting about scripts in ImageJ. Now with the convert to ASCII function in CalcImage its very easy to import maps into ImageJ. I attach a simple script which opens all the '.TXT' files in a folder (i.e. the grd files converted to ASCII), and adds a false color table, an intensity scale and a color scale, and saves the files. It is currently using 16_colors LUT, but this can easily be changed. If you wanted to specify scale width, and number of decimal places on intensity scale these could all be added as user prompts. If you don't want to close all images at end - remove closeall.



It is based on the scripts described in these webpages

https://imagej.net/Batch_Processing

https://forum.image.sc/t/batch-processing-several-folder/6636

Ben

Absolute hero, you've just saved me hours from having to work this out for myself!

[JonF]

Following on from Ben's lead, I've been working on expanding the macro for automating the import and conversion of text images from ProbeImage in to ImageJ and out again as jpeg with scale and calibration bars.

In lieu of a change log, some of the changes for this version include:

• black background instead of white (personal preference)
• the option to open either ProbeImage ascii .TXT files or JEOL matrix map .csv exports
• choice of several lookup tables (ones that are supplied with my current version of Fiji, at least). The default LUT is the one listed first, so if you prefer to use a different default - or one not listed - just change the order they appear in the script. I'm having a bit of a cyan hot moment, myself
• the option to process and export all the maps automatically or to go through the individual maps and alter the brightness/contrast to suit, or to simply input min and max values (I assume I'm not the only person who gets asked about that one?)
• offset the scale bar slightly to stop it overlapping images (particularly small ones), and made it white on the black background
• rounded the scale bar to (hopefully) the nearest 'nice' number e.g. a 500um scale bar instead of 512um, 1000um instead of 1024um etc (the code for this bit isn't the nicest, so feel free to suggest an alternative, elegant method!)
• prevent the scale bar exceeding 50% of the image width e.g. I was getting a 10um scale bar on a 12um map, which looked a bit silly

I've also annotated the script so that it should be easy for anyone to change things to their preference.

Hopefully it should work for everyone else (I've tested on two machines, at least...).

Code: [Select]

//ImageJ macro to format ProbeImage .txt outputs.

input = getDirectory("Input directory");
output = getDirectory("Output directory");

Dialog.create("Probe Image Processing");
Dialog.addNumber("pixel size in microns", 1);
Dialog.addChoice("Select file type:", newArray(".TXT", ".csv"));
Dialog.show();
pixelsize = Dialog.getNumber();
suffix = Dialog.getChoice();

//Set the LUT to use for the images, default Cyan Hot
Dialog.create("Probe Image Processing");
Dialog.addChoice("What lookup table (LUT) would you like to use?", newArray("Cyan Hot","16 colors","Fire","Grays","cool","mpl-viridis","Rainbow RGB","Thermal","Yellow Hot"));
Dialog.show();
LUT = Dialog.getChoice();

//Set whether to manually process brightness/contrast for each map or to run through automatically. Default yes.
Dialog.create("Probe Image Processing");
Dialog.addChoice("Would you like to process individual images?", newArray("Yes","No"));
Dialog.addCheckbox("Use preset Min/Max values?", false)
Dialog.show();
ProcLUT = Dialog.getChoice();
PresetMinMax = Dialog.getCheckbox
if(ProcLUT == "Yes") {
setBatchMode(false);
}
else {
setBatchMode(true);
}

if(PresetMinMax == true) {
ProcLUT = "No";
}
else ;

//Run processFolder function which in turn calls processFile function
processFolder(input);

function processFolder(input) {
    list = getFileList(input);
    for (i = 0; i < list.length; i++) {
        if(File.isDirectory(input + list[i]))   //if it's a directory, go to subfolder
            processFolder("" + input + list[i]);
        if(endsWith(list[i], suffix))   //if it's a .txt image, process it
            processFile(input, output, list[i], pixelsize);
        //if it's neither a .txt nor a directory, do nothing
    }
}


function processFile(input, output, file,pixelsize) {

//imports .txt file as text image
run("Text Image... ", "open="+"'"+input+file+"'");

//sets the scale according to the microns per pixel prompt above
run("Set Scale...", "distance=1 known="+pixelsize+" pixel=1 unit=um");
w = getWidth();
h = getHeight();

//add room for calibration bar
w1 = w+100;

//add room for scale bar (h1), plus 10px gap to avoid overlapping the image (h2)
h1 = h+50;
h2 = h+10;

//apply chosen LUT
run(LUT);

//if option to process individual maps, waits for each user to set min and max pixel values before adding scale bar etc
if(ProcLUT == "Yes") {
run("Brightness/Contrast...");
title = "Adjust Brightness/Contrast";
msg = "For each channel and Apply, Press OK to proceed";
waitForUser(title, msg);
}

else ;

//gives the user the option to add min and max values to maps on a per map basis
if (PresetMinMax == true) {
Dialog.create("Set Min Value");
Dialog.addString("Please enter the Min value", 0);
Dialog.show();
min = Dialog.getString;

Dialog.create("Set Max Value");
Dialog.addString("Please enter the Max value", 100);
Dialog.show();
max = Dialog.getString;
}
else {
getMinAndMax(min,max);  // get display range
}
setMinAndMax(min,max);

//set foreground and background colour to black (personal preference - this could be made another option)
setBackgroundColor(0, 0, 0);
setForegroundColor(0, 0, 0);
//increase the image size to make room for scale and calibration bars
run("Canvas Size...", "width="+w1+" height="+h1+" position=Top-Left");
makeRectangle(w, 0, 100, h1);
run("Fill","slice");
makeRectangle(0, h, w1, h1);
run("Fill","slice");
run("Calibration Bar...", "location=[Upper Right] fill=Black label=White number=5 decimal=2 font=12 zoom=1");

//determine scale bar size
scalewidth = d2s((w/2)*pixelsize,0);

//determine the denominator to be used to get a rounded scale bar size e.g. 500um rather than 512
if (pixelsize >= 10) {
scaledenom = 1000;
}
else if (pixelsize <=0.2) {
scaledenom = 10;
}
else scaledenom = 100;

//round of scale bar to appropriate size
scalewidthR = scaledenom*(round(scalewidth/scaledenom));

//stops the scale bar being bigger than half the image size e.g. a 10um scale bar for a 10um map
if (scalewidthR >= ((w*pixelsize)/2)) {
scalewidthR2 = scalewidthR/2;
}
else {
scalewidthR2 = scalewidthR;
};

//creates scale bar
makeRectangle(5,h2,1,1);
run("Scale Bar...", "width="+scalewidthR2+" height=4 font=14 color=White background=None location=[At Selection]");

//name the file and save as a jpeg
name = getTitle;
saveAs("Jpeg", output+name);
}
run ("Close All");


[Probeman]

Following on from Ben's lead, I've been working on expanding the macro for automating the import and conversion of text images from ProbeImage in to ImageJ and out again as jpeg with scale and calibration bars.

Hi Jon,
Very cool.  Some questions: 

1. When you say "conversion of text images from ProbeImage" do you mean you are reading and converting the .PrbImg Probe Image files directly?

2. Does your code work for both Cameca and JEOL "flavors" of .PrbImg Probe Image files?
john

[JonF]

1. When you say "conversion of text images from ProbeImage" do you mean you are reading and converting the .PrbImg Probe Image files directly?

2. Does your code work for both Cameca and JEOL "flavors" of .PrbImg Probe Image files?
john

Nothing so fancy, I'm afraid, it simply imports the ascii text files as exported from CalcImage (not Probe Image, my mistake!) or the csv files as exported by the JEOL mapping software. These could be either raw intensity data or concentrations.

I use this script to replace some of the functionality of Surfer (e.g. making the pretty final images) as most users don't have this (yes, they could use the offline processing PC, but learning Surfer seems overkill for most of what people would use it for). You could then use imageJ to export line profiles etc - although a lot of this functionality is done by the new polygon extraction features!

 

[John Donovan]

1. When you say "conversion of text images from ProbeImage" do you mean you are reading and converting the .PrbImg Probe Image files directly?

2. Does your code work for both Cameca and JEOL "flavors" of .PrbImg Probe Image files?
john

Nothing so fancy, I'm afraid, it simply imports the ascii text files as exported from CalcImage (not Probe Image, my mistake!) or the csv files as exported by the JEOL mapping software. These could be either raw intensity data or concentrations.

I use this script to replace some of the functionality of Surfer (e.g. making the pretty final images) as most users don't have this (yes, they could use the offline processing PC, but learning Surfer seems overkill for most of what people would use it for). You could then use imageJ to export line profiles etc - although a lot of this functionality is done by the new polygon extraction features!

Hi Jon,
OK, that makes sense now.  That will allow everyone to process quant x-ray maps from CalcImage in ImageJ, which is very nice.

Yes, Surfer is great for making fine adjustments for high quality presentations of x-ray map quant data, especially for publication, but it's not a cheap package, and really it's overkill for most basic operations of exporting selected pixel areas and/or min/max element filtering as you say. 

In fact I think we (and our students) should thank Gareth Seward again for working with us to implement the polygon extraction and pixel filtering feature in CalcImage.  It came out quite nice we think, and of course it is available to all students (and all their computers!):

https://probesoftware.com/smf/index.php?topic=1151.0

john

[Ben Buse]

Here's a variant on a theme, open all the oxide.txt files and save as a scaled stack for further processing or interrogation in imagej. Can easily be changed to open all quant.txt files by changing suffix.

Ben

Code: [Select]

input = getDirectory("Input directory");
output = getDirectory("Output directory");

suffix = "Oxide.TXT";  //you only want to apply to tiff images, no need to ask

pixelsize = getNumber("pixel size in microns", 1);

processFolder(input);

function processFolder(input) {
    list = getFileList(input);
    for (i = 0; i < list.length; i++) {
        if(File.isDirectory(input + list[i]))   //if it's a directory, go to subfolder
            processFolder("" + input + list[i]);
        if(endsWith(list[i], suffix))   //if it's a tiff image, process it
            processFile(input, output, list[i], pixelsize);
        //if it's neither a tiff nor a directory, do nothing
    }
}

function processFile(input, output, file,pixelsize) {

//here, you put the real processingprocessFolder(input);
run("Text Image... ", "open="+"'"+input+file+"'");
}

StackName = getString("Enter name of Stack, no spaces", "Oxide_Stack");
run("Images to Stack", "name="+StackName+" title=[] use");
run("Set Scale...", "distance=1 known="+pixelsize+" pixel=1 unit=um");
name = getTitle;
saveAs("Tiff", output+name);

[Ben Buse]

Hi, I've been playing with stacks - and looking at using a stack of channels.

In ImageJ, a regular stack has a slice for each image, if you change it to a channel for each image - using colour, each channel has its own lut, whilst retaining original values. (I've attached an avi showing it.)

Below is the change I've made to the code above - to create a channel stack, and apply auto brightness contrast to each channel

Code: [Select]

StackName = getString("Enter name of Stack, no spaces", "Oxide_Stack");
run("Images to Stack", "name="+StackName+" title=[] use");
run("Set Scale...", "distance=1 known="+pixelsize+" pixel=1 unit=um");
run("Stack to Hyperstack...", "order=xyczt(default) channels="+nSlices+" slices=1 frames=1 display=Color");
Stack.getDimensions(width,height,channels,slices,frames);
max2=channels+1;
for (i=1; i<max2; i++) {
Stack.setChannel(i);
run("Enhance Contrast", "saturated=0.35");
}
name = getTitle;
saveAs("Tiff", output+name);
In this case to measure the intensity within a selected area (box, circle, polygon) for the whole stack we use the following:

Code: [Select]

Stack.getDimensions(width,height,channels,slices,frames);
max2=channels+1;
for (i=1; i<max2; i++) {
Stack.setChannel(i);
run("Measure");
}
Similarly macros written to plot or export the profile data for all the images in the stack can be easily modified, I've attached the modified versions.

[JonF]

In ImageJ, a regular stack has a slice for each image, if you change it to a channel for each image - using colour, each channel has its own lut, whilst retaining original values. (I've attached an avi showing it.)

Ah, great spot. That's been bugging me for a while!

Thanks Ben!