ProbeLab ReImager

[Probeman]

Noah Kraft and Anette von der Handt's article on a new application for batch exporting and annotating images from Thermo Pathfinder software is available now in Microscopy Today:

Software Development

Probelab ReImager: An Open-Source Software for Streamlining Image Processing in an Electron Microscopy Laboratory

Noah Kraft, Anette von der Handt, pp. 38-41

[Nicholas Ritchie]

It's really great to see members of the community developing tools to extend the capabilities of the vendor's software (like for example PfE).  From what I've seen at QMA-2019, ReImager looks like a really nice tool.  JavaScript was likely a good choice to allow Windows, Linux and Mac users.  Annette, will the code be Open Sourced to allow other to help with the development?

[Anette von der Handt]

Hi Nicholas,

Thanks for the kind words. We are happy if the community will find this useful.

Yes, the code is available on Github through the webpage and licensed under MIT copyright license: https://reimager.probelab.net/

As Probeman said (thanks for posting) the article in Microscopy today is out and gives some more information and some examples:  https://www.cambridge.org/core/journals/microscopy-today/article/probelab-reimager-an-opensource-software-for-streamlining-image-processing-in-an-electron-microscopy-laboratory/0162C9669FACF28562C57672D3B62789

Check out the web version of Figure 2 in the article to see the difference in image quality by using ReImager. Also, the way opacity in X-ray map stacks is treated is turning out to be very useful.

Having a simple program for image annotation that also runs on a Mac has been really useful during the pandemic to say the least.

[Mia Kraft]

Hello everyone!

Just wanted to post that we have a new update available for Probelab ReImager (1.7.5) which has quite nice changes.

https://reimager.probelab.net/

The most impactful update for PFE users, 1.7.5 allows for integration of images taken in PFE without needing any extra software. This means you can take all of your images in PFE at analysis time without needing to using JEOL or Thermo software only for images if you want different visualization than PFE provides without extra image editing. It will automatically detect any points taken with PFE and it has all the same options as any Thermo or JEOL image.

Simply load the directory that contains both the PFE MDB and BIM files.

This works on any operating system thanks to Evan Miller(https://www.evanmiller.org/) maintaining MDB Tools. There are known issues for pre-catalina (10.15) versions of Mac that has to do with Mac.

Features:
- MDB support for all of your PFE files
- options to export to ACQ files for PictureSnap App support

Maintenance:
- Updated various packages for security fixes
- Implemented some better memory management for images

Fixes:
- UI overflow issues with large or oddly shaped images

[JonF]

That's a really nice bit of software. Very intuitive and I've already recommended it to some of my users. Thanks for sharing!

It works really well for the images in the .bim and data in the .mdb files from our JEOL, but it doesn't display the images from the .mdb file created on our Cameca?

With the JEOL, is there a way of displaying the data points acquired in PfE over an image taken in the JEOL software?

[John Donovan]

It works really well for the images in the .bim and data in the .mdb files from our JEOL, but it doesn't display the images from the .mdb file created on our Cameca?

That is odd.

Images from the JEOL and Cameca are exactly the same format in the PFE .BIM file.  It might have to do with the fact that JEOL images have an "anti-Cartesian" stage orientation, while Cameca images have a "Cartesian" stage orientation!

https://probesoftware.com/smf/index.php?topic=101.0

[Mia Kraft]

It might have to do with the fact that JEOL images have an "anti-Cartesian" stage orientation, while Cameca images have a "Cartesian" stage orientation!

This could totally be true, if there are some unexpected numbers coming into the program that I never thought to sanitize it could very well be causing some issues. I can try some changes to catch the potential issues and resolve them. I'll push it as a bug fix version if I get something together.

We don't have much access to Cameca data and rarely need it so I wasn't able to make sure it had full compatibility.

With the JEOL, is there a way of displaying the data points acquired in PfE over an image taken in the JEOL software?

I actually want to do this and make it much more encompassing so as not to limit integration between hardware, software, images, etc. I have a list of ideas to incorporate with that, though I am currently working with a lot of SXES data handling, transformation, and visualization which has higher priority.

The next minor version update will likely include these assuming I don't go all in on Cameca specific support.

[JonF]

I am currently working with a lot of SXES data handling, transformation, and visualization which has higher priority.

Let me know if you need a beta tester for any of that - we've got an SXES-LR (the low energy version with J200N and JS50XL diffractors) and getting the data into a useful format is certainly tricky!

[Mia Kraft]

The next minor version update will likely include these assuming I don't go all in on Cameca specific support.

I released a bug fix addressing the issues brought up since 1.7.5. 1.7.6 should fix any bugs reported to me in the last several days. This includes several fixes for reading BIM file images correctly.

https://reimager.probelab.net/

tl;dr:
It seems like I was reading the images as UInt8 when I shouldn't have been which was causing errors but only on some machines' image outputs. I think it was a bit depth issue. This was causing issues on both JEOL and Cameca images but only on some hardware and caused different visual issues on each.

[John Donovan]

The next minor version update will likely include these assuming I don't go all in on Cameca specific support.

I released a bug fix addressing the issues brought up since 1.7.5. 1.7.6 should fix any bugs reported to me in the last several days. This includes several fixes for reading BIM file images correctly.

https://reimager.probelab.net/

tl;dr:
It seems like I was reading the images as UInt8 when I shouldn't have been which was causing errors but only on some machines' image outputs. I think it was a bit depth issue. This was causing issues on both JEOL and Cameca images but only on some hardware and caused different visual issues on each.

All values in the BIM file are signed 32 bit integers (long integers).  Obviously the first two long ints in the BIM file are positive because they are the X and Y pixels dimensions of the following image. Followed by the two long int pixel dimensions of the next image, etc., etc.

Also the pixel intensities should all be positive values from 0 to the max intensity values for that image (which depends on the image interface, but we choose to store in 32 bits since that should accomodate all (most?) analog signal  intensities).

The stage extents are of course stored in the MDB file in the Image table as 32 bit floats, and can be positive and negative values.

[Mia Kraft]

All values in the BIM file are signed 32 bit integers (long integers).

Also the pixel intensities should all be positive values from 0 to the max intensity values for that image

I think my issue is the second part of that quote, where the max value is semi-arbitrary, I found a few ways to fix it. It seems that manually reading it as a UIntLE does work because of the way it is structured in this instance. I'm not sure why it is a signed int32 because it will only ever be 0 or higher (Uint) so that also threw me off as I didn't expect the BIM to be in that format. I went through a lot of options to find what the format was because 32 is really big for B-W images but never actually tried signed.

I'm comfortable with the current method I have of reading out images from the BIM (though I will mess around with it further to make it more compatible). There doesn't seem to be a record in the MDB relating to the max intensity for any single image which makes it more difficult but obviously not impossible to work around. Is this supposed to be related to ImageAnalogAverages (I'll do more testing with this later on regardless)?

The stage extents are of course stored in the MDB file in the Image table as 32 bit floats, and can be positive and negative values.

It works really well for the images in the .bim and data in the .mdb files from our JEOL, but it doesn't display the images from the .mdb file created on our Cameca?

With the JEOL, is there a way of displaying the data points acquired in PfE over an image taken in the JEOL software?

The issue you were experiencing with Cameca data related to the first issue where I wasn't reading some BIM file values correctly. It can currently load the images from Cameca correctly but the points are backwards and upside down (As stated earlier by John, stage orientation issues there).

I still need to do more work with this as it requires a bit larger of a modification and I'm not quite sure how I want to incorporate the changes for easy support. I will also use this opportunity to try to add some more functionality to the image and point interactions among some other suggestions I have gotten.

[John Donovan]

All values in the BIM file are signed 32 bit integers (long integers).

Also the pixel intensities should all be positive values from 0 to the max intensity values for that image

I think my issue is the second part of that quote, where the max value is semi-arbitrary, I found a few ways to fix it. It seems that manually reading it as a UIntLE does work because of the way it is structured in this instance. I'm not sure why it is a signed int32 because it will only ever be 0 or higher (Uint) so that also threw me off as I didn't expect the BIM to be in that format. I went through a lot of options to find what the format was because 32 is really big for B-W images but never actually tried signed.

Hi Mia,
The reason we use signed integers for the intensity values is because signed integers are the only integer data types VB allows!

https://docs.microsoft.com/en-us/dotnet/visual-basic/language-reference/data-types/

Some image interfaces store intensities in 16 bit or 32 bit values so using 32 bits can accomodate them all.

I'm comfortable with the current method I have of reading out images from the BIM (though I will mess around with it further to make it more compatible). There doesn't seem to be a record in the MDB relating to the max intensity for any single image which makes it more difficult but obviously not impossible to work around. Is this supposed to be related to ImageAnalogAverages (I'll do more testing with this later on regardless)?

In the code I sent you there are indeed min and max fields for the intensity values stored in the MDB file for reading the BIM file!   See the DataImageGetImage procedure:

Code: [Select]

tImage.ImageZmin& = PrRs("ImageZmin")
tImage.ImageZmax& = PrRs("ImageZmax")
One could also call a min/max function to return the min/max values of the image intensity array after it is read from the BIM file.

The field "ImageAnalogAverages" is only utilized for image acquisition (pixel averaging).

[Mia Kraft]

In the code I sent you there are indeed min and max fields for the intensity values stored in the MDB file for reading the BIM file!   See the DataImageGetImage procedure:

Code: [Select]

tImage.ImageZmin& = PrRs("ImageZmin")
tImage.ImageZmax& = PrRs("ImageZmax")

I never even thought that Z could relate to the intensities. That makes a lot more sense than what I was thinking before. I also didn't realize (somehow, must have been half asleep yesterday) that the BIM are actual intensity values. I was under the assumption that it was just actual pixel values of an image and was so confused why they were so large. This make so much sense and is better than I thought, thank you!

[John Donovan]

In the code I sent you there are indeed min and max fields for the intensity values stored in the MDB file for reading the BIM file!   See the DataImageGetImage procedure:

Code: [Select]

tImage.ImageZmin& = PrRs("ImageZmin")
tImage.ImageZmax& = PrRs("ImageZmax")

I never even thought that Z could relate to the intensities. That makes a lot more sense than what I was thinking before. I also didn't realize (somehow, must have been half asleep yesterday) that the BIM are actual intensity values. I was under the assumption that it was just actual pixel values of an image and was so confused why they were so large. This make so much sense and is better than I thought, thank you!

No worries.

It's probably derived from the Surfer GRD file format described here:

http://surferhelp.goldensoftware.com/topics/surfer_7_grid_file_format.htm

Where zlo and zhi are the grid data values.