Dating back to the 1980s, the real-time processing of digitally acquired data is still stuck in the past as the majority of data processing still takes place long after the acquisition on a separate computer.
Implementing real-time processing is essential for in-situ experiments since the next experimental step at the microscope is usually established by dynamic changes seen in the sample.
While a proficient operator can usually be really good at detecting subtle changes in complex images, Gatan’s latest in-situ cameras frames to disk storage exceed those of high-end video cards.
When employing rapid data collection rates, it is not feasible for the experimentalist to monitor each and every image captured. Although straightforward processing, such as live FFTs, can be utilized for analyzing data in real-time, only the subset of the frames acquired that are displayed on the computer screen are in fact analyzed at the microscope.
Therefore, a new workflow is required that facilitates the real-time processing of acquired data so that the researcher can use the results of the analysis at the microscope to make informed decisions about the status and subsequent steps for the experiment being performed.
Real-Time Feedback
- Computational analysis is necessary to identify subtle changes and adjust decision making during the experiment
- In-situ TEM facilitates observation of dynamic behavior
- Specific stimuli combinations are often chosen throughout observation to drive desired dynamics
- The need to detect dynamics accurately during observation to make informed decisions about what to do next
Dataflow for Proof of Concept Experiment
Conventional TEM workflows work by initially acquiring all data then transmitting this data to another PC for processing and analysis once work on the microscope is finished. The real-time feedback workflow demands all raw data be transferred during capture. Some of the data is then processed, and the results are fed back to the TEM in the course of acquisition.
Typical setup and data flow for in-situ experiments. Image Credit: Gatan Inc.
Real-time feedback setup and data flow for this experiment. Image Credit: Gatan Inc.
Hardware Used
A parallel file system that could be scaled with hundreds of terabytes of storage from DDN Storage was acquired to save all data from a Gatan OneView® IS camera as it was captured during an in-situ heating experiment.
While the filesystem was still writing the data, a second computer running Gatan Microscopy Suite® (GMS) software was used for the processing of the incoming data via scripting. In this case, the script translated a selected area electron diffraction pattern into a 1D profile so that the TEM operator could easily identify any subtle changes.
- 560 TB raw (400 TiB useable) capacity with 8+2 RAID6 protection
- Data acquired using OneView IS 4k x 4k camera, GMS 3 software and integrated DENSsolutions Wildfire heating holder
- GMS 3 PC linked to DDN GS7K
- Windows server (Intel Xeon 3.2 GHz, 64 GB RAM)
Experimental Details and Results
Experiment
VO2 sample heated to induce phase transition; this is subtle in the 2D pattern below.
Algorithm
The 2D diffraction pattern is translated to a 1D profile (Radial Maximum) - (Radial Average).
Data Flow
Data captured and stored at 1600 MB/s, 64 MB/s is processed and results in 0.45 MB/s processed data.
Illustration of Algorithm: 2D diffraction pattern converted to a 1D profile (Radial Maximum) - (Radial Average). Image Credit: Gatan Inc.
Results from a VO2 Monoclinic to Rutile phase transformation demonstrating one application of this live processing. Image Credit: Gatan Inc.
Comparison of the live processing versus full processing after the end of the microscope session. Real-time processing provides feedback at the microscope to detect subtle phase change and enhance decision-making for setting in-situ conditions. Image Credit: Gatan Inc.
Future Directions
Computation will be enhanced by more effective algorithms and improved processing hardware, such as GPU-based processing, a more powerful server or a high-performance cluster.
Other analyses may be conducted, including complex reconstructions, de-noising algorithms, particle/interface tracking and model-based algorithms.
Conclusions
The DDN storage system facilitates the real-time processing of high-frame-rate in-situ images from the OneView IS camera. Diffraction patterns were evaluated in real-time to identify phase transformation in VO2.
This information has been sourced, reviewed and adapted from materials provided by Gatan, Inc.
For more information on this source, please visit Gatan, Inc.