SenseAI has been published in a wide variety of Microscopy, Science and Industry publications and papers. Copies of any of the following are available on request.
2025
Broad, Zoë; Robinson, Alex W.; Wells, Jack; Nicholls, Daniel; Moshtaghpour, Amirafshar; Kirkland, Angus I.; Browning, Nigel D.
Compressive electron backscatter diffraction imaging Journal Article
In: Journal of Microscopy, vol. 298, no. 1, pp. 44–57, 2025, ISSN: 0022-2720, 1365-2818.
@article{broad_compressive_2025,
title = {Compressive electron backscatter diffraction imaging},
author = {Zoë Broad and Alex W. Robinson and Jack Wells and Daniel Nicholls and Amirafshar Moshtaghpour and Angus I. Kirkland and Nigel D. Browning},
url = {https://onlinelibrary.wiley.com/doi/10.1111/jmi.13379},
doi = {10.1111/jmi.13379},
issn = {0022-2720, 1365-2818},
year = {2025},
date = {2025-04-01},
urldate = {2025-12-19},
journal = {Journal of Microscopy},
volume = {298},
number = {1},
pages = {44–57},
abstract = {Abstract
Electron backscatter diffraction (EBSD) has developed over the last few decades into a valuable crystallographic characterisation method for a wide range of sample types. Despite these advances, issues such as the complexity of sample preparation, relatively slow acquisition, and damage in beam‐sensitive samples, still limit the quantity and quality of interpretable data that can be obtained. To mitigate these issues, here we propose a method based on the subsampling of probe positions and subsequent reconstruction of an incomplete data set. The missing probe locations (or pixels in the image) are recovered via an inpainting process using a dictionary‐learning based method called beta‐process factor analysis (BPFA). To investigate the robustness of both our inpainting method and Hough‐based indexing, we simulate subsampled and noisy EBSD data sets from a real fully sampled Ni‐superalloy data set for different subsampling ratios of probe positions using both Gaussian and Poisson noise models. We find that zero solution pixel detection (inpainting un‐indexed pixels) enables higher‐quality reconstructions to be obtained. Numerical tests confirm high‐quality reconstruction of band contrast and inverse pole figure maps from only 10% of the probe positions, with the potential to reduce this to 5% if only inverse pole figure maps are needed. These results show the potential application of this method in EBSD, allowing for faster analysis and extending the use of this technique to beam sensitive materials.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Electron backscatter diffraction (EBSD) has developed over the last few decades into a valuable crystallographic characterisation method for a wide range of sample types. Despite these advances, issues such as the complexity of sample preparation, relatively slow acquisition, and damage in beam‐sensitive samples, still limit the quantity and quality of interpretable data that can be obtained. To mitigate these issues, here we propose a method based on the subsampling of probe positions and subsequent reconstruction of an incomplete data set. The missing probe locations (or pixels in the image) are recovered via an inpainting process using a dictionary‐learning based method called beta‐process factor analysis (BPFA). To investigate the robustness of both our inpainting method and Hough‐based indexing, we simulate subsampled and noisy EBSD data sets from a real fully sampled Ni‐superalloy data set for different subsampling ratios of probe positions using both Gaussian and Poisson noise models. We find that zero solution pixel detection (inpainting un‐indexed pixels) enables higher‐quality reconstructions to be obtained. Numerical tests confirm high‐quality reconstruction of band contrast and inverse pole figure maps from only 10% of the probe positions, with the potential to reduce this to 5% if only inverse pole figure maps are needed. These results show the potential application of this method in EBSD, allowing for faster analysis and extending the use of this technique to beam sensitive materials.
Electron backscatter diffraction (EBSD) has developed over the last few decades into a valuable crystallographic characterisation method for a wide range of sample types. Despite these advances, issues such as the complexity of sample preparation, relatively slow acquisition, and damage in beam‐sensitive samples, still limit the quantity and quality of interpretable data that can be obtained. To mitigate these issues, here we propose a method based on the subsampling of probe positions and subsequent reconstruction of an incomplete data set. The missing probe locations (or pixels in the image) are recovered via an inpainting process using a dictionary‐learning based method called beta‐process factor analysis (BPFA). To investigate the robustness of both our inpainting method and Hough‐based indexing, we simulate subsampled and noisy EBSD data sets from a real fully sampled Ni‐superalloy data set for different subsampling ratios of probe positions using both Gaussian and Poisson noise models. We find that zero solution pixel detection (inpainting un‐indexed pixels) enables higher‐quality reconstructions to be obtained. Numerical tests confirm high‐quality reconstruction of band contrast and inverse pole figure maps from only 10% of the probe positions, with the potential to reduce this to 5% if only inverse pole figure maps are needed. These results show the potential application of this method in EBSD, allowing for faster analysis and extending the use of this technique to beam sensitive materials.
Moshtaghpour, Amirafshar; Velazco‐Torrejon, Abner; Nicholls, Daniel; Robinson, Alex W.; Kirkland, Angus I.; Browning, Nigel D.
Diffusion distribution model for damage mitigation in scanning transmission electron microscopy Journal Article
In: Journal of Microscopy, vol. 297, no. 1, pp. 57–77, 2025, ISSN: 0022-2720, 1365-2818.
@article{moshtaghpour_diffusion_2025,
title = {Diffusion distribution model for damage mitigation in scanning transmission electron microscopy},
author = {Amirafshar Moshtaghpour and Abner Velazco‐Torrejon and Daniel Nicholls and Alex W. Robinson and Angus I. Kirkland and Nigel D. Browning},
url = {https://onlinelibrary.wiley.com/doi/10.1111/jmi.13351},
doi = {10.1111/jmi.13351},
issn = {0022-2720, 1365-2818},
year = {2025},
date = {2025-01-01},
urldate = {2025-12-19},
journal = {Journal of Microscopy},
volume = {297},
number = {1},
pages = {57–77},
abstract = {Abstract
Despite the widespread use of Scanning Transmission Electron Microscopy (STEM) for observing the structure of materials at the atomic scale, a detailed understanding of some relevant electron beam damage mechanisms is limited. Recent reports suggest that certain types of damage can be modelled as a diffusion process and that the accumulation effects of this process must be kept low in order to reduce damage. We therefore develop an explicit mathematical formulation of spatiotemporal diffusion processes in STEM that take into account both instrument and sample parameters. Furthermore, our framework can aid the design of Diffusion Controlled Sampling (DCS) strategies using optimally selected probe positions in STEM, that constrain the cumulative diffusion distribution. Numerical simulations highlight the variability of the cumulative diffusion distribution for different experimental STEM configurations. These analytical and numerical frameworks can subsequently be used for careful design of 2‐ and 4‐dimensional STEM experiments where beam damage is minimised.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Despite the widespread use of Scanning Transmission Electron Microscopy (STEM) for observing the structure of materials at the atomic scale, a detailed understanding of some relevant electron beam damage mechanisms is limited. Recent reports suggest that certain types of damage can be modelled as a diffusion process and that the accumulation effects of this process must be kept low in order to reduce damage. We therefore develop an explicit mathematical formulation of spatiotemporal diffusion processes in STEM that take into account both instrument and sample parameters. Furthermore, our framework can aid the design of Diffusion Controlled Sampling (DCS) strategies using optimally selected probe positions in STEM, that constrain the cumulative diffusion distribution. Numerical simulations highlight the variability of the cumulative diffusion distribution for different experimental STEM configurations. These analytical and numerical frameworks can subsequently be used for careful design of 2‐ and 4‐dimensional STEM experiments where beam damage is minimised.
Despite the widespread use of Scanning Transmission Electron Microscopy (STEM) for observing the structure of materials at the atomic scale, a detailed understanding of some relevant electron beam damage mechanisms is limited. Recent reports suggest that certain types of damage can be modelled as a diffusion process and that the accumulation effects of this process must be kept low in order to reduce damage. We therefore develop an explicit mathematical formulation of spatiotemporal diffusion processes in STEM that take into account both instrument and sample parameters. Furthermore, our framework can aid the design of Diffusion Controlled Sampling (DCS) strategies using optimally selected probe positions in STEM, that constrain the cumulative diffusion distribution. Numerical simulations highlight the variability of the cumulative diffusion distribution for different experimental STEM configurations. These analytical and numerical frameworks can subsequently be used for careful design of 2‐ and 4‐dimensional STEM experiments where beam damage is minimised.
2024
Robinson, A W; Wells, J; Moshtaghpour, A; Nicholls, D; Huang, C; Velazco-Torrejon, A; Nicotra, G; Kirkland, A I; Browning, N D
Real-time four-dimensional scanning transmission electron microscopy through sparse sampling Journal Article
In: Chinese Physics B, vol. 33, no. 11, pp. 116804, 2024, ISSN: 1674-1056, 2058-3834.
@article{robinson_real-time_2024,
title = {Real-time four-dimensional scanning transmission electron microscopy through sparse sampling},
author = {A W Robinson and J Wells and A Moshtaghpour and D Nicholls and C Huang and A Velazco-Torrejon and G Nicotra and A I Kirkland and N D Browning},
url = {https://iopscience.iop.org/article/10.1088/1674-1056/ad8a4a},
doi = {10.1088/1674-1056/ad8a4a},
issn = {1674-1056, 2058-3834},
year = {2024},
date = {2024-11-01},
urldate = {2025-12-19},
journal = {Chinese Physics B},
volume = {33},
number = {11},
pages = {116804},
abstract = {Abstract
Four-dimensional scanning transmission electron microscopy (4-D STEM) is a state-of-the-art image acquisition mode used to reveal high and low mass elements at atomic resolution. The acquisition of the electron momenta at each real space probe location allows for various analyses to be performed from a single dataset, including virtual imaging, electric field analysis, as well as analytical or iterative extraction of the object induced phase shift. However, the limiting factor in 4-D STEM is the speed of acquisition which is bottlenecked by the read-out speed of the camera, which must capture a convergent beam electron diffraction (CBED) pattern at each probe position in the scan. Recent developments in sparse sampling and image inpainting (a branch of compressive sensing) for STEM have allowed for real-time recovery of sparsely acquired data from fixed monolithic detectors, Further developments in compressive sensing for 4-D STEM have also demonstrated that acquisition speeds can be increased, i.e., live video rate 4-D imaging is now possible. In this work, we demonstrate the first practical implementations of compressive 4-D STEM for real-time inference on two different scanning transmission electron microscopes.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Four-dimensional scanning transmission electron microscopy (4-D STEM) is a state-of-the-art image acquisition mode used to reveal high and low mass elements at atomic resolution. The acquisition of the electron momenta at each real space probe location allows for various analyses to be performed from a single dataset, including virtual imaging, electric field analysis, as well as analytical or iterative extraction of the object induced phase shift. However, the limiting factor in 4-D STEM is the speed of acquisition which is bottlenecked by the read-out speed of the camera, which must capture a convergent beam electron diffraction (CBED) pattern at each probe position in the scan. Recent developments in sparse sampling and image inpainting (a branch of compressive sensing) for STEM have allowed for real-time recovery of sparsely acquired data from fixed monolithic detectors, Further developments in compressive sensing for 4-D STEM have also demonstrated that acquisition speeds can be increased, i.e., live video rate 4-D imaging is now possible. In this work, we demonstrate the first practical implementations of compressive 4-D STEM for real-time inference on two different scanning transmission electron microscopes.
Four-dimensional scanning transmission electron microscopy (4-D STEM) is a state-of-the-art image acquisition mode used to reveal high and low mass elements at atomic resolution. The acquisition of the electron momenta at each real space probe location allows for various analyses to be performed from a single dataset, including virtual imaging, electric field analysis, as well as analytical or iterative extraction of the object induced phase shift. However, the limiting factor in 4-D STEM is the speed of acquisition which is bottlenecked by the read-out speed of the camera, which must capture a convergent beam electron diffraction (CBED) pattern at each probe position in the scan. Recent developments in sparse sampling and image inpainting (a branch of compressive sensing) for STEM have allowed for real-time recovery of sparsely acquired data from fixed monolithic detectors, Further developments in compressive sensing for 4-D STEM have also demonstrated that acquisition speeds can be increased, i.e., live video rate 4-D imaging is now possible. In this work, we demonstrate the first practical implementations of compressive 4-D STEM for real-time inference on two different scanning transmission electron microscopes.
Robinson, Alex W.; Moshtaghpour, Amirafshar; Wells, Jack; Nicholls, Daniel; Chi, Miaofang; MacLaren, Ian; Kirkland, Angus I.; Browning, Nigel D.
High‐speed 4‐dimensional scanning transmission electron microscopy using compressive sensing techniques Journal Article
In: Journal of Microscopy, vol. 295, no. 3, pp. 278–286, 2024, ISSN: 0022-2720, 1365-2818.
@article{robinson_highspeed_2024,
title = {High‐speed 4‐dimensional scanning transmission electron microscopy using compressive sensing techniques},
author = {Alex W. Robinson and Amirafshar Moshtaghpour and Jack Wells and Daniel Nicholls and Miaofang Chi and Ian MacLaren and Angus I. Kirkland and Nigel D. Browning},
url = {https://onlinelibrary.wiley.com/doi/10.1111/jmi.13315},
doi = {10.1111/jmi.13315},
issn = {0022-2720, 1365-2818},
year = {2024},
date = {2024-09-01},
urldate = {2025-12-19},
journal = {Journal of Microscopy},
volume = {295},
number = {3},
pages = {278–286},
abstract = {Abstract
Here we show that compressive sensing allows 4‐dimensional (4‐D) STEM data to be obtained and accurately reconstructed with both high‐speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4‐D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal‐to‐noise ratio in the recovered phase using 0.3% of the total data.
Lay abstract
: Four‐dimensional scanning transmission electron microscopy (4‐D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2‐dimensional signal is acquired at each 2‐D probe location, equating to a 4‐D dataset.
Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4‐D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256ˆ2 probe locations can take on the order of 30s, approximately 100‐1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4‐D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination.
Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10‐100 times. The acquired data is then processed to fill‐in or inpaint the missing data, taking advantage of the inherently low‐complex signals which can be linearly combined to recover the information.
In this work, similar methods are demonstrated for the acquisition of 4‐D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4‐D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100‐300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Here we show that compressive sensing allows 4‐dimensional (4‐D) STEM data to be obtained and accurately reconstructed with both high‐speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4‐D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal‐to‐noise ratio in the recovered phase using 0.3% of the total data.
Lay abstract
: Four‐dimensional scanning transmission electron microscopy (4‐D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2‐dimensional signal is acquired at each 2‐D probe location, equating to a 4‐D dataset.
Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4‐D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256ˆ2 probe locations can take on the order of 30s, approximately 100‐1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4‐D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination.
Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10‐100 times. The acquired data is then processed to fill‐in or inpaint the missing data, taking advantage of the inherently low‐complex signals which can be linearly combined to recover the information.
In this work, similar methods are demonstrated for the acquisition of 4‐D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4‐D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100‐300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.
Here we show that compressive sensing allows 4‐dimensional (4‐D) STEM data to be obtained and accurately reconstructed with both high‐speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4‐D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal‐to‐noise ratio in the recovered phase using 0.3% of the total data.
Lay abstract
: Four‐dimensional scanning transmission electron microscopy (4‐D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2‐dimensional signal is acquired at each 2‐D probe location, equating to a 4‐D dataset.
Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4‐D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256ˆ2 probe locations can take on the order of 30s, approximately 100‐1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4‐D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination.
Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10‐100 times. The acquired data is then processed to fill‐in or inpaint the missing data, taking advantage of the inherently low‐complex signals which can be linearly combined to recover the information.
In this work, similar methods are demonstrated for the acquisition of 4‐D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4‐D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100‐300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.
Nicholls, Daniel; Kobylynska, Maryna; Broad, Zoë; Wells, Jack; Robinson, Alex; McGrouther, Damien; Moshtaghpour, Amirafshar; Kirkland, Angus I; Fleck, Roland A; Browning, Nigel D
The Potential of Subsampling and Inpainting for Fast Low-Dose Cryo FIB-SEM Imaging Journal Article
In: Microscopy and Microanalysis, vol. 30, no. 1, pp. 96–102, 2024, ISSN: 1431-9276, 1435-8115.
@article{nicholls_potential_2024,
title = {The Potential of Subsampling and Inpainting for Fast Low-Dose Cryo FIB-SEM Imaging},
author = {Daniel Nicholls and Maryna Kobylynska and Zoë Broad and Jack Wells and Alex Robinson and Damien McGrouther and Amirafshar Moshtaghpour and Angus I Kirkland and Roland A Fleck and Nigel D Browning},
url = {https://academic.oup.com/mam/article/30/1/96/7602171},
doi = {10.1093/micmic/ozae005},
issn = {1431-9276, 1435-8115},
year = {2024},
date = {2024-03-01},
urldate = {2025-12-19},
journal = {Microscopy and Microanalysis},
volume = {30},
number = {1},
pages = {96–102},
abstract = {Abstract
Traditional image acquisition for cryo focused ion-beam scanning electron microscopy (FIB-SEM) tomography often sees thousands of images being captured over a period of many hours, with immense data sets being produced. When imaging beam sensitive materials, these images are often compromised by additional constraints related to beam damage and the devitrification of the material during imaging, which renders data acquisition both costly and unreliable. Subsampling and inpainting are proposed as solutions for both of these aspects, allowing fast and low-dose imaging to take place in the Focused ion-beam scanning electron microscopy FIB-SEM without an appreciable loss in image quality. In this work, experimental data are presented which validate subsampling and inpainting as a useful tool for convenient and reliable data acquisition in a FIB-SEM, with new methods of handling three-dimensional data being employed in the context of dictionary learning and inpainting algorithms using a newly developed microscope control software and data recovery algorithm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Traditional image acquisition for cryo focused ion-beam scanning electron microscopy (FIB-SEM) tomography often sees thousands of images being captured over a period of many hours, with immense data sets being produced. When imaging beam sensitive materials, these images are often compromised by additional constraints related to beam damage and the devitrification of the material during imaging, which renders data acquisition both costly and unreliable. Subsampling and inpainting are proposed as solutions for both of these aspects, allowing fast and low-dose imaging to take place in the Focused ion-beam scanning electron microscopy FIB-SEM without an appreciable loss in image quality. In this work, experimental data are presented which validate subsampling and inpainting as a useful tool for convenient and reliable data acquisition in a FIB-SEM, with new methods of handling three-dimensional data being employed in the context of dictionary learning and inpainting algorithms using a newly developed microscope control software and data recovery algorithm.
Traditional image acquisition for cryo focused ion-beam scanning electron microscopy (FIB-SEM) tomography often sees thousands of images being captured over a period of many hours, with immense data sets being produced. When imaging beam sensitive materials, these images are often compromised by additional constraints related to beam damage and the devitrification of the material during imaging, which renders data acquisition both costly and unreliable. Subsampling and inpainting are proposed as solutions for both of these aspects, allowing fast and low-dose imaging to take place in the Focused ion-beam scanning electron microscopy FIB-SEM without an appreciable loss in image quality. In this work, experimental data are presented which validate subsampling and inpainting as a useful tool for convenient and reliable data acquisition in a FIB-SEM, with new methods of handling three-dimensional data being employed in the context of dictionary learning and inpainting algorithms using a newly developed microscope control software and data recovery algorithm.
2023
Wells, Jack; Moshtaghpour, Amirafshar; Nicholls, Daniel; Robinson, Alex W.; Zheng, Yalin; Castagna, Jony; Browning, Nigel D.
SenseAI: Real-Time Inpainting for Electron Microscopy Miscellaneous
2023, (arXiv:2311.15061).
@misc{wells_senseai:_2023,
title = {SenseAI: Real-Time Inpainting for Electron Microscopy},
author = {Jack Wells and Amirafshar Moshtaghpour and Daniel Nicholls and Alex W. Robinson and Yalin Zheng and Jony Castagna and Nigel D. Browning},
url = {http://arxiv.org/abs/2311.15061},
doi = {10.48550/arXiv.2311.15061},
year = {2023},
date = {2023-11-01},
urldate = {2025-12-19},
publisher = {arXiv},
abstract = {Despite their proven success and broad applicability to Electron Microscopy (EM) data, joint dictionary-learning and sparse-coding based inpainting algorithms have so far remained impractical for real-time usage with an Electron Microscope. For many EM applications, the reconstruction time for a single frame is orders of magnitude longer than the data acquisition time, making it impossible to perform exclusively subsampled acquisition. This limitation has led to the development of SenseAI, a C++/CUDA library capable of extremely efficient dictionary-based inpainting. SenseAI provides N-dimensional dictionary learning, live reconstructions, dictionary transfer and visualization, as well as real-time plotting of statistics, parameters, and image quality metrics.},
note = {arXiv:2311.15061},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Despite their proven success and broad applicability to Electron Microscopy (EM) data, joint dictionary-learning and sparse-coding based inpainting algorithms have so far remained impractical for real-time usage with an Electron Microscope. For many EM applications, the reconstruction time for a single frame is orders of magnitude longer than the data acquisition time, making it impossible to perform exclusively subsampled acquisition. This limitation has led to the development of SenseAI, a C++/CUDA library capable of extremely efficient dictionary-based inpainting. SenseAI provides N-dimensional dictionary learning, live reconstructions, dictionary transfer and visualization, as well as real-time plotting of statistics, parameters, and image quality metrics.
Compressive Cryo FIB-SEM Tomography Proceedings Article
In: Proceedings of the Microscience Microscopy Congress 2023 incorporating EMAG 2023, Royal Microscopical Society, 2023.
@inproceedings{noauthor_compressive_2023,
title = {Compressive Cryo FIB-SEM Tomography},
url = {https://www.mmc-series.org.uk/abstract/compressive-cryo-fib-sem-tomography.html},
doi = {10.22443/rms.mmc2023.161},
year = {2023},
date = {2023-07-01},
urldate = {2025-12-19},
booktitle = {Proceedings of the Microscience Microscopy Congress 2023 incorporating EMAG 2023},
publisher = {Royal Microscopical Society},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Broad, Zoë; Nicholls, Daniel; Wells, Jack; Moshtaghpour, Amirafshar; Robinson, Alex W; Masters, Robert; Hughes, Louise; Browning, Nigel D
Subsampling Methods for Fast Electron Backscattered Diffraction Analysis for SEM Journal Article
In: Microscopy and Microanalysis, vol. 29, no. Supplement_1, pp. 467–469, 2023, ISSN: 1431-9276, 1435-8115.
@article{broad_subsampling_2023,
title = {Subsampling Methods for Fast Electron Backscattered Diffraction Analysis for SEM},
author = {Zoë Broad and Daniel Nicholls and Jack Wells and Amirafshar Moshtaghpour and Alex W Robinson and Robert Masters and Louise Hughes and Nigel D Browning},
url = {https://academic.oup.com/mam/article/29/Supplement_1/467/7228222},
doi = {10.1093/micmic/ozad067.220},
issn = {1431-9276, 1435-8115},
year = {2023},
date = {2023-07-01},
urldate = {2025-12-19},
journal = {Microscopy and Microanalysis},
volume = {29},
number = {Supplement_1},
pages = {467–469},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Nicholls, D.; Wells, J.; Robinson, A. W.; Moshtaghpour, A.; Kobylynska, M.; Fleck, R. A.; Kirkland, A. I.; Browning, N. D.
A Targeted Sampling Strategy for Compressive Cryo Focused Ion Beam Scanning Electron Microscopy Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, IEEE, Rhodes Island, Greece, 2023, ISBN: 9781728163277.
@inproceedings{nicholls_targeted_2023,
title = {A Targeted Sampling Strategy for Compressive Cryo Focused Ion Beam Scanning Electron Microscopy},
author = {D. Nicholls and J. Wells and A. W. Robinson and A. Moshtaghpour and M. Kobylynska and R. A. Fleck and A. I. Kirkland and N. D. Browning},
url = {https://ieeexplore.ieee.org/document/10096157/},
doi = {10.1109/ICASSP49357.2023.10096157},
isbn = {9781728163277},
year = {2023},
date = {2023-06-01},
urldate = {2025-12-19},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1–5},
publisher = {IEEE},
address = {Rhodes Island, Greece},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Robinson, Alex W.; Wells, Jack; Nicholls, Daniel; Moshtaghpour, Amirafshar; Chi, Miaofang; Kirkland, Angus I.; Browning, Nigel D.
Towards real‐time STEM simulations through targeted subsampling strategies Journal Article
In: Journal of Microscopy, vol. 290, no. 1, pp. 53–66, 2023, ISSN: 0022-2720, 1365-2818.
@article{robinson_towards_2023,
title = {Towards real‐time STEM simulations through targeted subsampling strategies},
author = {Alex W. Robinson and Jack Wells and Daniel Nicholls and Amirafshar Moshtaghpour and Miaofang Chi and Angus I. Kirkland and Nigel D. Browning},
url = {https://onlinelibrary.wiley.com/doi/10.1111/jmi.13177},
doi = {10.1111/jmi.13177},
issn = {0022-2720, 1365-2818},
year = {2023},
date = {2023-04-01},
urldate = {2025-12-19},
journal = {Journal of Microscopy},
volume = {290},
number = {1},
pages = {53–66},
abstract = {Abstract
Scanning transmission electron microscopy images can be complex to interpret on the atomic scale as the contrast is sensitive to multiple factors such as sample thickness, composition, defects and aberrations. Simulations are commonly used to validate or interpret real experimental images, but they come at a cost of either long computation times or specialist hardware such as graphics processing units. Recent works in compressive sensing for experimental STEM images have shown that it is possible to significantly reduce the amount of acquired signal and still recover the full image without significant loss of image quality, and therefore it is proposed here that similar methods can be applied to STEM simulations. In this paper, we demonstrate a method that can significantly increase the efficiency of STEM simulations through a targeted sampling strategy, along with a new approach to independently subsample each frozen phonon layer. We show the effectiveness of this method by simulating a SrTiO
3
grain boundary and monolayer 2H‐MoS
2
containing a sulphur vacancy using the abTEM software. We also show how this method is not limited to only traditional multislice methods, but also increases the speed of the PRISM simulation method. Furthermore, we discuss the possibility for STEM simulations to
seed
the acquisition of real data, to potentially lead the way to self‐driving (correcting) STEM.
,
Lay Description
Scanning Transmission Electron Microscopy (STEM) simulation is used to validate contrast in experimental images through matching procedures. Parameter searches are performed over various combinations of aberrations and sample structures until an approximation is made with respect to the experimental image. The most common approach for calculating simulations is the multislice approach, where the sample potential is approximated as a series of two‐dimensional slices. The propagation of the beam between slices involves multiple Fast Fourier Transform calculations, which must be performed at all probe locations, hence it is computationally expensive. This means that STEM multislice simulations take a significant amount time to perform, even with specialist hardware such as graphical processing units for parallelisation of calculation.
An alternative to this is a recently developed algorithm known as the plane‐wave reciprocal‐space interpolated scattering matrix (PRISM) method. PRISM uses the fact that the electron probe can also be expressed as a plane‐wave expansion, and a basis set of plane‐waves can be propagated through the sample (as in the multislice step) independently of one another. After the plane‐waves are propagated through the sample, they form what is known as a scattering matrix and can be superimposed with appropriate coefficients to approximate the exit wavefunction/probe. This makes PRISM significantly faster to compute exit probes, at a small expense of accuracy with respect to the ground truth multislice simulation.
In this paper, we present a novel method for increasing STEM simulation computation time through targeted probe sub‐sampling. This method takes advantage of developments within the compressive sensing (CS) community, whereby a significant proportion of probe locations can be omitted during calculation, yet inpainted through a dictionary learning and sparse coding algorithms (such as Beta Process Factor Analysis, BPFA) with minimal loss of accuracy. The method is implemented with both the PRISM and multislice algorithms, and demonstrations of the method are applied to HAADF simulations of a low energy strontium titanate grain boundary, and the 2H phase of monolayer molybdenum disulfide with sulfur vacancies. The methods yield results which are functionally identical to the ground truth (fully sampled) multislice simulations, but can be performed up to 350× faster by using the PRISM method with sub‐sampling.
Furthermore, we also demonstrate how STEM simulations can seed the recovery of experimental CS‐STEM data through (dictionary) transfer learning. This is applied to sub‐sampled HAADF images of yttrium silicide, and shows a 9% increase in PSNR versus only using BPFA.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Scanning transmission electron microscopy images can be complex to interpret on the atomic scale as the contrast is sensitive to multiple factors such as sample thickness, composition, defects and aberrations. Simulations are commonly used to validate or interpret real experimental images, but they come at a cost of either long computation times or specialist hardware such as graphics processing units. Recent works in compressive sensing for experimental STEM images have shown that it is possible to significantly reduce the amount of acquired signal and still recover the full image without significant loss of image quality, and therefore it is proposed here that similar methods can be applied to STEM simulations. In this paper, we demonstrate a method that can significantly increase the efficiency of STEM simulations through a targeted sampling strategy, along with a new approach to independently subsample each frozen phonon layer. We show the effectiveness of this method by simulating a SrTiO
3
grain boundary and monolayer 2H‐MoS
2
containing a sulphur vacancy using the abTEM software. We also show how this method is not limited to only traditional multislice methods, but also increases the speed of the PRISM simulation method. Furthermore, we discuss the possibility for STEM simulations to
seed
the acquisition of real data, to potentially lead the way to self‐driving (correcting) STEM.
,
Lay Description
Scanning Transmission Electron Microscopy (STEM) simulation is used to validate contrast in experimental images through matching procedures. Parameter searches are performed over various combinations of aberrations and sample structures until an approximation is made with respect to the experimental image. The most common approach for calculating simulations is the multislice approach, where the sample potential is approximated as a series of two‐dimensional slices. The propagation of the beam between slices involves multiple Fast Fourier Transform calculations, which must be performed at all probe locations, hence it is computationally expensive. This means that STEM multislice simulations take a significant amount time to perform, even with specialist hardware such as graphical processing units for parallelisation of calculation.
An alternative to this is a recently developed algorithm known as the plane‐wave reciprocal‐space interpolated scattering matrix (PRISM) method. PRISM uses the fact that the electron probe can also be expressed as a plane‐wave expansion, and a basis set of plane‐waves can be propagated through the sample (as in the multislice step) independently of one another. After the plane‐waves are propagated through the sample, they form what is known as a scattering matrix and can be superimposed with appropriate coefficients to approximate the exit wavefunction/probe. This makes PRISM significantly faster to compute exit probes, at a small expense of accuracy with respect to the ground truth multislice simulation.
In this paper, we present a novel method for increasing STEM simulation computation time through targeted probe sub‐sampling. This method takes advantage of developments within the compressive sensing (CS) community, whereby a significant proportion of probe locations can be omitted during calculation, yet inpainted through a dictionary learning and sparse coding algorithms (such as Beta Process Factor Analysis, BPFA) with minimal loss of accuracy. The method is implemented with both the PRISM and multislice algorithms, and demonstrations of the method are applied to HAADF simulations of a low energy strontium titanate grain boundary, and the 2H phase of monolayer molybdenum disulfide with sulfur vacancies. The methods yield results which are functionally identical to the ground truth (fully sampled) multislice simulations, but can be performed up to 350× faster by using the PRISM method with sub‐sampling.
Furthermore, we also demonstrate how STEM simulations can seed the recovery of experimental CS‐STEM data through (dictionary) transfer learning. This is applied to sub‐sampled HAADF images of yttrium silicide, and shows a 9% increase in PSNR versus only using BPFA.
Scanning transmission electron microscopy images can be complex to interpret on the atomic scale as the contrast is sensitive to multiple factors such as sample thickness, composition, defects and aberrations. Simulations are commonly used to validate or interpret real experimental images, but they come at a cost of either long computation times or specialist hardware such as graphics processing units. Recent works in compressive sensing for experimental STEM images have shown that it is possible to significantly reduce the amount of acquired signal and still recover the full image without significant loss of image quality, and therefore it is proposed here that similar methods can be applied to STEM simulations. In this paper, we demonstrate a method that can significantly increase the efficiency of STEM simulations through a targeted sampling strategy, along with a new approach to independently subsample each frozen phonon layer. We show the effectiveness of this method by simulating a SrTiO
3
grain boundary and monolayer 2H‐MoS
2
containing a sulphur vacancy using the abTEM software. We also show how this method is not limited to only traditional multislice methods, but also increases the speed of the PRISM simulation method. Furthermore, we discuss the possibility for STEM simulations to
seed
the acquisition of real data, to potentially lead the way to self‐driving (correcting) STEM.
,
Lay Description
Scanning Transmission Electron Microscopy (STEM) simulation is used to validate contrast in experimental images through matching procedures. Parameter searches are performed over various combinations of aberrations and sample structures until an approximation is made with respect to the experimental image. The most common approach for calculating simulations is the multislice approach, where the sample potential is approximated as a series of two‐dimensional slices. The propagation of the beam between slices involves multiple Fast Fourier Transform calculations, which must be performed at all probe locations, hence it is computationally expensive. This means that STEM multislice simulations take a significant amount time to perform, even with specialist hardware such as graphical processing units for parallelisation of calculation.
An alternative to this is a recently developed algorithm known as the plane‐wave reciprocal‐space interpolated scattering matrix (PRISM) method. PRISM uses the fact that the electron probe can also be expressed as a plane‐wave expansion, and a basis set of plane‐waves can be propagated through the sample (as in the multislice step) independently of one another. After the plane‐waves are propagated through the sample, they form what is known as a scattering matrix and can be superimposed with appropriate coefficients to approximate the exit wavefunction/probe. This makes PRISM significantly faster to compute exit probes, at a small expense of accuracy with respect to the ground truth multislice simulation.
In this paper, we present a novel method for increasing STEM simulation computation time through targeted probe sub‐sampling. This method takes advantage of developments within the compressive sensing (CS) community, whereby a significant proportion of probe locations can be omitted during calculation, yet inpainted through a dictionary learning and sparse coding algorithms (such as Beta Process Factor Analysis, BPFA) with minimal loss of accuracy. The method is implemented with both the PRISM and multislice algorithms, and demonstrations of the method are applied to HAADF simulations of a low energy strontium titanate grain boundary, and the 2H phase of monolayer molybdenum disulfide with sulfur vacancies. The methods yield results which are functionally identical to the ground truth (fully sampled) multislice simulations, but can be performed up to 350× faster by using the PRISM method with sub‐sampling.
Furthermore, we also demonstrate how STEM simulations can seed the recovery of experimental CS‐STEM data through (dictionary) transfer learning. This is applied to sub‐sampled HAADF images of yttrium silicide, and shows a 9% increase in PSNR versus only using BPFA.
Browning, Nigel D.; Castagna, Jony; Kirkland, Angus I.; Moshtaghpour, Amirafshar; Nicholls, Daniel; Robinson, Alex W.; Wells, Jack; Zheng, Yalin
The advantages of sub-sampling and Inpainting for scanning transmission electron microscopy Journal Article
In: Applied Physics Letters, vol. 122, no. 5, pp. 050501, 2023, ISSN: 0003-6951, 1077-3118.
@article{browning_advantages_2023,
title = {The advantages of sub-sampling and Inpainting for scanning transmission electron microscopy},
author = {Nigel D. Browning and Jony Castagna and Angus I. Kirkland and Amirafshar Moshtaghpour and Daniel Nicholls and Alex W. Robinson and Jack Wells and Yalin Zheng},
url = {https://pubs.aip.org/apl/article/122/5/050501/2874756/The-advantages-of-sub-sampling-and-Inpainting-for},
doi = {10.1063/5.0135245},
issn = {0003-6951, 1077-3118},
year = {2023},
date = {2023-01-01},
urldate = {2025-12-19},
journal = {Applied Physics Letters},
volume = {122},
number = {5},
pages = {050501},
abstract = {Images and spectra obtained from aberration corrected scanning transmission electron microscopes (STEM) are now used routinely to quantify the morphology, structure, composition, chemistry, bonding, and optical/electronic properties of nanostructures, interfaces, and defects in many materials/biological systems. However, obtaining quantitative and reproducible atomic resolution observations from some experiments is actually harder with these ground-breaking instrumental capabilities, as the increase in beam current from using the correctors brings with it the potential for electron beam modification of the specimen during image acquisition. This beam effect is even more acute for in situ STEM observations, where the desired outcome being investigated is a result of a series of complicated transients, all of which can be modified in unknown ways by the electron beam. The aim in developing and applying new methods in STEM is, therefore, to focus on more efficient use of the dose that is supplied to the sample and to extract the most information from each image (or set of images). For STEM (and for that matter, all electron/ion/photon scanning systems), one way to achieve this is by sub-sampling the image and using Inpainting algorithms to reconstruct it. By separating final image quality from overall dose in this way and manipulating the dose distribution to be best for the stability of the sample, images can be acquired both faster and with less beam effects. In this paper, the methodology behind sub-sampling and Inpainting is described, and the potential for Inpainting to be applied to novel real time dynamic experiments will be discussed.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Images and spectra obtained from aberration corrected scanning transmission electron microscopes (STEM) are now used routinely to quantify the morphology, structure, composition, chemistry, bonding, and optical/electronic properties of nanostructures, interfaces, and defects in many materials/biological systems. However, obtaining quantitative and reproducible atomic resolution observations from some experiments is actually harder with these ground-breaking instrumental capabilities, as the increase in beam current from using the correctors brings with it the potential for electron beam modification of the specimen during image acquisition. This beam effect is even more acute for in situ STEM observations, where the desired outcome being investigated is a result of a series of complicated transients, all of which can be modified in unknown ways by the electron beam. The aim in developing and applying new methods in STEM is, therefore, to focus on more efficient use of the dose that is supplied to the sample and to extract the most information from each image (or set of images). For STEM (and for that matter, all electron/ion/photon scanning systems), one way to achieve this is by sub-sampling the image and using Inpainting algorithms to reconstruct it. By separating final image quality from overall dose in this way and manipulating the dose distribution to be best for the stability of the sample, images can be acquired both faster and with less beam effects. In this paper, the methodology behind sub-sampling and Inpainting is described, and the potential for Inpainting to be applied to novel real time dynamic experiments will be discussed.
2022
Robinson, Alex W.; Nicholls, Daniel; Wells, Jack; Moshtaghpour, Amirafshar; Kirkland, Angus; Browning, Nigel D.
SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation Journal Article
In: Ultramicroscopy, vol. 242, pp. 113625, 2022, ISSN: 03043991.
@article{robinson_sim-stem_2022,
title = {SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation},
author = {Alex W. Robinson and Daniel Nicholls and Jack Wells and Amirafshar Moshtaghpour and Angus Kirkland and Nigel D. Browning},
url = {https://linkinghub.elsevier.com/retrieve/pii/S0304399122001449},
doi = {10.1016/j.ultramic.2022.113625},
issn = {03043991},
year = {2022},
date = {2022-12-01},
urldate = {2025-12-19},
journal = {Ultramicroscopy},
volume = {242},
pages = {113625},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Ortega, Eduardo; Jonge, Niels De
Sparsity and Noise Effects on the Reconstruction of Subsampled Scanning Transmission Electron Microscopy Data Journal Article
In: Microscopy and Microanalysis, vol. 28, no. S1, pp. 2136–2137, 2022, ISSN: 1435-8115, 1431-9276.
@article{ortega_sparsity_2022,
title = {Sparsity and Noise Effects on the Reconstruction of Subsampled Scanning Transmission Electron Microscopy Data},
author = {Eduardo Ortega and Niels De Jonge},
url = {https://academic.oup.com/mam/article/28/S1/2136/6995641},
doi = {10.1017/S1431927622008273},
issn = {1435-8115, 1431-9276},
year = {2022},
date = {2022-08-01},
urldate = {2025-12-19},
journal = {Microscopy and Microanalysis},
volume = {28},
number = {S1},
pages = {2136–2137},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Nicholls, D.; Robinson, A.; Wells, J.; Moshtaghpour, A.; Bahri, M.; Kirkland, A.; Browning, N.
Compressive Scanning Transmission Electron Microscopy Proceedings Article
In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1586–1590, IEEE, Singapore, Singapore, 2022, ISBN: 9781665405409.
@inproceedings{nicholls_compressive_2022,
title = {Compressive Scanning Transmission Electron Microscopy},
author = {D. Nicholls and A. Robinson and J. Wells and A. Moshtaghpour and M. Bahri and A. Kirkland and N. Browning},
url = {https://ieeexplore.ieee.org/document/9746478/},
doi = {10.1109/ICASSP43922.2022.9746478},
isbn = {9781665405409},
year = {2022},
date = {2022-05-01},
urldate = {2025-12-19},
booktitle = {ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1586–1590},
publisher = {IEEE},
address = {Singapore, Singapore},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Nicholls, Daniel; Wells, Jack; Stevens, Andrew; Zheng, Yalin; Castagna, Jony; Browning, Nigel D.
Sub-Sampled Imaging for STEM: Maximising Image Speed, Resolution and Precision Through Reconstruction Parameter Refinement Journal Article
In: Ultramicroscopy, vol. 233, pp. 113451, 2022, ISSN: 03043991.
@article{nicholls_sub-sampled_2022,
title = {Sub-Sampled Imaging for STEM: Maximising Image Speed, Resolution and Precision Through Reconstruction Parameter Refinement},
author = {Daniel Nicholls and Jack Wells and Andrew Stevens and Yalin Zheng and Jony Castagna and Nigel D. Browning},
url = {https://linkinghub.elsevier.com/retrieve/pii/S0304399121002278},
doi = {10.1016/j.ultramic.2021.113451},
issn = {03043991},
year = {2022},
date = {2022-03-01},
urldate = {2025-12-19},
journal = {Ultramicroscopy},
volume = {233},
pages = {113451},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2021
Ortega, Eduardo; Nicholls, Daniel; Browning, Nigel D.; Jonge, Niels De
High temporal-resolution scanning transmission electron microscopy using sparse-serpentine scan pathways Journal Article
In: Scientific Reports, vol. 11, no. 1, pp. 22722, 2021, ISSN: 2045-2322.
@article{ortega_high_2021,
title = {High temporal-resolution scanning transmission electron microscopy using sparse-serpentine scan pathways},
author = {Eduardo Ortega and Daniel Nicholls and Nigel D. Browning and Niels De Jonge},
url = {https://www.nature.com/articles/s41598-021-02052-1},
doi = {10.1038/s41598-021-02052-1},
issn = {2045-2322},
year = {2021},
date = {2021-11-01},
urldate = {2025-12-19},
journal = {Scientific Reports},
volume = {11},
number = {1},
pages = {22722},
abstract = {Abstract
Scanning transmission electron microscopy (STEM) provides structural analysis with sub-angstrom resolution. But the pixel-by-pixel scanning process is a limiting factor in acquiring high-speed data. Different strategies have been implemented to increase scanning speeds while at the same time minimizing beam damage via optimizing the scanning strategy. Here, we achieve the highest possible scanning speed by eliminating the image acquisition dead time induced by the beam flyback time combined with reducing the amount of scanning pixels via sparse imaging. A calibration procedure was developed to compensate for the hysteresis of the magnetic scan coils. A combination of sparse and serpentine scanning routines was tested for a crystalline thin film, gold nanoparticles, and in an in-situ liquid phase STEM experiment. Frame rates of 92, 23 and 5.8 s
-1
were achieved for images of a width of 128, 256, and 512 pixels, respectively. The methods described here can be applied to single-particle tracking and analysis of radiation sensitive materials.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Scanning transmission electron microscopy (STEM) provides structural analysis with sub-angstrom resolution. But the pixel-by-pixel scanning process is a limiting factor in acquiring high-speed data. Different strategies have been implemented to increase scanning speeds while at the same time minimizing beam damage via optimizing the scanning strategy. Here, we achieve the highest possible scanning speed by eliminating the image acquisition dead time induced by the beam flyback time combined with reducing the amount of scanning pixels via sparse imaging. A calibration procedure was developed to compensate for the hysteresis of the magnetic scan coils. A combination of sparse and serpentine scanning routines was tested for a crystalline thin film, gold nanoparticles, and in an in-situ liquid phase STEM experiment. Frame rates of 92, 23 and 5.8 s
-1
were achieved for images of a width of 128, 256, and 512 pixels, respectively. The methods described here can be applied to single-particle tracking and analysis of radiation sensitive materials.
Scanning transmission electron microscopy (STEM) provides structural analysis with sub-angstrom resolution. But the pixel-by-pixel scanning process is a limiting factor in acquiring high-speed data. Different strategies have been implemented to increase scanning speeds while at the same time minimizing beam damage via optimizing the scanning strategy. Here, we achieve the highest possible scanning speed by eliminating the image acquisition dead time induced by the beam flyback time combined with reducing the amount of scanning pixels via sparse imaging. A calibration procedure was developed to compensate for the hysteresis of the magnetic scan coils. A combination of sparse and serpentine scanning routines was tested for a crystalline thin film, gold nanoparticles, and in an in-situ liquid phase STEM experiment. Frame rates of 92, 23 and 5.8 s
-1
were achieved for images of a width of 128, 256, and 512 pixels, respectively. The methods described here can be applied to single-particle tracking and analysis of radiation sensitive materials.
Lee, Juhan; Nicholls, Daniel; Browning, Nigel D.; Mehdi, B. Layla
Controlling radiolysis chemistry on the nanoscale in liquid cell scanning transmission electron microscopy Journal Article
In: Physical Chemistry Chemical Physics, vol. 23, no. 33, pp. 17766–17773, 2021, ISSN: 1463-9076, 1463-9084.
@article{lee_controlling_2021,
title = {Controlling radiolysis chemistry on the nanoscale in liquid cell scanning transmission electron microscopy},
author = {Juhan Lee and Daniel Nicholls and Nigel D. Browning and B. Layla Mehdi},
url = {https://xlink.rsc.org/?DOI=D0CP06369J},
doi = {10.1039/D0CP06369J},
issn = {1463-9076, 1463-9084},
year = {2021},
date = {2021-01-01},
urldate = {2025-12-19},
journal = {Physical Chemistry Chemical Physics},
volume = {23},
number = {33},
pages = {17766–17773},
abstract = {When high-energy electrons from scanning transmission electron microscope (STEM) are interacting with the liquid, the vast majority of the chemical reactions that are observed are induced by the radiolysis breakdown of the liquid molecules.
,
When high-energy electrons from a scanning transmission electron microscope (STEM) are incident on a liquid, the vast majority of the chemical reactions that are observed are induced by the radiolysis breakdown of the liquid molecules. In the study of liquids, the radiolysis products of pure water are well known, and their rate of formation for a given flux of high-energy electrons has been studied intensively over the last few years for uniform TEM illumination. In this paper, we demonstrate that the temporal and spatial distribution of the electron illumination can significantly affect the final density of radiolysis products in water and even change the type of reaction taking place. We simulate the complex array of possible spatial/temporal distributions of electrons that are accessible experimentally by controlling the size, the scan rate and the hopping distance of the electron probe in STEM mode and then compare the results to the uniformly illuminated TEM mode of imaging. By distributing the electron dose both spatially and temporally in the STEM through a randomised “spot-scan” mode of imaging, the diffusion overlap of the radiolysis products can be reduced, and the resulting reactions can be more readily controlled. This control allows the resolution of the images to be separated from the speed of the induced reaction (which is based on beam current alone) and this facet of the experiment will allow a wide range of chemical reactions to be uniquely tailored and observed in all liquid cell STEM experiments.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
When high-energy electrons from scanning transmission electron microscope (STEM) are interacting with the liquid, the vast majority of the chemical reactions that are observed are induced by the radiolysis breakdown of the liquid molecules.
,
When high-energy electrons from a scanning transmission electron microscope (STEM) are incident on a liquid, the vast majority of the chemical reactions that are observed are induced by the radiolysis breakdown of the liquid molecules. In the study of liquids, the radiolysis products of pure water are well known, and their rate of formation for a given flux of high-energy electrons has been studied intensively over the last few years for uniform TEM illumination. In this paper, we demonstrate that the temporal and spatial distribution of the electron illumination can significantly affect the final density of radiolysis products in water and even change the type of reaction taking place. We simulate the complex array of possible spatial/temporal distributions of electrons that are accessible experimentally by controlling the size, the scan rate and the hopping distance of the electron probe in STEM mode and then compare the results to the uniformly illuminated TEM mode of imaging. By distributing the electron dose both spatially and temporally in the STEM through a randomised “spot-scan” mode of imaging, the diffusion overlap of the radiolysis products can be reduced, and the resulting reactions can be more readily controlled. This control allows the resolution of the images to be separated from the speed of the induced reaction (which is based on beam current alone) and this facet of the experiment will allow a wide range of chemical reactions to be uniquely tailored and observed in all liquid cell STEM experiments.
,
When high-energy electrons from a scanning transmission electron microscope (STEM) are incident on a liquid, the vast majority of the chemical reactions that are observed are induced by the radiolysis breakdown of the liquid molecules. In the study of liquids, the radiolysis products of pure water are well known, and their rate of formation for a given flux of high-energy electrons has been studied intensively over the last few years for uniform TEM illumination. In this paper, we demonstrate that the temporal and spatial distribution of the electron illumination can significantly affect the final density of radiolysis products in water and even change the type of reaction taking place. We simulate the complex array of possible spatial/temporal distributions of electrons that are accessible experimentally by controlling the size, the scan rate and the hopping distance of the electron probe in STEM mode and then compare the results to the uniformly illuminated TEM mode of imaging. By distributing the electron dose both spatially and temporally in the STEM through a randomised “spot-scan” mode of imaging, the diffusion overlap of the radiolysis products can be reduced, and the resulting reactions can be more readily controlled. This control allows the resolution of the images to be separated from the speed of the induced reaction (which is based on beam current alone) and this facet of the experiment will allow a wide range of chemical reactions to be uniquely tailored and observed in all liquid cell STEM experiments.
2020
Nicholls, Daniel; Lee, Juhan; Amari, Houari; Stevens, Andrew; Layla, B. Mehdi; Browning, Nigel
Distributing the Electron Dose to Minimise Electron Beam Damage in Scanning Transmission Electron Microscopy Journal Article
In: Microscopy and Microanalysis, vol. 26, no. S2, pp. 3070–3071, 2020, ISSN: 1431-9276, 1435-8115.
@article{nicholls_distributing_2020,
title = {Distributing the Electron Dose to Minimise Electron Beam Damage in Scanning Transmission Electron Microscopy},
author = {Daniel Nicholls and Juhan Lee and Houari Amari and Andrew Stevens and B. Mehdi Layla and Nigel Browning},
url = {https://academic.oup.com/mam/article/26/S2/3070/6893780},
doi = {10.1017/S1431927620023727},
issn = {1431-9276, 1435-8115},
year = {2020},
date = {2020-08-01},
urldate = {2025-12-19},
journal = {Microscopy and Microanalysis},
volume = {26},
number = {S2},
pages = {3070–3071},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Nicholls, Daniel; Lee, Juhan; Amari, Houari; Stevens, Andrew J.; Mehdi, B. Layla; Browning, Nigel D.
Minimising damage in high resolution scanning transmission electron microscope images of nanoscale structures and processes Journal Article
In: Nanoscale, vol. 12, no. 41, pp. 21248–21254, 2020, ISSN: 2040-3364, 2040-3372.
@article{nicholls_minimising_2020,
title = {Minimising damage in high resolution scanning transmission electron microscope images of nanoscale structures and processes},
author = {Daniel Nicholls and Juhan Lee and Houari Amari and Andrew J. Stevens and B. Layla Mehdi and Nigel D. Browning},
url = {https://xlink.rsc.org/?DOI=D0NR04589F},
doi = {10.1039/D0NR04589F},
issn = {2040-3364, 2040-3372},
year = {2020},
date = {2020-01-01},
urldate = {2025-12-19},
journal = {Nanoscale},
volume = {12},
number = {41},
pages = {21248–21254},
abstract = {Determining the optimum electron dose distribution for damage mitigated scanning transmission electron microscopy imaging using subsampling and image inpainting.
,
Beam damage caused during acquisition of the highest resolution images is the current limitation in the vast majority of experiments performed in a scanning transmission electron microscope (STEM). While the principles behind the processes of knock-on and radiolysis damage are well-known (as are other contributing effects, such as heat and electric fields), understanding how and especially when beam damage is distributed across the entire sample volume during an experiment has not been examined in detail. Here we use standard models for damage and diffusion to elucidate how beam damage spreads across the sample as a function of the microscope conditions to determine an “optimum” sampling approach that maximises the high-resolution information in any image acquisition. We find that the standard STEM approach of scanning an image sequentially accelerates damage because of increased overlap of diffusion processes. These regions of accelerated damage can be significantly decelerated by increasing the distance between the acquired pixels in the scan, forming a “spotscan” mode of acquisition. The optimum distance between these pixels can be broadly defined by the fundamental properties of each material, allowing experiments to be designed for specific beam sensitive materials. As an added bonus, if we use inpainting to reconstruct the sparse distribution of pixels in the image we can significantly increase the speed of the STEM process, allowing dynamic phenomena, and the onset of damage, to be studied directly.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Determining the optimum electron dose distribution for damage mitigated scanning transmission electron microscopy imaging using subsampling and image inpainting.
,
Beam damage caused during acquisition of the highest resolution images is the current limitation in the vast majority of experiments performed in a scanning transmission electron microscope (STEM). While the principles behind the processes of knock-on and radiolysis damage are well-known (as are other contributing effects, such as heat and electric fields), understanding how and especially when beam damage is distributed across the entire sample volume during an experiment has not been examined in detail. Here we use standard models for damage and diffusion to elucidate how beam damage spreads across the sample as a function of the microscope conditions to determine an “optimum” sampling approach that maximises the high-resolution information in any image acquisition. We find that the standard STEM approach of scanning an image sequentially accelerates damage because of increased overlap of diffusion processes. These regions of accelerated damage can be significantly decelerated by increasing the distance between the acquired pixels in the scan, forming a “spotscan” mode of acquisition. The optimum distance between these pixels can be broadly defined by the fundamental properties of each material, allowing experiments to be designed for specific beam sensitive materials. As an added bonus, if we use inpainting to reconstruct the sparse distribution of pixels in the image we can significantly increase the speed of the STEM process, allowing dynamic phenomena, and the onset of damage, to be studied directly.
,
Beam damage caused during acquisition of the highest resolution images is the current limitation in the vast majority of experiments performed in a scanning transmission electron microscope (STEM). While the principles behind the processes of knock-on and radiolysis damage are well-known (as are other contributing effects, such as heat and electric fields), understanding how and especially when beam damage is distributed across the entire sample volume during an experiment has not been examined in detail. Here we use standard models for damage and diffusion to elucidate how beam damage spreads across the sample as a function of the microscope conditions to determine an “optimum” sampling approach that maximises the high-resolution information in any image acquisition. We find that the standard STEM approach of scanning an image sequentially accelerates damage because of increased overlap of diffusion processes. These regions of accelerated damage can be significantly decelerated by increasing the distance between the acquired pixels in the scan, forming a “spotscan” mode of acquisition. The optimum distance between these pixels can be broadly defined by the fundamental properties of each material, allowing experiments to be designed for specific beam sensitive materials. As an added bonus, if we use inpainting to reconstruct the sparse distribution of pixels in the image we can significantly increase the speed of the STEM process, allowing dynamic phenomena, and the onset of damage, to be studied directly.
