Introduction to electron ptychography for materials scientists

INTRODUCTION

From 2000 to 2002, Kuo published a series of six articles titled "A Brief History of Metallography" in the journal "Materials Science and Engineering" sponsored by Zhejiang University^[1–6]. These articles vividly recount the stories of materials scientists who, over the past two hundred years, have continued to explore the microscopic world of materials through the development of methods such as optical microscopy, X-ray diffraction, and electron microscopy, and laying the foundations of materials science. Materials are the carriers of human civilization, and the scientific stories behind them can be said to unfold on a magnificent scale. The last article in this series was published in 2002, at a time when the resolution of electron microscopes was approaching 1.2 Å. Kuo concludes his article by writing, "Since a two-dimensional electron micrograph corresponds to the projection of a crystal in the direction of electron beam, and atoms inevitably overlap in the direction of projection, there is also a need to develop electron microscopes with higher resolution (e.g., 0.1 nm)."

Over the past two decades, with the development of aberration-corrected electron microscopes^[7,8], sub-angstrom resolution electron microscopy with resolutions exceeding 0.1 nm has rapidly become widespread [9,10]. In recent years, the development of electron ptychography^[11–15], which is a computational imaging method that utilizes the rich information in four-dimensional scanning transmission electron microscopy (4D-STEM) datasets, has pushed the resolution of electron microscopes into deep-sub-angstrom resolution^[16–18]. Following the principle of "solving real problems with real solutions", this article first discusses the challenges that microstructural analysis in materials science (the broad sense of metallography) still faces in the era of aberration-corrected electron microscopy, then introduces the basic principles and main algorithms of electron ptychography, and finally illustrates the characteristics of electron ptychography in addressing the aforementioned challenges through application examples.

We would like to make some notes on the organization of this paper.

First, as the title suggests, this paper is written as an invitation to materials scientists. It is not intended to be an exhaustive review on what has been done in the fields of ptychography and 4D-STEM, both of which cover a much wider range of topics than this paper.

Second, there are no mathematical formulas in the main body of this paper. As a computational imaging method, ptychography-related literature is inevitably filled with mathematical formulas, which can easily obscure the core ideas behind the algorithms and may not be friendly to beginners. Simplicity is greatness. We believe that the main algorithms can be explained clearly without the need for formulas. An appendix is given at the end of the paper for those interested in the mathematical details of ptychography.

Third, this paper is an update of the original Chinese version^[19] written in commemoration of the 100th anniversary of K.H. Kuo's birth. The figures are sourced from our own published works on crystalline materials for illustrative purposes only and should not be regarded as the best examples.

Challenges in atomic resolution imaging

Modern electron microscopes combine diffraction, imaging, and spectroscopy, serving as a comprehensive platform for materials structure analysis. External stimuli or environments such as force, heat, electricity, magnetism, light, gas, and liquid can also be introduced to turn the sample holder in an electron microscope into a miniature physics or chemistry laboratory^[20,21]. Each technique or environment has corresponding technical specifications. However, if a single metric were to be chosen to describe the capability of an electron microscope, it would undoubtedly be spatial resolution. This is also the primary strength that distinguishes electron microscopy from other methods of structural analysis.

Before the implementation of aberration correction, the point resolution of high-resolution electron microscopes is about 2 Å. While this can resolve atomic structure of most crystals in the low-index zone axes, the accuracy of atomic positions is relatively low, around 20 pm, which is larger than the atomic displacement induced by defects. Therefore, conventional high-resolution electron microscopes are rarely used for quantitative analysis of atomic displacements. Aberration correction achieved at the end of the 20th century has overcome the main obstacle limiting the resolution of electron microscopes, namely, the aberrations of the objective lens, increasing the point resolution of electron microscope by about three times, currently around 0.5-0.6 Å. The high resolution not only resolves atoms with smaller spacings, but also significantly improves the signal-to-noise ratio (SNR) of the image, contributing to higher accuracy of quantitative analysis. At the same time, aberration correction essentially eliminates the contrast delocalization, giving a better correspondence between atomic columns in the specimen and image spots. The combination of the two improves the measurement accuracy of the atomic positions by a factor of about three as well, approaching 5 pm, which is already significantly smaller than the defect-induced atomic displacements. Therefore, aberration-corrected electron microscopes have been widely used in quantitative studies of materials surfaces, interfaces, defects, precipitates, ferroelectric domains, and other microstructures, and are gradually becoming a standard tool in electron microscopy laboratories^[9,10].

So, is aberration-corrected electron microscopy the ideal high-resolution imaging method? Certainly not. In fact, even with aberration-corrected electron microscopy, there are still some unresolved issues when analyzing materials' microstructures at the atomic scale.

The first issue is residual aberrations. No matter how carefully the operators tune the electron microscope, aberrations of the objective lens cannot be completely eliminated. Some aberrations are intentionally retained to form an optimized contrast transfer function (CTF) through their combination, such as negative spherical aberration imaging techniques^[22]. Some residual aberrations are limited by the precision of the instrument or the skill of the operator. Either way, the residual aberration will cause the CTF to deviate from ideal conditions.

The second challenge is the misorientation between the electron beam and the zone axis of crystals. Atomic resolution imaging requires the incident electron beam to be parallel to the zone axis of the sample. Even a milliradian deviation can degrade the image quality, introduce artifacts, and reduce measurement accuracy. Aberration-corrected electron microscopes, aiming for higher resolution, impose stricter requirements on the crystal tilt. For perfect single crystals, precise zone axis conditions can be achieved through careful tilt of the sample (provided that the sample can endure electron irradiation). However, more attention is often paid to lattice defects, such as dislocations, grain boundaries, and ferroelectric domain boundaries, where the crystal orientation actually varies continuously, and where a region is in a precise zone-axis condition, the neighboring regions must not be. Consequently, achieving precise zone-axis conditions and high-quality atomic-scale imaging across the entire region of interest becomes impossible.

The third issue is surface damage. The thickness of samples for high-resolution transmission electron microscopy (TEM) should be less than 50 nm. Thinning methods, such as ion milling or chemical polishing, induce surface damage. The surface of metals may undergo oxidation or corrosion. Most sample surfaces may also adsorb organic contaminants. The surface layer differs from the bulk structure, and their diffraction and image signals are superimposed, complicating the analysis of intrinsic structural information of the sample.

The fourth challenge is irradiation damages. To achieve sufficient penetration ability and shorter wavelengths, high-resolution electron microscopes typically operate at a voltage of 200 to 300 kV. These high-energy incident electrons inevitably cause irradiation damage to the samples. Most metals, ceramics, and semiconductors exhibit strong radiation resistance, enabling detailed studies of their microstructures at the atomic scale. However, many materials systems of interest are beam sensitive, such as zeolites, metal-organic frameworks (MOFs), polymers, and halide perovskites. They lag significantly behind metals, ceramics, and semiconductors in microstructural studies at the atomic scale.

The above analysis shows that the factors limiting the spatial resolution and atomic positional accuracy of aberration-corrected electron microscopes are more related to the electron scattering process of the sample than to the design and manufacture of the instrument. Therefore, the solution to the above dilemma requires that the scattering process of electrons in the sample be described as accurately as possible.

Electron ptychography collects a two-dimensional diffraction pattern at each scanning position, providing a comprehensive set of electron scattering information. Each dataset contains ten thousand to one million diffraction patterns. Compared to a typical image size of around 10 MB in conventional electron microscopy, the dataset size used for electron ptychography increases by at least 100 times. Based on this massive data, state-of-the-art ptychography algorithms make it possible to recover the electron probe function and the electron scattering processes in the sample, thereby resolving residual aberrations and crystal tilt issues, substantially improving the spatial resolution of aberration-corrected electron microscopy. Moreover, the reconstruction results have depth resolution in the direction of electron beam propagation, which makes it easy to remove the effect of surface damage layers and obtain the intrinsic structural information of the sample. As the detector records nearly all scattered electrons, electron ptychography exhibits high dose efficiency, reducing electron irradiation damages and is particularly suitable for low-dose imaging. Additionally, electron ptychography is characterized by high phase accuracy and orientation imaging, and other characteristics. Thus, electron ptychography provides a comprehensive solution for addressing the aforementioned challenges in microstructure analysis.

However, it is also necessary to point out that the computational reconstruction process of electron ptychography is currently time-consuming, with typical iterative algorithms requiring time in hours, and is still some way from real-time imaging. However, it is reasonable to believe that with the development of information technology and reconstruction algorithms, the cost of electron ptychography in terms of data volume and reconstruction time will decrease while its functionality will become increasingly powerful. This will lead to a vigorous revitalization of electron ptychography.

The BASIC IDEA OF ELECTRON PTYCHOGRAPHY

Both ptychography^[11–15,23] and holography^[24–27] are categorized as phase retrieval or phase imaging techniques, which extract phase information from the intensity recorded in experiments. The history of both techniques is also intriguing: they both began with electron microscopy and initially succeeded in other fields. Currently, ptychography is extensively applied in visible light and X-ray imaging^[14,15,23]. Both holography and ptychography are also umbrella terms for a large class of methods, rather than one particular method. For instance, Cowley once identified 20 different forms of holography^[25]. Ptychography appeared more than two decades after holography, but has also taken many forms, such as Fourier ptychography^[28,29], Bragg ptychography^[30,31], and near-field ptychography^[32,33].

Set within the context of high-resolution electron microscopy, this article introduces the form of electron ptychography best suited for TEM, that is, combining STEM with coherent diffraction imaging. It involves several critical components: a coherent electron beam, scanning of the electron beam, overlapping of neighboring beam spots, and 2D diffraction intensities. Therefore, electron ptychography can be defined as "overlapping scanning coherent diffraction imaging". It is both a phase retrieval method based on scanning data and a computational imaging method based on diffraction data.

Since electron ptychography can be viewed as a combination of STEM and coherent diffraction imaging, it is insightful to understand electron ptychography from these two perspectives separately. Figure 1 compares relevant experimental methods and corresponding detector setups of different STEM imaging techniques. The background of the image is a convergent electron diffraction pattern. Due to the large convergence angle (25 mrad), the diffraction disks overlap with one another, making it hard to distinguish individual disks. The bright-field disk at the center of the diffraction pattern is actually composed of a superposition of the transmitted disk and numerous surrounding diffraction disks, with its boundary defined by the convergence angle. This diffraction pattern with a large convergence angle is known as a Ronchigram.

Figure 1. Different imaging methods of STEM and the corresponding detector settings. Typically, N ≥ 128^[19].

The commonly used annular dark-field (ADF) imaging technique collects the scattered electrons outside the bright-field disk through an annular detector. The intensity value of each pixel in the ADF image is determined by the integration of the electrons scattered onto the annular detector within the dwell time at each scanning position. In contrast, the annular bright-field (ABF) imaging technique places the detector inside the bright-field disk^[34]. Differential phase contrast (DPC) and integrated DPC (iDPC) technologies utilize 4-segmented detectors, where electrons within each segment are summed up independently^[35–37]. To capture more signals in reciprocal space, detectors can be segmented into multiple regions, such as 8 segments^[38], 16 segments^[39], 40 segments^[40], and so forth. ADF (ABF) and DPC (iDPC) detectors reduce the entire 2D diffraction pattern at each scanning position to one and 4 (or more, depending on the number of detector segments) pixels, respectively.

The widely discussed 4D-STEM technique captures complete diffraction patterns at each scanning position by using pixelated detectors^[41]. Electron ptychography is a form of the 4D-STEM technique. Commercial pixelated detectors typically feature 128 $$ \times $$ 128 or more pixels^[42–49], leading to a significant increase of data size compared to ADF and DPC technologies. The term "pixelated" emphasizes that diffraction patterns are finely segmented in contrast to ADF and DPC detectors, indicating a high sampling rate in reciprocal space. In fact, all image detectors are pixelated, typically having 2, 048 $$ \times $$ 2, 048 pixels. However, many of these detectors are not single-electron sensitive and the accuracy of the recorded diffraction patterns is not high. Moreover, if the rate at which diffraction patterns are recorded per second is limited, sample drift becomes a significant issue during extended acquisition times.

Now we consider the ptychography from the perspective of coherent diffraction imaging^[50,51]. Diffraction provides information in reciprocal space, while imaging conveys information in real space. Coherent diffraction imaging starts with a diffraction pattern and uses computational reconstruction to obtain an image. Typically, the information in real space and reciprocal space is equivalent, and they are related to each other through the Fourier transform. However, this presupposes that the information is complete, encompassing both intensity and phase. Yet, the recorded diffraction patterns only contain intensity data, representing the squared modulus of the electron wave function, with the phase information lost. Unfortunately, the structural information of the object is more encoded in the phase of the diffracted wave, so the half that is lost happens to be more important. This is known as the famous phase problem in structural analysis.

Coherent diffraction imaging is an important solution to the phase problem of nonperiodic structures^[50,51]. Common phase retrieval algorithms require two two-dimensional intensity distributions, known as intensity constraints^[50,52,53]. The first intensity constraint, referred to as the Fourier constraint, is in reciprocal space, i.e., the diffraction pattern. To recover the lost phase information from the diffraction pattern, oversampling of the diffraction pattern is necessary^[54]. After all, you cannot make anything out of nothing. The second intensity constraint, known as the isolated sample constraint or support constraint, is in real space and requires that the object or illumination area be of finite size (such as small particles or small probe areas). The reconstruction process begins with a random phase, iteratively generating the object function and its diffraction pattern through Fourier transforms, while continuously applying these two constraints until convergence. Ultimately, the reconstruction yields the amplitude and phase of the object function, where its Fourier transform corresponds to the amplitude and phase of the diffraction pattern, satisfying both the isolated sample constraint and Fourier constraint.

The isolated sample constraint is a natural condition for small particles. However, most samples are extended, making the isolated sample constraint impractical. Ptychography retains the Fourier constraint but replaces the isolated sample constraint with the overlap constraint that utilizes the overlap of probe illumination regions between adjacent scan positions. Implementing the overlap constraint can be achieved in two ways: scanning the probe over a stationary sample or translating the sample relative to a stationary probe; both methods are equivalent. X-ray ptychography primarily translates the sample, while electron microscopy usually shifts the electron beam.

The concept of ptychography is explained above from the perspectives of STEM and coherent diffraction imaging. It is worth noting that Hoppe introduced the idea of ptychography in 1969 not specifically for STEM nor coherent diffraction imaging, but to address the phase problem in crystal diffraction^[11–13].

The core idea involves two aspects.

First, interference occurs between adjacent diffraction disks. By measuring the intensity in the interference region, the phase difference between neighboring diffraction disks can be determined. This requires a large convergence angle of the electron beam to ensure the adjacent diffraction disks overlap. Additionally, the coherence of the electron probe must be sufficient to fill the diffraction disks coherently for the interference to occur in the overlap region, resulting in the superposition of complex amplitudes, rather than simply intensity superposition.

Second, there is a relative shift between the electron beam and the sample. As mentioned above, measuring the interference intensity of the adjacent diffraction beams allows the determination of their phase difference. However, this difference is not unique; it can be positive or negative. When the electron beam is shifted relative to the sample, each diffraction beam gains an additional phase, with different diffraction beams acquiring different additional phases. As a result, the intensities in the interference region also vary. By analyzing the intensity in the interference regions before and after the shift, the phase difference between adjacent diffraction beams can be uniquely determined in terms of magnitude and sign.

Since the phase difference between any neighboring diffracted beams is determined, by passing them through each other, the phase difference between all diffracted beams is also determined. The phase problem is solved.

For periodic structures, the diffracted beams are discrete and limited in number. This is actually a typical crystallographic problem. The required number of probe positions is not very high, and two are theoretically sufficient. Rodenburg and Nellist^[55] applied Hoppe's ptychography concept to determine the phase of each diffraction beam in the [110] zone axis of silicon in a line-scan manner with 64 probe positions. They reconstructed an image of the dumbbell structure of silicon (spacing of 1.36 Å).

However, microscopy is concerned mainly with non-periodic structures. The diffracted beams are no longer discrete diffraction spots, but continuously varying, and it is no longer practical to list explicitly all the diffracted beams and the phase differences between them. However, the idea of ptychography on the interference of diffracted beams and the real-space translation is still applicable and is implicit in various reconstruction algorithms and experimental methods.

BRIEF HISTORY OF PTYCHOGRAPHY ALGORITHMS

Ptychography originates from the fundamental idea proposed by Hoppe in 1969^[11–13]. Today, ptychography differs from Hoppe's original idea, both in experimental and algorithmic aspects^{[14,15,23,56]}. As previously discussed, ptychography in the field of electron microscopy manifests as a combination of STEM and coherent diffraction imaging, known as "overlapping scanning coherent diffraction imaging". Theoretically, the number of scanning positions (M $$ \times $$ M) has no upper limit, but it is practically constrained by sample stability and computational resources, typically with 100 $$ \leq $$ M $$ \leq $$ 1,000. To extract structural information from over ten thousand diffraction patterns, ptychography algorithms have undergone a series of developments.

The widely used ptychography algorithms can be categorized into two classes: iterative and non-iterative. Non-iterative algorithms primarily include WDD (Wigner-distribution deconvolution)^[57] and SSB (single-side band)^[58]. Their advantages lie in clear analytical form and high speed, enabling real-time reconstruction^[59,60]. However, they require high sampling rates in real space, i.e., small scan steps. While these methods can image thick samples in experiment^[61], a strict limitation exists theoretically on sample thickness to satisfy the phase object or weak phase object approximations for proper contrast interpretation. In contrast, iterative algorithms treat the recovery of the electron probe and object functions as a multi-parameter optimization problem. They can effectively handle experimental conditions such as probe aberrations, sample tilt, surface damage, multiple scattering of thick objects, probe and sample drift during scanning, and other complexities in the electron scattering process. Although they are computationally intensive and time-consuming, and are still far from real-time reconstruction, they are more flexible than the non-iterative methods. Here, we focus on iterative algorithms; readers interested in non-iterative methods are encouraged to refer to relevant literature^{[14,15,55,57–59,61–64]}.

Various forms of iterative algorithms have been developed for ptychography. The commonly used algorithms in electron ptychography can be categorized into two types. The first type includes the "ptychographic iterative engine" algorithm (PIE) proposed by Rodenburg and Faulkner in 2004^[65] and its derived forms^[66–68]. The PIE algorithm is a natural extension of traditional coherent diffraction imaging algorithms in the framework of overlapping scanning experiments. It is conceptually clear and easy to understand. The second type involves non-linear optimization algorithms, treating ptychography as a multi-parameter optimization problem based on multiple diffraction patterns, allowing for a more flexible implementation^[18,69,70].

Figure 2 illustrates the PIE algorithm, focusing specifically on the Fourier and overlap constraints. For simplicity, the example utilizes only two scan positions. The extension to multiple scan positions is obvious.

Figure 2. Illustration of the PIE algorithm^[19]. The index 1 and 2 stand for adjacent scan positions sharing an overlapped illuminated region.

For coherent diffraction imaging of thin samples, it is assumed that the exit wave function after the probe-object interaction is the product of the probe function $$ P(\boldsymbol{r}) $$ and the object function $$ O(\boldsymbol{r}) $$. After initializing the two functions, the iteration proceeds as follows:

(1) Compute the diffraction pattern $$ S_1 $$ by taking the Fourier transform of the exit wave of subregion $$ s_1 $$ of the object function $$ O(\boldsymbol{r}) $$.

(2) Replace the amplitude of $$ S_1 $$ with the square root of the experimental diffraction intensity $$ D_1 $$ of subregion $$ s_1 $$, keeping the phase, to obtain $$ S'_1 $$. This step corresponds to the Fourier constraint.

(3) Perform the inverse Fourier transform on $$ S'_1 $$ to retrieve $$ s'_1 $$.

(4) As there is an overlap between subregions $$ s_1 $$ and $$ s_2 $$, the update in $$ s'_1 $$ means the update of $$ s_2 $$. This is the overlap constraint.

(5) Repeat steps 1-4 for subregion $$ s_2 $$ and all the other subregions.

(6) Return to step 1 to start a new round of computations until convergence.

Convergence occurs when all computed diffraction patterns differ sufficiently little from the experimental ones and change sufficiently little as the iteration continues. The object function at this point is the result of the reconstruction, including amplitude and phase.

The PIE algorithm demonstrated above only updates the object function $$ O(\boldsymbol{r}) $$ during the reconstruction. If the probe function $$ P(\boldsymbol{r}) $$ is accurately known, the reconstructed object function will also be accurate. However, in most cases, the probe function is unknown. Therefore, reconstruction can only assume an approximate probe function, resulting in an object function of low quality. To address this issue, Thibault et al. proposed in 2008 a difference map method that updates both the object function and probe function simultaneously^[71]. In 2009, Maiden and Rodenburg extended the PIE algorithm to update the probe function, known as the ePIE, significantly enhancing the quality of reconstructed object function^[66]. More importantly, the object function thus obtained is theoretically independent of the probe function, allowing zero-aberration imaging and solving the residual aberration problem in the challenges mentioned above.

Due to the strong interaction between electrons and samples, multiple scattering or dynamic diffraction cannot be ignored in electron microscopy. It usually leads to undesirable effects in diffraction, imaging, and spectroscopy. For example, dynamical electron diffraction prevents direct Fourier analysis in crystallography. The multislice method^[72,73] is commonly used to deal with dynamic diffraction in simulations of atomic-resolution images. Maiden, Humphry, and Rodenburg introduced this method into ptychography in 2012, addressing not only dynamical diffraction problem but also giving some resolution in the depth direction^[68].

Surface slices can be employed to analyze surface structures or surface damage mechanisms. If only the microstructure inside the bulk is of interest, it is possible to analyze only the central slices of the sample, thus addressing the surface damage problem among the challenges mentioned above. It is worth mentioning that ptychography only requires a single scan to obtain structural information at different depths. Although ADF and iDPC techniques also offer depth resolution, they require multiple scans with electron beams focusing at different depths, significantly increasing the electron dose.

While multiple scattering usually leads to undesirable effects in electron diffraction and imaging, a method called bright-field ptychography^[74] is developed recently that employs multiple scattering to improve the spatial resolution. An information limit of 15 pm is achieved using only the bright-field disk (i.e., small numerical aperture) in the diffraction pattern, indicating that the high-frequency information has been encoded in the bright-field disc due to the multiple scattering of electrons in the objects, making it possible to break the diffraction limit of imaging via bright-field ptychography^[74].

The next important advance in algorithms is the introduction of the mixed-state method into ptychography^[75–77]. Ptychography, as a coherent diffraction imaging method, implies that the illumination is perfectly coherent. However, this assumption is impractical as a perfectly coherent illumination does not exist in experiments. Thibault and Menzel proposed a mixed-state method in 2013^[75–77], considering partially coherent illumination as the incoherent superposition of orthogonal coherent illuminations, significantly improving cality^[76–78].

Representing the probe and object functions as two-dimensional arrays of pixels, all of the above algorithms are proposed in the context of continuum structures. For the convenience of the following discussion, these algorithms are collectively referred to as conventional pixelated ptychography (CPP). While they can be applied equivalently to continuum and discrete atomic structures, algorithms specialized for atomic structures have the potential to be advantageous. The authors’ group started a project in 2019 on local-orbital ptychography (LOP), local-orbital tomography^[79,80], local-orbital ptychographic tomography, and local-orbital exitwave reconstruction (the latter three are not covered in this paper) specifically for the atomic and electronic structures of materials. The term local-orbital is traced back to a method called APW+lo proposed around 2000^[81], when the senior author of this paper experienced a transition from WIEN97 to WIEN2k in his DFT calculations and recognized the advantages of locality in the atomic world.

LOP describes the object as the sum of discrete atomic-orbital-like functions and the probe in terms of aberration functions [Figure 3]. As the name indicates, LOP was designed for the reconstruction of atomic orbitals and charge densities in materials based on 4D-STEM datasets. Neglecting the orbital angular momentum of atomic orbitals and using only spherical functions to represent atoms, LOP is reduced to a method of reconstructing atomic structure of materials. Using LOP in this sense, the information limit of microscopic imaging was improved to 14 pm in 2023^[18]. By using analytic functions rather than pixel arrays to describe the object and probe functions, LOP not only describes the latter more accurately, but also reduces substantially the number of optimization parameters and improves the SNR, resulting in higher spatial resolution and phase accuracy. In addition, LOP decomposes the total phase into elemental components, thus revealing the elemental dependence of the information limit.

Figure 3. Illustration of the different ideas of conventional pixelated ptychography and local orbital ptychography.

Another appealing feature of LOP is that the partial coherence can be well handled by smearing the diffraction intensities (see Appendices for details), making it unnecessary to include multiple states for the probe.

It is generally assumed that the probe positions form a regular grid as it scans across the sample. In experiments, however, this is not the case due to the sample drift or scanning errors. An important condition for the success of a ptychographic reconstruction is to find the exact positions of the probe on the sample. There are mainly two kinds of probe positioning or position correction approaches: One pre-corrects the scan positions before reconstruction^[63,82–84], while the other is integrated into iterative algorithms that optimize scan positions based on cross-correlation^[85] or gradient descent^[86–89].

As mentioned earlier, mistilt between the zone axis of the crystalline sample and the incident electron beam poses a big challenge in atomic-resolution imaging, greatly degrading the spatial resolution and measurement accuracy of atomic positions. A post-processing method was proposed to align the slices in-plane to correct the misorientation after multislice reconstructions^[17]. However, this method requires a good convergence, which is hard to achieve with large mistilt angles. The authors of this paper introduced a more efficient and robust method to correct the misorientation^[90]. For convenience, the new method is referred to as adaptive-propagator ptychography (APP), while the traditional multislice ptychography is termed fixed propagator ptychography (FPP). Simulation results demonstrate that in the presence of misorientation, due to the limited depth resolution of ptychography (currently around 3-4 nm), atomic columns inside the individual slices reconstructed by FPP are misaligned, and post-processing can only correct the inter-slice misalignment. On the contrary, the APP method effectively aligns the crystal zone axis with the electron beam direction, ensuring that the reconstruction result is free from both inter- and intra-slice misalignment, thereby achieving complete correction of misorientation. Simulation and experimental results indicate that even in the presence of noticeable misorientation in the diffraction patterns, the APP method enables deep sub-angstrom resolution imaging and picometer accuracy in atomic positions, laying a foundation for the widespread application of electron ptychography in materials science.

Figure 4 summarizes the main algorithms used in electron ptychography.

Figure 4. Illustration of various algorithms related to electron ptychography.

It is worth noting that the ptychographic reconstruction directly gives the amplitude and phase of the sample transmission function, which can be considered as the object function itself and is therefore directly interpretable. In contrast, most other electron microscopy methods give an image that is not the object function itself, but rather the convolution of the exit wave function with some function describing the electron optical system, thus reducing the interpretability of the image. For example, a conventional high-resolution image with parallel illumination is the intensity of the convolution of the exit wave and a point spread function, and a high-angle annular dark field (HAADF) image is the convolution of the intensity of the exit wave and the intensity of a probe function.

SPATIAL RESOLUTION AND INFORMATION LIMIT

The deep sub-angstrom resolution is the most notable feature of electron ptychography compared to other electron microscopy imaging techniques. Before delving into the discussion of deep sub-angstrom resolution, it is necessary to clarify a question: what exactly is meant when talking about deep sub-angstrom resolution?

There exists a resolution limit in microscopic imaging due to wave diffraction, known as the diffraction limit. In the history of microscopy, people have given different values for the diffraction limit based on varying criteria. Common examples include the Rayleigh resolution (0.61 $$ \lambda $$/NA), Abbe resolution (0.5 $$ \lambda $$/NA), and Sparrow resolution (0.47 $$ \lambda $$/NA). NA refers to the numerical aperture, which equals the collection half-angle $$ \beta $$ of the objective lens for conventional TEM or the convergence half-angle $$ \alpha $$ for the condenser lens for STEM. These limits define the ability to resolve adjacent points in a sample. Super-resolution refers to achieving spatial resolution beyond the diffraction limit through various methods, such as structured illumination or stimulated emission depletion in optical microscopy^[91].

For traditional imaging techniques such as HAADF and iDPC in STEM, the resolution is mainly determined by the focused probe size, which depends on the convergence semi-angle $$ \alpha $$ and the aberrations of the condenser lens. For electron ptychography, where the probe is often defocused, the convergence semi-angle $$ \alpha $$ of the condenser lens is not the primary resolution-limiting factor and is, therefore, not suitable to be used as the numerical aperture. As a coherent diffraction imaging technique, ptychography collects diffraction patterns, and thus, the collection semi-angle $$ \beta $$ corresponds to the highest angle in the diffraction pattern. The detected maximum electron scattering angle, however, is actually $$ \alpha+\beta $$, which is a more suitable parameter for the numerical aperture. The selection of which angle to be used as the numerical aperture determines the nominal diffraction limit.

Point resolution in conventional high-resolution phase-contrast electron microscopy is well defined as the spatial frequency of the intersection of the phase-CTF and the horizontal axis. However, there are multiple choices for the Scherzer defocus, corresponding to different CTFs, leading to different numerical values of point resolution. High-resolution phase-contrast images typically contain high-frequency information beyond the point resolution. However, these high-frequency components may not contribute to the resolution as they could result from non-linear components due to the interference between diffraction beams or are not directly interpretable due to oscillations in the transfer functions.

All of the above can be confusing for beginners. Furthermore, when the spatial resolution of the electron microscope exceeds the size of the atoms themselves, the point resolution itself is no longer suitable to describe the spatial resolution. For instance, when the full width at half maximum (FWHM) of the electrostatic potential of an atom is 0.3 Å, and the projected spacing between two atoms is only 0.2 Å, these two atoms cannot be resolved even with an ideal microscope (with a point spread function as the $$ \delta $$ function).

So, what is the best metric to describe the spatial resolution obtained in ptychography reconstructions? We believe it is the information limit, i.e., the spatial frequency cutoff in the Fourier transform of the reconstructed phase. This has both reliability and clarity reasons.

In terms of reliability, multislice ptychography eliminates the effects of dynamical diffraction, avoiding the artifacts caused by non-linear components. All the frequencies up to the information limit represent the true structural details of the material. Moreover, as ptychography reconstructs the object function itself, eliminating the influence of the aberrations of the condenser or objective lens and the information up to the highest spatial frequency in the image can be directly interpreted.

In terms of clarity, the definition of the information limit is clear and unambiguous. It is independent of the specific criterion of spatial resolution, the choice of numerical aperture (whether $$ \alpha $$, $$ \beta $$, or $$ \alpha+\beta $$), and the sample structure. There is no need to search for the closest pair of image points that happen to be resolved in the reconstructed phase.

Returning to the question above, the answer is that by referring to deep sub-angstrom resolution, we mean the information limit.

MAIN FEATURES OF ELECTRON PTYCHOGRAPHY

This section introduces the main features of electron ptychography, especially APP and LOP, including high spatial resolution, high phase accuracy, high dose efficiency, depth resolution, and orientation imaging.

High spatial resolution

On a modern transmission electron microscope equipped with an aberration corrector, the information limit of electron ptychography reaches 39 pm in 2018^[16], 23 pm in 2021^[17], and 14 pm in 2023^[18]. On an uncorrected electron microscope, the information limit of 41 pm for 2D materials has been achieved recently^[92].

However, in order to achieve deep-sub-angstrom resolution, conventional fixed-propagator ptychography (FPP) requires the incident electron beam to be nearly strictly parallel to the crystallographic zone axis of the sample. Such harsh experimental conditions have limited the wide application of electron ptychography in materials science. This paper illustrates the main features of deep-subangstrom-resolution electron ptychography with several application examples of the APP method.

Figure 5 compares the effects of misorientation on different atomic-resolution imaging methods through image simulations. It shows that the HAADF method is robust to the misorientation, and presents clear atomic-resolution images even under the condition of 12 mrad misorientation. This also explains why HAADF is by far the most widely used aberration-corrected imaging method. Unfortunately, HAADF has difficulty imaging light atoms (e.g., C, N, O). ABF and iDPC can image light atoms, but are more susceptible to misorientation, resulting in artifacts in the image contrast and the measurement of relative displacement between different types of atomic columns. FPP can give deep sub-angstrom resolution under ideal experimental conditions, but the influence of mistilt is also large. APP can give images with deep sub-angstrom resolution and precise atomic displacement even at large mistilt.

Figure 5. Comparison of different STEM imaging methods under different magnitudes of misorientation^[90]. The name of imaging method is labeled on the top of each image montage. Scale bars, 3 Å.

Figure 6 compares the reconstruction results of experimental datasets by FPP and APP. The [001]-orientated SrTiO₃ sample was used for data acquisition. Same experimental datasets are used for FPP and APP reconstructions. The mistilt of dataset 1 (Case 1) is about 3.8 mrad (about 0.2°), and that of dataset 2 (Case 2) is about 5.5 mrad (about 0.3°). The results show that FPP is sensitive to the misorientation. The information limits obtained by FPP under the mistilt of 3.8 and 5.5 mrad are 0.39 and 0.49 Å, respectively. Although the limits in both cases reach deep subangstrom resolution ($$ < $$ 0.5 Å), atomic columns in the phases recovered from FPP are stretched due to the uncorrected misorientation. In contrast, the sample thickness and mistilt angles are optimized in APP reconstructions, allowing elimination of the misorientation no matter how thick the object slice is, thus providing phase images with undistorted atomic columns and high information limit (0.20 and 0.21 Å, respectively). The enhancement of the information limit facilitated by the APP method is accompanied by improved accuracy in atomic positions to 1 pm or higher. APP has been successfully applied to reveal the tiny polar displacements in a ferromagnetic polar metal Ca₃Co₃O₈ [Figure 7]^[93].

Figure 6. Reconstruction of experimental datasets with misorientation with FPP and APP^[90]. (A and C) Phase images averaged over slices recovered with FPP after post-alignment (left) and APP (right). Scale bar, 2 Å. The insets are PACBEDs. (B and D) Diffractograms corresponding to (A and C). Scale bar, 1 Å⁻¹.

Figure 7. Total phase image of Ca₃Co₃O₈ measured at room temperature^[93]. The bar charts show averaged experimental relative shifts ($$ d_{CoO} $$) along the c axis between the Co atom and neighbor equatorial O atoms at two different octahedral layers (at room temperature), and the calculated results for different magnetic states. The net relative shift is a summation between these two layers, which is notably larger than the standard error. The illustration indicates the coupling between the build-in OOP electric field E and IP spins S, resulting in a Rashba spin–orbit coupling. The E and E' with opposite directions are not equal in magnitude as a result of the broken mirror (m) symmetry. Experimental data are presented as mean values ± standard error of the mean (error bar) from the relative shifts of n = 20 Co atoms. The purple, yellow and red circles represent the calcium, cobalt and oxygen atoms, respectively.

When employed in conjunction with the APP method, LOP can further improve the information limit. Figure 8 compares the reconstructed results for experimental datasets by LOP and CPP. The sample is DyScO₃ in the [001] zone axis. The quality of both the probe and the object reconstructed with LOP is significantly better than the results reconstructed with CPP. The information limit for the LOP method is 14 pm, significantly improved compared with 20 pm in the CPP result. Figure 8G shows the development of microscope resolution over the last two centuries.

Figure 8. Ptychography reconstructions by LOP and CPP^[18]. (A and B) Recovered probe amplitude. (C and D) Recovered phase of DyScO₃. (E and F) Diffractograms of the object phase with the information limit outlined by circles. (G) The development of microscope resolution. Scale bars in (A-D), 5 Å; (E and F), 2 Å$$ ^{-1} $$.

High phase accuracy

When mentioning high spatial resolution earlier in Section "SPATIAL RESOLUTION AND INFORMATION LIMIT", we gave an example: when the FWHM of the electrostatic potential of an atom is 0.3 Å, even an ideal electron microscope cannot distinguish two atoms with a projected spacing of only 0.2 Å. That being the case, what is the point of pursuing deep subangstrom resolution? The answer lies in high phase accuracy. The higher the spatial resolution, the smaller the blurring effect and the higher the phase accuracy. This is important for analyzing subtle structural features in materials, such as measurements of interstitial atoms and vacancies. With high phase accuracy, it is even possible to analyze electronic structure information such as charge, spin, orbitals, etc., directly in real space at the atomic scale.

Besides high spatial resolution, removing the noise and artifacts caused by surface damage layers is also crucial for achieving high phase accuracy. As shown in Figure 9, the actual samples have nanoscale damage layers on the upper and lower surfaces (including mechanical damages introduced by the grinding process, irradiation damages introduced by the ion thinning process, surface oxidation of metal samples, and surface adsorption layers). Atoms in the damage layers also scatter electrons and interfere with the intrinsic structural information within the bulk. Especially when studying point defects with weak signals, such as interstitial atoms and vacancies, the electron scattering from the damage layer can even be much stronger than that of the point defects. As mentioned earlier, multislice electron ptychography not only effectively solves the problem of dynamic diffraction in samples, but also achieves a certain depth resolution. This feature is of great significance for "eliminating the false and keeping the true" in microstructural analysis. Since multislice electron ptychography can obtain depth sections along the electron beam, it is possible to remove the upper and lower damaged layers, leaving only the information of the central layers for subsequent studies.

Figure 9. Illustration of the surface damage in real samples^[19]. The red balls represent interstitial atoms.

Figure 10 shows the ptychographic reconstruction of the 18-nm-thick TiZrNb medium-entropy alloy doped with oxygen. The top and bottom surface sections are featured with oxidation and damages [Figure 10A and C]. In TEM or STEM images without depth sectioning, it is easy to mistake surface damage for interstitial atoms in the bulk. In ptychographic phase images, it is possible to exclude surface sections for analyzing the intrinsic structures inside the bulk. By comparing the phase image of the central layers (as shown in Figure 10B and Figure 11A) with the structure model that includes candidate interstitial atoms [Figure 11B], it is determined that the interstitial atoms in the alloy predominantly occupy the tetrahedral interstitial sites^[94]. Contrary to the traditional understanding that interstitial atoms in body-centered cubic structures typically occupy octahedral sites, the new phenomena identified in this study are attributed to the high oxygen concentration^[94].

Figure 10. Surface damage revealed by multislice ptychography in experiment^[94]. The top, central and bottom slices are displayed in (A-C), respectively. The depth values are labeled on the bottom left of each image. The sample is an 18-nm-thick TiZrNb medium-entropy alloy doped with oxygen. The object is separated into 1.2-nm-thick slices during the reconstruction. Surface damage can be seen from the top and bottom slices.

Figure 11. Phase images of the oxygen-doped TiZrNb medium-entropy alloy viewed along the [110] zone axis^[94]. (A) The phase image reconstructed via the APP algorithm, the inset shows the corresponding diffractogram, indicating an information limit of 0.5 Å; (B) Comparison between the reconstructed phase images and the structure models with interstitials, indicating that oxygen interstitials mainly occupy the tetrahedral sites.

Recently, three-dimensional imaging and localization of individual interstitial atoms has been achieved, based on which the mid-range ordering of interstitial atoms has been revealed^[95].

In addition to its application in the retrieval of atomic structure, the high phase precision of ptychography makes it a powerful method for electronic structure analysis. The phase precision was improved to the order of $$ 10^{-4} $$ by correcting most experimental imperfections (partial coherence, sample drift during scanning, misorientation, surface damage, $$ etc $$.), enabling the identification of phase difference between atoms with different magnetic moments^[96]. Given the weak phase change caused by the magnetic vector potential (nearly thousands of times weaker than the electrostatic potential), it is crucial to eliminate the electrostatic phase of the surface layers. Figure 12 shows a step on the top surface of $$ \alpha $$-Fe₂O₃. Figure 13 shows the lattice-resolved imaging of antiferromagnetism (AFM) in $$ \alpha $$-Fe₂O₃. The phase undulation corresponding to the AFM structure is apparent along the [0001] direction.

Figure 12. Recovered phases at different depths of the α-Fe₂O₃ sample^[96]. One of the top and central slices are shown in (A and B), respectively. In the top slice (A), the phase image is quite noisy due to surface defects. A surface step can be seen. The top and bottom slices are excluded for quantitative phase analysis. Scale bar, 5 Å.

Figure 13. Experimental magnetic phase imaging of α-Fe₂O₃^[96]. (A) Total phase averaged over the central slices. The Fe and O ions are overlaid on the image with the white arrows indicating the spin directions of Fe; (B) The corresponding diffractogram (on a logarithmic scale) with the arrows indicating magnetic reflections; (C) The intensity profile along the dashed line marked in the diffractogram in (B); (D) The magnetic phase extracted from the total phase by Fourier filtering; (E) The projection of the magnetic phase to the vertical axis; (F) Intensity profiles of the total phase and magnetic phases along the dashed line marked in (A and D). Sample thickness, 20 nm. The electron dose used for dataset acquisition is $$ 9.0\times10^5 $$ e/Ų.

High dose efficiency

Electron ptychography has been proven as a powerful tool to image beam-sensitive materials^{[64,97–101]}. Ptychography utilizes most of the scattered electrons and their fine distribution in the reciprocal space for the reconstruction, resulting in massive redundancy in the diffraction dataset, which is beneficial regarding the shot-noise tolerance of ptychography. Besides the ability of removing residual aberrations, multislice ptychography overcomes the negative effect of multiple scattering and improves the reconstruction quality of thick samples^[74], which is beneficial for beam-sensitive materials. Capable of correcting residual mistilt between the zone axis of the objects and the beam direction, the APP method becomes very powerful in low-dose experiments, where it is difficult to minimize the mistilt. The object reconstructed via LOP exhibits significantly higher SNR than those reconstructed by CPP, making LOP a powerful method for low-dose imaging^[18].

Figure 14 compares the multislice ptychography and iDPC for imaging zeolites, a typical class of beam-sensitive materials. It can be seen that the resolution of the iDPC decreases as the sample thickness increases. For samples as thick as 100 nm, the iDPC has difficulty in resolving oxygen atoms. Whereas the contrast and resolution of ptychography increases with sample thickness, due to the fact that thicker samples change the phase of the probe to a larger extent and carry more high-frequency information in the scattered electrons that can be utilized by the algorithms to improve the reconstruction resolution. In the experiment, subangstrom-resolution imaging of zeolites with a thickness of 100 nm was achieved at dose-efficient (1, 100 e/Å$$ ^2 $$) condition through multislice electron ptychography. Note that the dose of a typical HAADF-STEM image is about 10$$ ^5 $$$$ \sim $$10$$ ^6 $$ e/Å$$ ^2 $$. The three-dimensional deformation of zeolite was quantitatively measured^[99]. The results show that a higher electron dose (3, 600 e/Å$$ ^2 $$) will cause significant damage to zeolite, including the mass transfer from the Si-O framework to the channel and the overall mass loss. Sub-angstrom-resolution imaging of molecules adsorbed in zeolites and boundaries between different zeolite phases have been realized^[101].

Figure 14. Ptychographic phases (left) and iDPC images (right) calculated from simulated 4D datasets at different sample thicknesses^[99]. The atomic structure is overlaid on the phase image. Si and O atoms are shown with blue and red circles, respectively. The 4D datasets are simulated at the zone axis and without any beam aberrations. Scale bar, 5 Å.

In addition, it is worth mentioning that electron ptychography, with its high dose efficiency, has also been successfully applied to low-dose imaging of biological samples such as viruses, DNA and protein^[102–106], obtaining better contrast than traditional imaging methods with large defocus.

Depth resolution

The multislice ptychography iscapable of obtainingstructural informationat different depths in thesample bya single scan^{[99,107–112]}.Currently the depth resolution of multislice ptychography is about 3~4 nm, substantially lower than the lateral resolution (deep-subangstrom) in the image plane. It might be appropriately called quasi-3D imaging. Figure 15 shows the reconstruction results of a dislocation in a freestanding SrTiO₃ thin film^[109]. Figure 15A is the total phase summing over all the reconstructed slices, representing the projected structure along the electron beam direction. It can be seen that there is an edge dislocation in this region, with several atomiccolumnsaround the dislocationcore split into two peaks. Figure 15B is the depth variationacrossa split atom column. It reveals that the split atom in the projection results from the transverse displacement of the atomic columns along the depth direction, indicating that the edge dislocation in Figure 15A is not a straight one, but a kink. Figure 15C shows the phase integrated over an atomic column at different depths, revealing the lateral displacement of the dislocation core along the dislocation line.

Figure 15. Three-dimensional visualization of an edge dislocation^[109]. (A) Total phase of the edge dislocation. Scale bar, 5 Å. (B) Depth variation of the phase across the split columns marked with A and B in (A). (C) Phase images at 2.4 and 12.0 nm in depth. Scale bars, 5 Å.

Recently, polar vortex arrays have been revealed in twisted SrTiO₃ bilayers utilizing the depth sectioning ability of multislice ptychography^[111]. It is surprising that the polar vortex forms in a paraelectric material.

Orientation imaging

The APP method is able to give a spatial distribution of the sample orientation, i.e., orientation imaging, in addition to correcting for the overall tilt of the sample with respect to the electron beam. Figure 16 presents the results for a larger area containing multiple dislocations. Figure 16A is a reconstructed phase image, which reveals the presence of three dislocations in this region. From the depth sectioning, we know that all these dislocations have lateral displacement in the depth direction. The crystal orientation distribution in this region is given in Figure 16B, with white arrows indicating the in-plane projection of the [001] zone axis of the crystal and the color indicating the direction of tilting. The blue arrows indicate the direction of the lateral displacements of the dislocation cores. Based on the depth-dependent structure and the orientation mapping around the dislocation cores, the relationship between the geometry of the dislocations and the deformation mode of thin filmscan be summarized. As shown in Figure 16C, when torsion occurs ina film normal to the dislocation slip plane, the upper and lower portions of the film are subjected to shear stresses in opposite directions, and the dislocations will slip. When bending occurs in the film normal to the dislocation slip plane, the upper and lower portions of the film are subjected to normal stresses in opposite directions, and the dislocations climb.

Figure 16. Crystal tilt mapping of multiple dislocations in SrTiO₃^[109]. (A) Total phase image of regions containing three dislocations. (B) Crystal tilt reconstructed in the same region of (A) with total phase image superimposed on it. The white arrow stands for the in-plane projection of the [001] zone axis. The blue arrows stand for the direction of lateral shift of the dislocation. Scale bars in (A and B) are 15 Å. (C) Schematics of lateral shift of dislocations induced by crystal tilt. The lattice vector c changes to c' by a tilt vector t, i.e., c' = c + t. Black squares are used to illustrate the strain state. Two modes are considered (left, torsion; right, bending). In both modes, the strain is inverted for the upper and lower part of the thin film, leading to the lateral shift of dislocations to opposite directions. The dislocation in the upper left in (B) corresponds to the torsion condition. The dislocations in the center and upper right of (B) correspond to a mixture of the torsion and bending.

Conclusion

Aimed at the bottleneck problems in microstructure analysis (the broad sense of metallography) in materials science, i.e., residual aberration, zone axis deviation, surface damage, irradiation damage, this paper briefly introduces the basic principles and main algorithms of electron ptychography, and illustrates the main features of electron ptychography with examples, including high spatial resolution, high phase accuracy, high dosage efficiency, depth resolution, and orientation imaging, etc. These features are the very basis on which electron ptychography can solve the above bottleneck problems. It can be expected that with the development of computer hardware and algorithms, electron ptychography will gradually be widely used, the information limit will be improved to within 0.1 Å, and the electronic structure, such as charge and orbital configurations, will be revealed, thus bringing the analysis of the microstructures of the materials to a new height.

DECLARATIONS

Appendices