Introduction

The transmission electron microscope (TEM) was developed with the goal to achieve imaging at a spatial resolution beyond the diffraction limit of optical microscopy [1]. The modern TEM not only achieved this goal, but surpassed expectations with the development of atomic-scale electronic and chemical mapping [2, 3], elemental contrast imaging in scanning transmission electron microscopy (STEM) [4, 5], atomic-scale tomographic reconstructions of small volumes of material [4, 6, 7], and imaging at picosecond temporal resolution of atomic events [8, 9]. The true power of the S/TEM is not only the atomic-scale information, but the site specificity that allows researchers to find correlations between defects, impurities, grain boundaries, or dislocations in structures that can be tracked during a material process. These advancements are truly revolutionary, as demonstrated by several TEM-related Nobel prizes in the last century [1, 10, 11]. The next frontier is investigating complex (multi-stimulus) dynamic processes, mimicking real-world conditions (operando environments) [12], in materials while retaining the same structural and chemical precision already demonstrated on idealized materials within a high vacuum, without introducing artifacts on the sample. We propose an adaptive in-situ S/TEM that will pull together these state-of-the-art capabilities into an integrated multimodal solution for understanding a single specimen with all the associated heterogeneities, and then quantify reactions of that specimen to multiple stimuli.

The primary developments that have been made to optimize the S/TEM have been focused on improving spatial resolution [13, 14], electron energy loss spectroscopy (EELS) at high-energy resolution [15, 16], mapping strain and magnetic moments [17, 18], imaging reversible or irreversible processes at high-temporal resolution [9, 19], or environmental control over the specimen [20, 21]. The challenge that the electron microscopy community faces is this optimized S/TEM development method, which results in backward-compatibility issues to use combinations of these major advances on one single S/TEM to solve complicated dynamic materials problems that cannot be solved with other tools (X-ray, Raman, optical microscopy/spectroscopy, or scanning probe) [22, 23]. Currently, there is no single S/TEM in the world capable of collecting atomic-scale Z-contrast imaging, with single-digit meV energy-resolved chemical mapping, and ultrafast picosecond imaging on a sample under a combination of tailored environments.

At present, to fully understand a material system or a reaction in-situ at the nanoscale, researchers must move the sample between optimized S/TEMs (accelerating voltage [24], spatial resolution [25], energy resolution [26], ultrafast temporal resolution [19], combined in-situ capabilities [27], direct-electron detection [28], or contrast-optimized imaging modes [13]) housed in multiple laboratories across the globe. However, multimodal S/TEM investigations are currently limited by samples that can become compromised during transit between S/TEMs that may produce unpredictably altered results that cannot be reproduced. Under those conditions, each machine provides fractions of a full dataset that must be combined to provide enough evidence of an atomic mechanism or reaction. For example, an environmental TEM corrosion experiment at high temperature would likely require transfer to a microscope optimized for STEM and EELS for corrosion product identification, where the transfer of specimens through different environments to another microscope may result in phase transformation of the corrosion product prior to identification. This divergence in S/TEM has been necessary, as in general, small research teams design microscope modifications to create a new capability that is unique or to provide a new way to collect data for material systems of interest (biological, metallic, ceramic, polymeric, and composite materials).

Our greatest challenge is that current S/TEMs cannot switch between imaging modes quickly enough to capture in-situ diffraction data, atomic-scale STEM chemical maps, high-resolution TEM images, and ultrafast TEM images from a dynamic reaction in a material, such as annealing or fracturing. Therefore, in this viewpoint, we present ideas for a state-of-the-art adaptive multimodal in-situ S/TEM that would allow for concurrent optimized electron beams for multiple imaging modes with optimal electron and photon collection, permitting enhanced characterization of dynamic process at atomic resolution. This proposed integrated TEM (ITEM) concept combines optimized electron optics for electrostatic switching between different imaging modes, with an adjustable specimen chamber optimized for complete environmental control and signal collection, followed by a projection system designed to move the transmitted beam to various electron detectors.

Scientific need to characterize transients

To advance in solving multitrillion dollar real-world materials problems (e.g., aircraft failure due to high-cycle fatigue [29], nuclear reactor failures due to radiation-induced creep, and bridge failures due to stress-corrosion cracking [30]), we need to identify and characterize atomic-scale structural and compositional changes in a material with adequate temporal resolution during sample evolution. The scientific need is for a tool that can revolutionize our understanding of the direct correlation between the material’s response to harsh combined environments (radiation of outer space, the core of a nuclear reactor, subsurface hydraulic fracturing, melt pools in additive manufactured materials, or nucleation and growth processes) and observe transient states, which are currently unobservable. These can then be used to directly correlate structure–property relationships to enhance design of engineered materials beyond the approaches based on the Time–Temperature-Transformation [31], Heckmann [32, 33], Ellingham [34], and similar diagrams.

Presently, to investigate these types of material reactions, researchers are faced with the complexity of a multimodal toolset, which requires a specimen to be prepared within a mobile platform for characterization between state-of-the-art instrumentation (X-rays, Raman, optical spectroscopy, scanning probe microscopy, and/or electron microscopy). Additionally, the burden of multimodal data analysis requires accurate overlays in the images, spectra, and property measurement in relative time. Experimentally, ideal specimen preparation (volume, surface, etc.), on a relative scale, is not identical for all these various instruments, and therefore, compromises must be made for multimodal investigations, which result in sample preparation inhomogeneities and data artifacts. Few incremental advancements have been made toward specimen holders for multimodal interrogation [35, 36]. In contrast, a multimodal approach that does not require multiple sample parameters is the use of different S/TEMs that have been individually optimized for imaging modes, in-situ environments, ultrafast detection, and/or spectroscopy, providing site-specific characterization that can identify dynamic materials phenomena with an atomic-scale mechanistic view.

For current S/TEM designs, most attempts to characterize samples in harsh environments in-situ have been limited to modifications to the stage itself. Despite this limitation, many S/TEM experiments demonstrate control over several environmental variables at once. However, this is often at the sacrifice of accuracy of control or measurement, which may be only within 10–15% of the real values [34]. Even fewer studies have been able to holistically encompass the experimental conditions required to achieve true operando control that allow for the imaging of partial states and nonequilibrium conditions that arcuately reproduce the operational conditions of space applications [33]. In contrast, a fully operando experiment would offer complete environmental control over the material: in surface chemical adsorption, mass flow, temperature flux, liquid or gas pressure, applied force, and in an applied external magnetic or electric field. This breadth of options would allow for accurate experimental verification that could feed into predictive models and accelerate the design and implementation of new materials into current and future applications.

A more comprehensive challenge is the steady development of new techniques that sometimes are challenging to reintegrate into in-situ S/TEM instruments or may not be backward compatible, which can lead to a bifurcation of the electron microscopy field into more specialized tools, illustrated in Fig. 1a. For example, enhancements to the electron optics, sample environment, or data collection can impose limits on the techniques that could otherwise be performed simultaneously or sequentially. Despite this general trend, efforts have been made by many research groups to overlap techniques (Fig. 1b), for example, by combining two mature but separate S/TEM techniques, as with the aberration-corrected environmental TEMs [13], which are now commercially available. Other work has emphasized the combination of S/TEM with other analytical tools, for example, by integrating nanoindentation into a TEM stage design [37] or Raman spectroscopy within the TEM column [38]. A third trend has paired imaging advances with in-situ S/TEM technology, as demonstrated in the recent bright-field (BF) and dark-field (DF) STEM characterization of samples during mechanical deformation [39] or environmental TEM [40]. A small number of even more ambitious research advances have even combined three or more techniques, although the resulting experiment generally requires multiple operators for success, as with the recent combination of laser irradiation, ion irradiation, and quantitative nanopillar compression [41]. This simultaneous environment experiment allows for mechanisms that would not be active in a sequential experiment, and the subsequent success showcases the need for tools that can achieve such advanced experimental complexity while mitigating the specimen alterations that can occur during transfer between several optimized tools. At present, the most successful high-order couplings are seen in areas with significant similarity between mature in-situ techniques. Figure 1b provides a schematic of the potential overlap of related and commonly combined in-situ techniques that control the local stressors and the S/TEM sample environment during real-time observation.

Figure 1
figure 1

Current and perspective abilities for imaging/data collection modes and combined in-situ experiments in the S/TEM. a Specialization of the electron microscopy field, though many can operate on one tool. b Current standardized controllable and overlapping stressors. c Combined environments obtainable with the integrated TEM for operando experiments

Full size image

If we can obtain fully operando characterization of fundamental site-specific chemical and structural phenomena, as illustrated in Fig. 1c, then we will be able to unlock the design rules necessary for overcoming critical problems in material failure, wear, fracture, fatigue, corrosion, operational hysteresis, phase transformations, aggregation, and numerous other destructive and complicated challenges facing the materials science community. The discovery of these design rules would then enable the formulation of a fundamental material structure–property relationship database, encompassing information relating to defects, impurities, and structural heterogeneities, and therefore enabling predictive modeling, development, and redesign with regard to performance for specific applications from satellites to biocompatible materials.

In this viewpoint, we challenge the electron microscopy community to explore the concept of developing an all-encompassing instrument that juxtaposes current advanced analytical and in-situ S/TEM techniques. This instrument development would provide a characterization platform unmatched by other techniques for site-specific sub-nanometer resolution imaging, diffraction, and spectroscopy that could identify new materials phenomena under stimuli/environments that have never been investigated previously at this scale. The remainder of this viewpoint is a Gedanken experiment, considering how a large ensemble of capabilities would be constructed to best benefit the materials science community for broadly impacting our society as a whole.

Integrated TEM

Over the rich nearly 90-year history of electron microscopy [1], creative scientists and engineers have developed numerous electron microscope optics designs (Fig. 2). Each of these variations has its own unique electron ray path that enables exploration of the real and reciprocal space of a material. This can be applied for both applications in conventional TEM (CTEM), Fig. 2a, and scanning TEM (STEM), Fig. 2b. In addition, complex electron optics allow for monochromation or high-spatial resolution with an aberration corrector (Figs. 2c, e respectively). Drastically altering the temporal operation of the electron optics has led to new insights into materials, as has been achieved in stroboscopic and dynamic TEM (Fig. 2d). Many of these optical paths have resulted in the commercial production of specialized S/TEM instruments, but we wonder whether these optical paths could become configurational variations of the same electron microscope. As the Fig. 2 diagrams indicate, the majority of these designs have the sample mounted in a stable holder to minimize drift and located in a good- to ultrahigh vacuum in order to maintain the path and phase of the electrons, with the sample itself situated within the objective lens of the microscope. Therefore, we propose to integrate these designs where they are similar and to create separate electron pathways where they differ, innovating new design specifications in pursuit of a fully integrated transmission electron microscope.

Figure 2
figure 2

Electron ray diagrams for a conventional transmission electron microscopy (CTEM), b scanning transmission electron microscopy (STEM), c monochromated transmission electron microscopy (MonoTEM), d stroboscopic or dynamic transmission electron microscopy (DTEM), and e aberration-corrected transmission electron microscopy (ACTEM)

Full size image

Building off the integrated toolbox concept presented by I.M. Robertson, et al. [23], we propose here an experimentally versatile, adaptive platform of the ITEM for site-specific, operando, atomic-scale structural and compositional characterization. The general design of this concept instrument can be seen in Fig. 3. By capitalizing on the advancements in particle physics (super conducting magnets, fast-switching electrostatic deflectors, and high sensitivity detectors), as well as electrical engineering (faster circuitry) and computer science (advanced control algorithms), we believe that the design is viable using current technology. The prototype ITEM will be a truly integrated microscope (Fig. 3), featuring three major deviations from the current configuration of commercial electron microscopes:

  1. A.

    An illumination system that is interconnected by electrostatic lens directly over the objective pole piece providing fast and rapid switching between imaging, diffraction, and spectroscopy techniques.

  2. B.

    A robust and flexible specimen assembly in the objective lens pole piece and chamber design that customizes the experiment in a gap under extremes by opening up the largest steradians possible.

  3. C.

    An imaging, projector, and spectrometer system designed to utilize all the electron, photons, ions, and other signals that transmit through or are produced by the sample for imaging and spectral analysis.

Figure 3
figure 3

Proposed schematic of the integrated transmission electron microscope. Comprised of a multifaceted illumination system, an open-designed sample region, and a detection assembly design to synchronize imaging modes to detectors/cameras/spectrometer

Full size image

The illumination system represents the greatest departure from current commercial TEM designs. Because the incompatible electron optical ray pathways cannot be fully integrated in a single column above the specimen without compromising the operation of these advanced modes, we propose that they be combined, as shown in Fig. 3. This scheme retains nearly optimized ray pathways for all of the advanced illumination modes, then couples each to the objective lens pole piece through rapid switchable electrostatic lenses. With this configuration, it would be possible to switch between probe-corrected STEM, image-corrected TEM, and dynamic TEM with intervals on the order of nanoseconds. This configuration also allows for future modification of the incident beams or the addition of another electron source, similar to ion beam column designed instruments.

Of course, great care must be taken in the development of such an integrated system, so as not to lose the strengths and core imaging modes of the individual techniques. However, the incorporation of recent and rapid advancements in genetic algorithms [42,43,44] will mitigate a number of complex design variables by adapting existing designs for fully functional state-of-the-art control systems and by including multiple design options in a single device. It is our hope that this evolutionary approach would minimize the trade-offs in capabilities by finding unexpected and unconventional design paths not envisioned by classical microscope design strategies. If so, the ITEM would avoid falling into the classical conundrum of the “Jack of all trades and master of none” by becoming a “Johannes factotum” (“Johnny do-it-all”), a term used by Robert Greene to critique the breadth of work done by William Shakespeare [45]. Of course, this criticism came from an established writer in 1592, whom did not appreciate the transformative contributions of young Shakespeare that are widely celebrated today.

Flexible in-situ testing chamber

Current TEM designs encounter problems with operando environmental control and quantitative property measurements, where the structure–property measurement is coincident with the site-specific region of the sample being investigated. Commercially available in-situ sample holders somewhat mitigate the problem by increasing control over the specimen through coupled environmental control and property measurements (nanoindentation, biasing, heating, cryogenic temperatures, and electrochemistry), and several of these holders even have built-in double-tilt capabilities or combine temperature control with biasing, liquids, gasses, or mechanical loading. However, each capability generally requires a completely different sample holder and there are mechanical limitations in the current sample holder designs that limit innovation. For example, the tip geometry of the sample holder is limited by the objective lens pole piece gap, and both the sample mounting configuration and the feedthroughs (for controlling the electrical bias, mechanical loading, liquid/gas tubing, and property measurements) are limited by the inner diameter of the side-entry TEM holder. The experimental area is further limited by the restrictive number of experimental input/output ports and by the dimensions of the holder tip (~ 3 mm x ~ 10 mm) and rod (~ 250 mm long, ~ 8 mm in diameter). However, this limitation is not inherent to the TEM experiments, rather, it is historically rooted in the standard design of a commercial TEM pole piece, Fig. 4a. Several individual research groups (Frances Ross and Murray Gibson [46], NIMS [47], and others) have made initial attempts to increase the types of experiments that can be observed, primarily by statically increasing the pole piece gap to make room for additional instrumentation. In these early experiments, the cost in spatial resolution proved to be high, however, Eric A. Stach and others have shown that new aberration correction techniques can be used to increase the pole piece gap, while retaining angstrom resolution [22].

Figure 4
figure 4

Current a and the proposed adaptive b specimen assembly schematics. The specimen is positioned between the two metallic cones (upper and lower pole pieces of the objective lens in a TEM). The expanded gap (right) allows for more stimuli [ions (red dashes), laser (pink), feedthroughs (black wires)], detectors [backscattered electrons (green), X-rays (dk. red, × 4), light (yellow)], and specimen manipulation with a feedthrough-integrated [liquid/gas lines (grey)] cartridge design (blue) while retaining the objective aperture (copper) for imaging. Gaps in the pole piece structures identify differential pumping apertures for gaseous experiments of an environmental TEM

Full size image

The proposed ITEM, as illustrated in Fig. 4b, would institute three mechanical changes that would potentially open up the entire experimental region (4π steradians minus the cones of the objective lens pole piece), allowing individual users to find an optimal balance between maneuverability and resolution for multimodal in-situ experiments.

(1) We would improve the quantity and strategic placement of the surrounding the pole piece gap ports to increase the detection accessibility. Removing steric restrictions around the sample would also allow for the integration of complementary photon characterization techniques, such as energy-dispersive X-ray spectroscopy (EDS), cathodoluminescence (CL), ion beam induced luminescence (IBIL), Raman, time-domain thermoreflectance (TDTR), and transient grating spectroscopy (TGS) with electron characterization techniques such as EELS, precession electron diffraction (PED), fluctuation electron microscopy (FEM), electron tomography, and in-line or off-axis holography. (2) We would include a removable/interchangeable top-entry loading configuration to connect to additional stimuli feedthroughs, like the one successfully implemented at the University of Vienna [48]. (3) We would add an adjustable pole piece gap to provide increased access for the most complex multimodal experiments. The spatial resolution flexibility would allow researchers to tune the instrument for any given experiment or measurement, trading spatial resolution for increased collection angle in near-specimen detection, specimen tilt, or increased coincident stimuli on the specimen. Similarly, adjustable pole piece gap systems are already common in particle physics [49].

Although reconfiguring the objective lens chamber can increase mechanical risk factors (instability in the column, possible loss of electron beam coherency, and potential depletion of beam current), these risks are outweighed by the numerous advantages of the ITEM for acquiring novel data about nanoscale transient materials phenomena that take place in extremely complex systems. This adaptive specimen assembly would permit simultaneous in-situ heating, cooling, gas/liquid, electrical biasing, straining (indentation, compression, wear, fatigue, bending, etc.), irradiation (laser, electrons, and ions), and ion implantation, greatly increasing the number and combinations of stressor environments and analytical detectors that can be used to probe the sample, with the combined ability for the researcher to rapidly switch between various advanced microscopy techniques (CTEM, STEM, DTEM, etc.).

Signal detection synced to imaging modes

Over the last decade, there have been extensive advances made in both in direct and multimodal detection systems, greatly improving the spatial [50, 51] and temporal [52, 53] resolution of TEM cameras, as well as the energy resolution of EELS detectors [54]. These developments have greatly enhanced state-of-the-art analytical characterization, even permitting imaging of single atoms [55], single vacancies [56, 57], and even isotopic variations [58]. However, these advanced detectors have only rarely been applied to in-situ and operando measurements [53, 59, 60], partly because they are often incompatible with the instrumentation and partly because the programming does not easily sync with in-situ techniques. This separation was not intentional, and more likely reflects the research community’s general hesitation to risk expensive TEM equipment with the introduction of reactive gasses, liquids, or lasers [61].

However, the increased physical access to the sample in the ITEM will also enable a greater coupling of analytical and in-situ capabilities, because the additional space for sample-adjacent detectors should also permit the collection of a greater number of photons (visible and X-ray), electrons (forward and back scattered), and any other particle or molecule released from the sample during in-situ testing. In terms of the adaptive imaging modes, signal detection would need to be synchronized to the correlated incident electron beam source, to properly direct the post-specimen electrons onto the camera/detector/spectrometer. This design would additionally allow for a continuous imaging mode to be split between different detectors using electrostatic deflectors, which would enable the collection of higher and lower magnification TEM images or videos with merely nanoseconds lost for switching delays. This vison for the ITEM is approaching the vision put forward in the Basic Energy Science report “Future of electron scattering and diffraction” [62]. Just as importantly, these discoveries will facilitate the acquisition of complex experiments and data sets that will greatly increase our ability to understand and explore fundamental science at the nanoscale.

Primary challenges and future directions

The development of the ITEM will not be without significant challenges and obstacles. However, we believe that the expertise present in the global electron microscopy community and the demonstrated ability of the community to leverage other fields is more than adequate to realize such a concept, if three critical challenges can be met. The first challenge will be the development of small and fast electron-optics components capable of rapid switching with minimal uncompensated hysteresis, aberrations, or artifact inclusions. Such components will increase the versatility of the ITEM without sacrificing its valuable stability. The second significant challenge will be the development of adequate control systems and data handling algorithms that draw on recent discoveries in automated control, machine learning, and big data processing to permit programed experimental order operations, rapid response to experimental conditions, data transfer, data processing, and data filtering [63]. Ideally, switching between imaging modes should be rapid and flawless, with data analysis processed on the fly. The third and final challenge will be the dramatic redesign of the S/TEM to increase sample and environmental manipulation. To truly approach the vision of a “lab-in-a-gap” [60], the ITEM must depart from the rod-based TEM sample mount design, into a cartridge design, and utilize the full volume of the objective lens pole piece area for control of environment/stimuli, sample manipulation, property measurements, and signal detection.

If creative solutions are found to overcome these and other unforeseen challenges that will arise in the development of the ITEM, this instrument will be a powerful integrated tool that combines the strengths of numerous juxtaposed characterization tools and dramatically accelerates discovery for fundamental science. With the exception of the thin-foil limitation, which is fundamental to all common S/TEM techniques, ongoing experimental developments in hardware, software, and environmental manipulation have already independently demonstrated capabilities that the ITEM would centralize in a single instrument. Instead of collecting data at different optimized machines around the globe, researchers will be able to answer complex materials problems, solve interdisciplinary questions, provide faster iterations of nanoscale studies for true statistical qualifications, and ultimately provide deeper insight into the fundamental laws of physics and chemistry that govern transient states and control the evolution of mater.

Conclusions

Historically, most major advancements in the field of electron microscopy were developed on a single tool, which required reintegration into modern tools for benefit of the advancement, though sometimes the development resulted in more specialized fields and associated instruments. Here, we further the concepts developed out of the ‘lab-in-a-gap’ and ‘no electron left behind’ challenges, to propose an integrated transmission electron microscope (ITEM). The proposed instrument will permit various electron sources and electron optics to be available for high-resolution imaging, high-temporal resolution imaging, and chemical imaging, using fast-switching magnets to direct the different electron beams onto a single maneuverable objective pole piece, where the specimen resides. Such a tool would be designed to optimize the detection of electrons, photons, and secondary ions emitted spherically from the sample to provide an encompassing method for the structural and compositional acquisition before, during, and after exposure to accurately controlled and complex environmental stimuli. Multimodal in-situ characterization housed in a single instrument has the potential to revolutionize materials science, and the data collected from the proposed ITEM will enable the creation of a database that relates structure–property relationships of multicomponent materials to complex environments. The small investment required to develop such a tool pales in comparison to the economic impact that it could have for solving critical microstructural-property relationships in broad-application-space materials such as steels or ceramics.