2.1.

Fast FLIM Techniques

FLIM instrumentation has seen major advancements in recent years. FLIM is typically characterized by slow acquisition speed because high photon counts are needed for accurate decay rate estimation. Thus recent efforts have focused on improving FLIM instrumentation to increase acquisition speed. Karpf et al.⁶ developed the spectrotemporal laser imaging by diffracted excitation (SLIDE) microscope that achieves high-speed imaging by employing an electro-optic modulator, and high-speed swept source Fourier-domain mode-locked (FDML) laser, the output of which is then passed through a diffraction element. This results in spectrotemporal encoding of each pulse such that the wavelength provides point scanning, and the temporal information is used for signal to pixel mapping. The fluorescence is then demodulated using a fast digitizer to generate FLIM images. Figure 1(a) (left panel) shows a simplified schematic of the SLIDE microscope. The authors report FLIM acquisition speed of a million pixels per second. However, the limitation of this microscope is the requirement of bright samples for generating high photon count rates and higher two-photon excitation powers ($\sim 10$ to 100 mW). Alternatively, field programmable gate arrays (FPGA) can provide a fast FLIM modality in conventional FLIM collection schemes [Fig. 1(a), right panel]. A real-time digital frequency domain FLIM system using FPGAs at $250\mathrm{MS}/\mathrm{s}$ was implemented by Serafino et al.⁷ that is capable of simultaneous three-channel detection using avalanche photodiodes (APD).

Fig. 1

Advances in FLIM instrumentation. (a) Left panel: simplified schematics of SLIDE microscope that uses swept-source FDML laser that is pulse modulated such that each pulse is encoded both spectrally and in time, resulting in sequential pixel-wise excitation. This technique, along with a fast digitizer, achieves FLIM acquisition at high speed. Adapted with permission from Ref. 6. Right panel: simplified illustration of frequency domain lifetime imaging system implemented with FPGAs that modulates the excitation laser and digitizes the emissions using fixed gain APDs. Stylized image representing original publication with the permission of AIP Publishing.⁷ (b) Schematics of time-gated SPAD array consisting of SPAD photodiode and electronics to measure individual photon arrival times. Cross-sectional view of a SPAD showing key components—diode anode and cathode. FLIM systems implemented with SPAD arrays provide fast FLIM acquisition from large samples. (c) Super-resolved FLIM is acquired with SMLM techniques. Stylized illustration adapted with permission from Ref. 11 shows intensity frames including single-molecule localizations, single-molecule lifetime map, and histogram of photon arrival times for two indicated single-molecule localizations. (d) Simplified schematic showing FLIM with light-sheet illumination scheme that results in volumetric FLIM, adapted with permission from Ref. 13. (e) Simplified schematics showing fiber-optic-based FLIM system with TCSPC electronics for time domain FLIM data acquisition. Also shown are white light image of surgical FOV and FLIM FOV with augmented lifetime map overlay. Adapted with permission from Refs. 15 and 16. References to all relevant papers for each section have been noted.

2.2.

Wide-Field FLIM

Wide-field FLIM can image a larger field of view (FOV) by employing single-photon avalanche diode (SPAD) arrays or SPAD detectors with complementary-metal-oxide semiconductor (CMOS) technology, which has gained popularity in the last year. SPAD arrays can have temporal resolution in picoseconds and single-photon sensitivity that is advantageous for lifetime measurements. Figure 1(b) shows a schematic of a SPAD array. Lagarto et al. used a SPAD array ($32\times 64$ SPADs) detection scheme coupled with a fiber-optic probe for real-time FLIM and spectral imaging on a macroscopic scale.⁸ Lifetime was estimated using the phasor approach in real time using time-correlated single-photon counting (TCSPC). Additionally, multispectral FLIM was implemented by Ghezzi et al.⁹ using an $18\times 1$ linear SPAD array. Moreover, Zickus et al.¹⁰ aimed for real-time FLIM using a time-gated SPAD array capable of achieving widefield FLIM of $500\times 1024\text{pixels}$ with lifetime estimation performed by a custom artificial neural network (ANN) that was trained on simulated lifetime decay curves. Given the advantages of the SPAD array (Table 1), this technology is likely to increase in popularity for FLIM.

Table 1

Advantages and limitations of recent advances in FLIM instrumentation and analysis.

2.3.

Single-Molecule FLIM

A super-resolution microscope that can also acquire lifetime data is advantageous for detecting fluorophores with similar spectral properties and for lifetime-based Förster resonance energy transfer (FRET) measurements. Thiele et al.¹¹ combined a single-molecule localization microscopy (SMLM) technique such as direct stochastic optical reconstruction microscopy (dSTORM) and DNA points accumulation for imaging in nanoscale topography (DNA-PAINT) with FLIM [Fig. 1(b)]. Data were collected using a single APD and TSCPC electronics to generate two-channel images. Super-resolved separation of two species was achieved by Baysian pattern matching on confocal FLIM data. Figure 1(c) shows schematics of single-molecule FLIM with representative lifetime decays from two single-molecule localizations. This is the first time that super-resolved single-molecule localization FLIM was demonstrated, which provides a simpler alternative to point-spread function engineering methods such as stimulated emission depletion (STED). In a subsequent work, this group implemented a TCSPC-camera-based wide-field single-molecule localization approach for super-resolution imaging using a similar pattern matching method.¹²

2.4.

Light-Sheet FLIM

FLIM with a light-sheet illumination scheme enables volumetric lifetime data acquisition in large samples [Fig. 1(d)]. Hirvonen et al. merged wide-field TCSPC FLIM with light-sheet microscopy and microchannel plate detectors with a delay-line position sensitive anode.¹³ More recently, a digitally scanned laser light-sheet microscope with FLIM capabilities was implemented by Li et al.¹⁴ In this work, a gated optical intensifier (GOI) with a sCMOS camera was used for wide-field time-gated imaging.

2.5.

Fiber-Optic Clinical FLIM

Fiber-optics-based lifetime imaging enables FLIM for clinical applications. Marsden et al. presented a novel lifetime system based on fiber-optics excitation and detection. The point measurement system can be used intraoperatively and is capable of real-time processing ($>30$ frames per second).¹⁵ A similar real-time fiber-based system, simplified schematic shown in Fig. 1(e), was also implemented by Lagarto et al.¹⁶ that combined TSCPC detection and phasor analysis.

Table 1 summarizes some of the advantages and limitations of the instrumentation discussed.

3.1.

Cell Classifiers

Several recent papers have developed machine learning tools to analyze FLIM images. Some methods use standard labels to serve as ground truth to train algorithms that recognize specific cell types based on FLIM. For example, Sagar et al.¹⁸ used an ANN to differentiate microglia from other glia cell types in the brain based on label-free FLIM of reduced nicotinamide adenine dinucleotide (phosphate), NAD(P)H [Fig. 2(a)]. Their approach identified microglia in cell culture mixed with other glial cells with a true positive rate $>0.9$. They also identified microglia in fixed brain tissue with a true positive rate of 0.79. This is an important contribution because microglia are challenging to identify as most fluorescent labeling approaches cross-react with other immune cell types, are often insensitive to activation state, and require the use of multiple specialized antibody labels.

Fig. 2

Advances in FLIM analysis. (a) FLIM of the intrinsically fluorescent metabolic co-factor nicotinamide adenine dinucleotide (phosphate) [NAD(P)H] was combined with ANN-based machine learning to differentiate microglia from other glia cell types in the brain. The two-exponential decay lifetimes $({\tau }_1,{\tau }_2)$, weights $({\alpha }_1,{\alpha }_2)$, mean lifetime (${\tau }_m={\alpha }_1{\tau }_1+{\alpha }_2{\tau }_2$), and goodness-of-fit (${\chi }^2$) were used as input parameters to identify microglia in cell culture mixed with other glial cells and in fixed brain tissue. Adapted with permission from Ref. 18. (b). Left panel: phasor plots of fluorescence decays provide a visual distribution of fluorescence lifetimes and are derived from a Fourier transformation of the fluorescence lifetime decay with unitless horizontal ($G$) and vertical ($S$) axes.¹⁹ Here the phasor method was used to resolve the decay in each pixel of an image in live cells or mouse liver tissues with two or more exponential components without prior knowledge of the values of the components. Adapted with permission from Ref. 20. Right panel: spectral and lifetime imaging of NAD(P)H and FAD was combined with unsupervised $k$-means clustering of each pixel to identify regions within cells that have almost uniform metabolic properties. This method detects the cellular mitochondrial turnover and the metabolic activation state of intracellular compartments at the pixel level. Adapted with permission from Ref. 22. (c) FLIM and FRET decays acquired through any method can be analyzed with open-source software packages to quantify molecular changes and render images. These open-source packages include community forums to educate non-experts and foster continuous development by experts. Images adapted with permission from Refs. 26 and 27. References to all relevant papers for each section have been noted.

3.2.

Pixel-Level Analysis

Alternatively, pixel-level analysis methods can be used to classify molecular features in a FLIM image. The phasor approach plots the distribution of fluorescence lifetimes in a two-dimensional frequency space, which provides an intuitive method to group pixels that belong to molecular species of similar lifetimes.¹⁹ This approach has recently been expanded to automatically group pixels for improved accuracy and unbiased analysis [Fig. 2(b), left panel].²⁰ Specifically, the decay in each pixel of an image with two or more exponential components was recovered without prior knowledge of the values of the components using a graphical method for two components and a minimization method for three components. This specific use of the phasor approach is attractive because it can disentangle species with different lifetimes and spectra within the same pixel.

Like the phasor approach, machine learning has been used to achieve fit-free estimates of fluorescence lifetimes from decays on the pixel level,²¹ where simulated data were used to compare a machine learning approach to standard fitting algorithms. The root-mean-squared-error, which represents the difference between the reconstructed and measured decay traces, was observed to be lower for the machine learning approach compared to the standard fitting algorithm, including up to three-component analysis.

Fit-free analysis methods have also been developed for spectral-lifetime images. $K$-means clustering was used to cluster subcellular pixels with metabolically distinct properties using spectral and lifetime imaging of autofluorescence [Fig. 2(b), right panel].²² Two distinct clusters were identified as regions that have almost uniform metabolic properties, including the cytosolic activation state and the mitochondrial activation state. Inhibition of glycolysis and the electron transport chain confirmed the accurate isolation of these metabolically distinct clusters. Furthermore, deep learning methods have been developed to disentangle spectral and lifetime information on a pixel-wise basis. Intes et al.²³ unmixed spectral and lifetime data using the unmix multiple emissions (UNMIX-ME) algorithm. The algorithm was trained and validated in silico and benchmarked against a conventional least-squared lifetime fitting method for three- and four-exponential decays. UNMIX-ME was then used to monitor near-infrared FRET interactions in vitro and in vivo. The same group developed a convolutional neural network for spectrally resolved fluorescence lifetime imaging with compressed sensing.²⁴ This high compression enables in vivo spectral-lifetime imaging at enhanced resolution, acquisition, and processing speeds, without the need for experimental training datasets. This approach was validated in silico and demonstrated in vivo and in vitro with acquisition times reduced from 2.5 h to 3 min at 99% compression.

3.3.

Community Resources

One important component of FLIM analysis is dissemination to the community, standardization across labs so that results can be compared, and integration of FLIM analysis with image and data processing [Fig. 2(c)]. To address these needs, Gao et al.²⁵ created FLIMJ, an ImageJ plugin and toolkit that provides accessible image analysis workflows with FLIM data. FLIMJ offers FLIM fitting routines with seamless integration with many other ImageJ components and the option to extend features to create complex FLIM analysis workflows. FLIMJ routines can also be used with Jupyter notebooks and integrate naturally with science-friendly programming (e.g., Python and Groovy). FLIMJ was demonstrated on lifetime-based image segmentation and image colocalization with comparisons to standard FLIM analysis tools. Similarly, a Python software package named flimview was developed to read, explore, manage, and visualize FLIM images.²⁶ The flimview package is open source, with test data and documentation available on Github. Other open source FLIM analysis software includes FLIMfit, which is an OMERO client to analyze and visualize FLIM data,²⁷ and FLIM-FRET analyzer, which provides automated analysis of lifetime-based FRET images.²⁸

Table 1 summarizes some of the advantages and limitations of the analysis techniques discussed.

4.1.

Autofluorescence FLIM Applications

FLIM of autofluorescence is attractive for clinical use because it provides label-free images of tissue function. For cancer surgical guidance, Marsden et al. have developed a promising integrated fiber-based autofluorescence FLIM and analysis tool, which can spectrally resolve collagen, nicotinamide adenine dinucleotide (NADH), flavin adenine dinucleotide (FAD), and porphyrins. This tool called FLImBrush can be used in the operating room to detect cancer margins in a label-free manner and uses a deep learning-based image segmentation, block-matching-based motion correction, and interpolation-based visualization. This allows for real-time imaging and image processing to enable intraoperative imaging and surgical guidance, as demonstrated on surgeries of head and neck cancer patients [Fig. 3(a), left panel].¹⁵ Additional work to improve the overlay of the real-time FLIM images on the surgical field will yield an exciting new tool for image guided surgery. Another application, fluorescence lifetime imaging ophthalmoscopy (FLIO) has been in clinical use for 20 years.²⁹ Recent advances by Vitale et al. show that FLIO can detect an increase in autofluorescence lifetime in the diseased tissue of a small cohort of patients with choroideremia, an ocular disease, compared to normal surrounding tissue³⁰ [Fig. 3(a), middle panel].

Fig. 3

Examples of FLIM in biology and medicine. (a) Left panel: endogenous contrast applications include automated FLIM visualization using FLIMBrush on a clinical head and neck surgery case. Adapted with permission from Ref. 15. Middle and right panels: FLIM detection of choroideremia (middle)³⁰ and brain tumors (right)³¹ are shown with stylized images that represent changes observed in original publications. (b) Left panel: applications with exogenous contrast include quantitative analysis of HSA concentrations in cryosection from a low-grade and high-grade serous ovarian cancer patient, using the concentration dependent squarine dye (blue = high [HSA], green = mid [HSA], red = low [HSA]). Adapted with permission from Ref. 34. Right panel: FLIM of an ubiquitination-based dye, FUCCI-Red, which is a lifetime sensor of cell cycle state ($\text{blue}=S/G2/M$, $\text{green}=G1/S$, $\text{yellow}=G1$) shown with stylized images representing changes observed in original publication³⁵ (c) Top panel: FRET-FLIM applications include a new FLIM-FRET pair sensitive to chromatin compaction. Adapted with permission from Ref. 36. Pseudocolored chromatin compaction maps of the cells in the top row according to the palette defined below scaled between 2.5 ns (teal, 0% FRET, donor lifetime) and 2 ns (red, 21% FRET, quenched donor). Bottom panel: intracellular glucose concentration imaged with a FLIM-FRET pair.³⁸ A shift toward red indicates higher intracellular glucose, which can be seen in the glucose condition represented using stylized images. References to all relevant papers for each section have been noted.

Autofluorescence FLIM of ex vivo tissues have also been investigated this year. Erkkila et al. showed that FLIM of NAD(P)H, and protoporphyrin identify tumor margins in low- and high-grade human gliomas³¹ [Fig. 3(a), right panel]. With a larger sample size, this promising result could yield another useful imaging-based method for intraoperative guidance. Additional work by Bower et al. imaged changes in NAD(P)H lifetimes and calcium transients in ex vivo mouse brains and showed that there are significant differences in the NAD(P)H lifetime dynamics between neurons and astrocytes.³²

4.2.

FLIM of Exogenous Molecular Probes

New studies have generated molecular probes with fluorescence lifetimes that are sensitive to physical conditions, including new FLIM probes for protein concentration and cell cycle. Yi et al. developed a small squarine dye with a fluorescence lifetime that changes depending on serum albumin concentrations.³³ Recent work in human ovarian cancer biopsies by Lin et al.³⁴ shows that this dye is sensitive to concentrations of human serum albumin (HSA), which is a protein that is often concentrated near solid tumors [Fig. 3(b), left panel]. More importantly, the dye shows differences in the HSA microenvironment between low- and high-grade serous ovarian cancer in humans.

A novel fluorescence lifetime probe, FUCCI-Red, relies on the different fluorescence lifetimes of the red proteins mCherry and mKate2.³⁵ This all-red probe is an advancement over the standard FUCCI probe that uses fluorescence spectral shifts between multicolored fluorescent proteins to detect cell-cycle phases, thereby limiting the color channels available for multiplexed imaging. The new FUCCI-Red capitalizes on lifetime shifts and enables multiplexed detection of cell cycle information alongside autofluorescence FLIM on the single-cell level [Fig. 3(b), right panel].³⁵

4.3.

FLIM-FRET Applications

Finally, FLIM-FRET can capture extracellular and subcellular interactions on the nanoscale, including several studies this year. For example, an FLIM-FRET pair of eGFP and mCherry, both tagged to histones, detected changes in chromatin architecture after induction of DNA damage.³⁶ This FLIM-FRET method provides spatiotemporal visualization of chromatin compaction-dynamics in living cells, which is technically difficult without the use of super resolution microscopy because of the very small spatial scale on which chromatin compaction occurs [Fig. 3(c), top panel].^36,37 The FLIM-FRET method makes this analysis more accessible to labs that do not have super resolution microscopes. Another novel FLIM-FRET detector uses an eCFP-Citrine FRET pair tagged to a bacterial protein that changes conformation upon binding to glucose [Fig. 3(c), bottom panel].³⁸ This complex can be stably expressed in breast cancer cells to detect single-cell heterogeneity of intracellular glucose concentrations, under multiple culture conditions in vitro and in vivo. Intracellular glucose imaging of single cells provides an exciting new tool to investigate metabolic heterogeneity and its effects on treatment response.