Metasurface on integrated photonic platform: from mode converters to machine learning

Zi Wang; Yahui Xiao; Kun Liao; Tiantian Li; Hao Song; Haoshuo Chen; S. M. Zia Uddin; Dun Mao; Feifan Wang; Zhiping Zhou; Bo Yuan; Wei Jiang; Nicolas K. Fontaine; Amit Agrawal; Alan E. Willner; Xiaoyong Hu; Tingyi Gu

doi:10.1515/nanoph-2022-0294

Open Access Published by De Gruyter July 20, 2022

Metasurface on integrated photonic platform: from mode converters to machine learning

Zi Wang , Yahui Xiao , Kun Liao , Tiantian Li , Hao Song , Haoshuo Chen , S. M. Zia Uddin , Dun Mao , Feifan Wang , Zhiping Zhou , Bo Yuan , Wei Jiang , Nicolas K. Fontaine , Amit Agrawal , Alan E. Willner , Xiaoyong Hu and Tingyi Gu

From the journal Nanophotonics

https://doi.org/10.1515/nanoph-2022-0294

Abstract

Integrated photonic circuits are created as a stable and small form factor analogue of fiber-based optical systems, from wavelength-division multiplication transceivers to more recent mode-division multiplexing components. Silicon nanowire waveguides guide the light in a way that single and few mode fibers define the direction of signal flow. Beyond communication tasks, on-chip cascaded interferometers and photonic meshes are also sought for optical computing and advanced signal processing technology. Here we review an alternative way of defining the light flow in the integrated photonic platform, using arrays of subwavelength meta-atoms or metalines for guiding the diffraction and interference of light. The integrated metasurface system mimics free-space optics, where on-chip analogues of basic optical components are developed with foundry compatible geometry, such as low-loss lens, spatial-light modulator, and other wavefront shapers. We discuss the role of metasurface in integrated photonic signal processing systems, introduce the design principles of such metasurface systems for low loss compact mode conversion, mathematical operation, diffractive optical systems for hyperspectral imaging, and tuning schemes of metasurface systems. Then we perceive reconfigurability schemes for metasurface framework, toward optical neural networks and analog photonic accelerators.

Keywords: deep learning; metasurface; silicon photonics

1 Introduction

In recent years, photonic integrated circuits (PICs) have emerged as one of the key platforms for components in optical communications and computing. Especially, the expanding of components libraries in silicon photonics with verified performance through manufacturing manifests the complexity, functionality and scalability of PIC system [1]. Most of those components available in foundry process design kits are designed for fundamental mode only, because of the unacceptable crosstalk between higher order modes and loss due to the multimode interference. However, the high refractive index difference between the silicon core and silica cladding layer makes a very strong mode dispersion in silicon photonics, which make it possible to utilize multimode in PICs [2]. Parallel to the efforts on developing multi-mode waveguides-based components by profile trimming or inverse design, here we review the utilization of metasurface design concept for diffraction-based PICs. The diffractive PIC breaks the waveguides confinement designed for single or few modes waveguide, but builds on the ‘free-space propagation’ in silicon-on-insulator (SOI) slab waveguide.

Metasurfaces are planar photonic elements, composed of subwavelength distributed antennas with spatially varying geometric parameters, able to control the propagation of light at will [3]. Metasurfaces can be used for different applications, for example, focusing and holographic imaging. Meta-system can also be realized by cascading multiple layers for metasurfaces, which can be used for more complex applications, such as analog computing and spectroscopy. Most of the metasurface works have focused on controlling the light propagation in free-space, however, besides free-space metasurfaces, they can also be used for integrated photonics, to control the guided multimode in waveguides [3, 4]. Multimode waveguides increase the data throughput, and the diffraction and interference that occurs in multimode waveguides make it possible for optical signal processing. Compared with traditional methods, metasurface-based design method is focused on utilizing the diffraction of light, which is especially suitable for applications such as Fourier transform, analog signal processing, and convolution. All these applications require sufficient space for ‘free-space propagation’ in slab waveguide, and thus cannot be achieved using compact inverse design approaches.

In this paper, a review of recent progress in metasurfaces on integrated photonics platform is given. The design principle of high contrast transmitarrays (HCTAs)-based metasurfaces is discussed in Section 2. Mode converters based on metasurfaces are discussed in Section 3. In Sections 4 and 5, we discuss a meta-system that is used for analog computing and machine learning. In Section 6, we discuss the perspectives for integrated metasurfaces.

2 Design principle of integrated meta-atom/metasurface cell

The design of metasurface for integrated photonics platform follows the same rules as free space metasurface. By changing the geometric parameters of subwavelength unit cells, the amplitude or phase of the transmitted or reflected light is changed with subwavelength resolution. Arbabi et al. [5] designed a free space two-dimensional metalens with high numerical aperture based on HCTA, as shown in Figure 1(A). Figure 1(B) and (C) show the simulated transmission and phase shift of the HCTA structure versus post diameter and lattice constant. Figure 1(D) shows the transmission and phase shift of periodic hexagonal HCTAs with lattice constant of 800 nm, and varying post diameters. Figure 1(E) shows the deflection efficiency versus different deflection angle of the HCTA. Inspired by this free space HCTA metasurface, Wang et al. [6] designed an on-chip one-dimensional (1D) HCTA for analog computing. The HCTA is composed of 1D air slot array defined on the silicon slab waveguide, as shown in Figure 1(F). The phase shift and the transmission of the guided wave can be modified by changing the geometric parameters, the length and width, of the air slots, as shown in Figure 1(G) and (H). Figure 1(I) shows the transmission and the phase shift of the 1D HCTA versus slot length, with a fixed slot width of 140 nm. The large refractive index difference between silicon and air ensures light confinement within the slots. The effective refractive index of the guided wave in the slot and slab waveguide is different, which allows phase shift of the transmitted light to be controlled by varying the slot length. For the same incident angle, the deviation of the transmission and phase is small for different slot lengths, as shown in Figure 1(J). A typical diffractive surface is made of nanostructures with a fixed depth (z-direction) as the direction of the input light, and the topologies and dimensions in-plane (x–y directions) are designed to form one-to-one correspondence with the complex transmission coefficient through numerical simulations. Alternatively, the wavefront of in-plane guided wave can be controlled by one-dimensional metasurface defined in the thin film. The integrated metasurface allows the variation of the nanostructure length along the direction of light propagation. On widely used silicon-on-insulator substrates, the high refractive index contrast allows 2π phase shift of the input wave within ≈1 µm trench length variations. With such design principle, one can design the integrated metasurfaces to achieve almost arbitrary complex transmission for different applications.

Figure 1:

Free-space HCTA inspired on-chip metasurface phase array [5, 6]. (A) The schematic representations for free-space HCTA metasurface. (B) Transmission and (C) phase shift versus post diameter and lattice constant [5]. A zero to 2π phase shift and transmission above 90% can be achieved simultaneously. (D) Simulated transmission and phase of the transmission coefficient for a family of periodic hexagonal HCTAs with lattice constant of 800 nm, and varying post diameters. (E) Transmission and phase deviation versus incident angle and slot lengths of the free space 2D HCTA metasurface. (F) Schematics of the integrated metasurface defined in silicon-on-insulator substrate (SOI). (G) Phase shift and (H) transmission versus slot width and slot length. A zero to 2π phase shift and transmission above 90% can be achieved simultaneously for on-chip HCTA metasurfaces. (I) Transmission and phase versus slot lengths with slot width set as 140 nm. (E) The deflection efficiency of the free-space periodic HCTA. (J) Transmission and phase deviation versus incident angle and slot lengths of the on-chip HCTA metasurface. The figures are reproduced with permission. (A)–(E) Ref. [5] © 2015, Nature Publishing Group, a division of Macmillan Publishers Limited. All Rights Reserved. (F)–(J) Ref. [6] © 2019, The Author(s).

3 Ultra-compact, low loss and broadband mode converters by integrated metasurface

Photonic integrated circuits offer an attractive platform but still facing significant challenges, including miniaturizing device footprints, increasing device operation bandwidth and robustness, and reducing device insertion losses [7–10]. Integrated optical waveguides with gradient metasurface structures can help address some of these challenges. Meta-structures have emerged to control guided waves [11–15] and couple guided waves with waves propagating in free space [16–18]. They are arrays of sub-wavelength structures that can apply locally and spatially varying phase shifts to transmitted or reflected electromagnetic (EM) waves [19–21]. The gradient change of the nanostructure (for example, antenna shape, size, and orientation) in the sub-wavelength thin layer can manipulate the out-of-plane EM wave in free space, resulting in the shift from the simple components of the micro flat lens [5, 22–24] and the hologram [25–27] to the more complex analog signal processing system [28, 29] and spectrometers [30]. Especially, metasurfaces can convert on-chip guided mode to single and multiple beams with desired polarization, directionality and divergence, which provides exceptional robustness toward meeting the goal of on-chip atom trapping for quantum sensing applications [31].

The HCTA described in Section 2 provides broadband and low loss mode size conversion within compact dimensions. Xiao et al. designed a low-loss on-chip microsystem based on broadband metalens and a photonic crystal (PhC) resonator [32, 33]. By adjusting the on-chip metalens’ focusing length and mode dimension, the insertion loss between the metalens, the PhC resonator and waveguide structures is minimized through mode-matching. Three different PhC couplers with angles of 38°, 60°, and 120° are designed by removing individual air holes at the two-side edges of PhC to generate an input and output PhC taper [34, 35] (Figure 3A and B). The metalens can be placed on both input and output ports to assist with mode conversion. Figure 2A illustrates the metalens PhC waveguide coupling, and the metalens design allows direct coupling into the PhC cavity. The bilateral metalens system in Figure 3B shows slightly higher insertion loss than the broadband metalens-PhC waveguide design, concerning the tunneling process through the PhC cavity. The mode-size conversion significantly reduces the tapers length connecting the grating coupler and single mode waveguide (Figure 2C). With the integrated metasurface controlled wavefront (Figure 2D), low insertion loss < 1 dB can be achieved within the devices in Figure 2A and B (Figure 2E). The mechanical robustness is desired for improving sensitivity in nanophotonic sensors [36–38] and reducing operation power for PhC based active silicon photonic components. Without the metalens enabled low loss wavefront conversion, an insertion loss of the ≈13.7 µm taper between grating waveguide and single mode waveguide results in ≈18 dB loss (gray curve in Figure 2F). The total insertion loss can be reduced to less than 1 dB from 1480 to 1580 nm. Typical linear table requires more than 200 μm length to achieve similar level of insertion loss as the one in Figure 2C. Other than the transverse metalens described above, gradient ‘longitudinal metasurface’ placed parallel to the wave-propagation direction have been investigated by a number of groups, for inter-mode conversion in a few mode integrated photonic waveguides. The examples are shown in refs. [3, 4].

Figure 2:

Broadband and low loss mode size conversion with gradient transverse metasurface. (A) Device geometry of metalens – PhC WG system with light distribution of |Ey|² on the focal plane. (B) Device geometry of metalens – PhC L3 cavity – metalens system with mode profile at resonance. (C) The SEM image of the metalens enabled ultracompact low loss taper. (D) Zoom-in view of the simulated electrical field distribution of the HCTA based metalens in A–C. (E) Average transmission spectra of the metalens – PhC WG system in (A) and peak transmission of metalens – PhC single modecavity – metalens system in (B), with the PhC coupler angle of 38°, 60°, and 120° [33]. (F) Measured transmission spectra of the device in C (red), compared to the same taper design without metalens (gray) [6]. The figures are reproduced with permission. (A), (B) and (E) ref. [33] © 2021, The Author(s). (C), (D) and (F) ref. [6] © 2019, The Author(s).

$Figure 3: Fourier optic implementation of a spatial differentiator. (A) Schematics of a three-layer spatial differentiation system for diffraction. (B) Scanning microscope images (SEMs) of dielectric [f1] and metallic metasurface; the transmitted wavefront can be designed towards multi-layer metasurface systems. (C) Schematic illustration of a three-layer metasurface system, composed of two metalens for Fourier transformation and middle layer of designed mask [49]. (D) Design of integrated metalens by varying lattice constant and (E) trench length [50]. (F) SEM image of the 1D metasurface defined on SOI substrate. Scale bar: 1 µm. (G) SEM image of a spatial differentiator based on integrated metasurfaces [6]. © 2020, The Author(s).$

Figure 3:

Fourier optic implementation of a spatial differentiator. (A) Schematics of a three-layer spatial differentiation system for diffraction. (B) Scanning microscope images (SEMs) of dielectric [f1] and metallic metasurface; the transmitted wavefront can be designed towards multi-layer metasurface systems. (C) Schematic illustration of a three-layer metasurface system, composed of two metalens for Fourier transformation and middle layer of designed mask [49]. (D) Design of integrated metalens by varying lattice constant and (E) trench length [50]. (F) SEM image of the 1D metasurface defined on SOI substrate. Scale bar: 1 µm. (G) SEM image of a spatial differentiator based on integrated metasurfaces [6]. © 2020, The Author(s).

4 Integrated metasystem for mathematical operations

Most of the optical signal-processing systems are based on the Fourier transform property of a lens. Integrated metasurfaces enable arbitrary phase control of the transmitted light, and one can design integrated metalens and build the integrated 4-f system to perform analog optical signal processing. Each subwavelength unit-cell of a metasurface can be independently designed, toward the goal of achieving the desired transmission [22, 39] and reflection coefficients [40, 41]. Its amplitude of transmission coefficient can be independently controlled by the trench width or lattice constants. With full control of complex transmission coefficient ‘pixel-by-pixel’, metasurface systems can be designed towards complex spatial information transformations. Beyond one-step etching defined metasurface systems, gray-scale nanofabrication creates periodic and quasiperiodic sinusoidal Fourier surfaces with well controlled wavefront shaping capabilities [42].

4.1 Fourier optics and mathematical operation

The Fourier transform capabilities of a lens provide numerous applications in optical signal processing. The patterned surfaces with subwavelength features can interact with incoming beam and the interference of the diffractive wavefronts can serve as conventional lens, where each unit is designed in the way that the amplitude of the transmission coefficient is fixed at ‘1’ and phase needs to be tuned with the nanostructure geometry (also named as ‘phase-only metasurface’). Since the phase and amplitude of the transmission coefficient is usually proportional to the nanostructure dimensions, geometrically gradient variation creates gradient metasurface (e.g., metalens), and abrupt change between adjacent subwavelength unit-cell creates non-gradient metasurface (e.g., spatial Fourier domain filters or advanced functions of machine learning [43, 44]). The prior one performs Fourier transform at focal plane as conventional lens, and the latter one can be inserted at the focal plane as filters in spatial Fourier domain.

One important application of Fourier optics is edge detection [45]. Following the first demonstration of mathematical operation with metasurface [28], spatial differentiation has been implemented in a dielectric nanophotonic slab and metallic plasmonic surface [41, 46–48]. Recent research proposes that the spatial differentiation operator can be implemented with all-metasurface systems, where the first and the third layers of metalens serve as Fourier transformers, and the second metasurface are programed for a desired function t′ (Figure 3A). In free space, such metasurface systems can be composed of two-dimensional array of nanostructures with the same height (in z-direction) (Figure 3B). Cascaded metalens–metasurface–metalens systems perform spatial edge detection (Figure 3C) [49]. The light propagation in planar waveguides can be controlled by in-plane one-dimensional metasurface. On SOI platforms, the amplitude and phase of transmission coefficient for each pixel is determined by lattice constant (Figure 3D) or trench length (Figure 3E) [50]. Figure 2F shows a top view of non-gradient metasurface for in-plane wave. Without active alignment, the lithography defined system perform one-dimensional spatial differentiation (Figure 3G) [6].

4.2 Broadband metasurface convolver

Based on convolution theorem, one can easily achieve convolution using a 4-f system. There are various methods for designing on-chip dielectric metasurfaces, including using the effective medium theory to simplify the metasurface structures, transforming the problem of the structural size design into a problem of refractive index distribution in geometric optics [6, 51]; using the transmission matrix to describe the propagation of optical modes, obtaining metasurface structures by optimizing the transmission matrix, or searching the required structure parameters through the target transmission matrix [52, 53]; In addition to above forward direct design methods to obtain required metasurface structures, structural parameters of the metasurface can also be searched by inverse design [14, 15]. Different from the traditional intuition-based parameters scanning, the design scheme optimizes the optimal structure parameters through intelligent algorithms with greater flexibility and controllability to approach the limits of device performance [54, 55]. Convolution [56] is one of the most basic and important operations in signal processing, especially in the field of signal analysis and image processing. Since all-optical convolution operation gives full play to the ability of optical parallel computing [57, 58], it is one of the most promising directions in all-optical computing. Recent research reports a strategy to utilize genetic algorithm (GA) to assist the design of a dispersionless metalens and a nanophotonic convolver based on silicon metasurface [59].

The dispersionless metalens is realized by two air slot arrays with different periods etched in SOI substrate. The center of focal points for the two wavelengths are displaced less than 100 nm, far less than the nominal focal length of 8 μm, demonstrating the two focal planes can be considered the same to enable this system to perform parallel processing for dual wavelengths simultaneously (Figure 4A–D). The focusing efficiency is as high as 79 and 85% at wavelengths of 1000 and 1550 nm respectively (Figure 4E and F). Two sets of identical dispersionless metalens placed symmetrically about the focal plane form the 4f optical processing system. As shown in Figure 5G and H, a given structure with a certain transmission function is placed on the focal plane of the 4f system. And finally, the convolution result of the transmission function presented by the structure and the spatial waveform of the input signal is obtained at the output ports with the measured deviation of less than 8% (Figure 4I and J). This on-chip nanophotonic convolver provides the basis for more complex integrated all-optical computing tasks.

Figure 4:

Broadband metasurface convolver [59]. (A) and (B) |E _y|², in plane (x–y plane) light distribution of the proposed dispersionless metalens at wavelength of 1550 and 1000 nm, respectively. The white dotted line shows the focal plane. (C) and (D) Cross-section (y–z plane) view of light distribution at the focal plane with wavelength of 1550 and 1000 nm, respectively. (E) and (F) Measured light intensity distribution on focal plane along the y-direction compared with simulation results at the dual wavelengths, respectively. Inset: the scanning electron microscope (SEM) image of the characteristic structure of the dispersionless metalens. The scale bar here is 3 μm. (G) The overall view of SEM image of the proposed convolver. (H) The characteristic structure of the nanophotonic convolver. The area enclosed by the middle dashed box is the given structure with its transmission function t(y′). (I) and (J) Measured spatial convolution results along the y-direction compared with simulation and theoretical results at the two wavelengths, respectively. The figures are reproduced with permission. Ref. [59] © 2020, The Author(s).

Figure 5:

1 × N switch via on-chip tunable metasurface [85]. (A) The fabricated 1 × 11 switch by Advanced Micro Foundry (AMF). (B) Simulated temperature distribution by powering heater 2, and correspondent (B) wavefront of the transmitted wave incident on the second metalens. (C) The simulated and measured optical intensity profile of each output port on the focal plane. (D) Zoom-in optical image of the output ports on the focal plane.

5 Metasystem for machine learning

In the recent years electronics-based deep neural network (DNN) hardware accelerators have been extensively studied in both academia and industry. To date, tremendous efforts from different technical stacks, ranging from device to circuit to architecture, have been devoted to improving DNN hardware performance. At the device level, DNN hardware can be fabricated on standard complementary metal-oxide-semiconductor (CMOS) [60–65] and emerging devices such as memristors [66–68] etc. At the circuit level, both the digital [69–71] and analog circuit design [72–74] can be adopted for implementation strategy. At the architecture level, the classical von Neumann architecture [61, 75] and emerging non-von Neumann architecture [74–78] can serve as the underlying architectural paradigm. Fulfilling DNN in the optical domain has received increasing attention recently. Without considering the phase corrections, ideal optical computing systems inherently offer massive parallelism and power efficiency. Here we compare the metasystem’s performance to the other integrated photonic architectures (Section 5.1) and free-space Fourier-optic image classifiers (Section 5.2).

5.1 Matrix operation in integrated metasurface system

Single-mode waveguides offers the best dispersion control and modal stability. It serves as the fundamental building blocks for broadband directional couplers (for bar, cross and power splitters) and microring resonators (wavelength multiplication). Circuit architectures for signal routing in photonics using cascaded Mach–Zehnder interferometers (MZI) to perform singular value decomposition (SVD) or Fourier transform (FT) is achieved with butterfly circuit topology. Temporal-spectral multiplications implement multi-wavelength modulation and summations [65, 79, 80]. Different from those approaches, the integrated metasurface platform expands the complexity of information processing via diffraction and in-plane free-space propagation, inheriting the integrated family for ‘imaging devices’ (such as multi-mode interferometers and star couplers) [81].

Vector-by-matrix multiplication (VMM) is one of the fundamental operations in the accelerator hardware [82]. Table 1 compares the VMM power efficiency, throughput, and footprint of integrated photonic circuits based on MZI, microring resonators (MRRs), and integrated metasystem.

Table 1:

Integrated photonic frameworks for VMM.

Method for	Multi-wavelength modulation	Singular value	Fourier transform	Diffraction
matrix operation	and summation [79, 80, 87]	decomposition [83]	based [84]	equation [76]
Device architecture	MRR weight banks [79, 87] and directional couplers [80]	MZI [83]	MZI with butterfly-style mesh topology	Cascaded metasurfaces
Fabrication offset correction	Electro-optic tuning	Electro-optic tuning	Electro-optic tuning	Included during the training
Weight matrix	16 × 16 [80]	4 × 4 [83]	16 × 16	450 × 2
Footprint (mm²)	16 [80]	0.75 [83]	0.4	0.135
Throughput (Tb/s)	11 [87]	1 [83]	1.536 [84]	5–10
Insertion loss	27 dB [80]	Not reported	Not reported	15 dB (average)
Operational power	17 fJ per MAC	1 pJ per FLOP	1.7 × 10⁴ fJ per TOPS [84]	10⁻⁵ fJ per FLOP

Calculations of throughput and power consumption in Table 1 are detailed in references [76, 83, 84]. As shown in the table, compared with the other methods for VMM, the integrated metasystem has denser weight elements per footprint, along with lower operational power requirement. Such advantage in accordance with the multimode nature of the diffraction-based method, allows more information capacity. However, lack of tunability limits the usage of the integrated metasystem platform.

5.2 Image classifier

The integrated metasystem can also be designed for FT-based neural network. Compared with FT achieved with butterfly circuit topology, the FT achieved by integrated metalens requires no operation power and has a smaller footprint. Compared with the 2D metasystem in free space, the metasurface on the integrated platform also offers better mechanical robustness, with the advantages of lower insertion loss and feasible fabrications for multi-layer structures. Currently, the main technical challenge is the layout design of a large number of input/output (I/O) ports on an integrated photonic platform with tolerable phase distortions from nanofabrication. Theoretically, a 2D metasurface with subwavelength unit-cells owns significant computing capabilities. However, experimental implementation of such a system for machine learning has never been reported in telecommunication wavelength or infrared, but feasible if the fabrication or alignment errors are considered in the training process. Commercially available components (digital micromirror device, DMD or diffractive optical elements) have a typical cell number of 10⁴ to 10⁶. Single layer component has been utilized for high-accuracy image classifications. The integrated photonic platform can eliminate out-of-plane light diffraction, and thus result in orders of magnitude lower insertion loss compared to free-space optical systems (Table 2).

Table 2:

Comparison of neuron networks-based image classifiers.

Neuron network	Convolution neural network [99]		Convolution neural network [100]	Diffraction neural network (metasystems)
Programmed layer(s)	One amplitude-only layer of DMD		One phase-only layer of diffractive optic elements	Phase-only layers of metasurface
Reconfigurable	Yes		No	No
Postprocessing	Required		Required	Maximal only
Hyperspectral	No		Possible	Yes
Kernel size	16 × 208 × 208		16 × 32 × 32	450 × 2
Dataset	MNIST	CIFAR	CIFAR-10	MNIST
Accuracy	98%(s)	63%(s)	51% (e)	96% (s)
Footprint	Bulk		Bulk	mm²

(s): Numerical simulation result. (e): experimental measurements.

6 Homogeneous multi-pixel switching schemes

Dynamic control of each individual pixel is critical for applications but challenging given high refractive index contrast is required for subwavelength light–matter interaction lengths. Conventional approach for integrated photonic device tunings, such as thermal-optic tuning, carrier injection or depletion, only introduces refractive index difference of around 10⁻³. Reprogrammable metasurface are achieved through tuning the refractive index of cladding liquid crystal [86] or bistable chalcogenide phase change materials [87]. Independent control of each metasurface unit-cell is usually required for full degree of freedom, which can be controlled by laser selective heating [80, 87] or arrays of liquid crystals [86]. Homogeneous change of refractive index, however, can also achieve a few special functions in metalens system with specialized material selection [88] or system design [85].

6.1 Electro-thermal tunability

Metasurface can also been used for high-performance optical switches with optimized heater design as shown in Figure 5A [85]. Focusing point of the metalens can be tuned using the thermo-optic effect. The effective index gradient in the silicon waveguide region caused by the designed temperature profile is shown in Figure 5B. The wavefront produces a phase difference along the y direction when the light wave propagates in the silicon waveguide under electrical power applied to the heater (Figure 5C). The focused beam can be steered towards upper and lower side by using heater 1 and heater 2, respectively. The simulated and measured optical intensity of each output port at the focal plane are plotted and compared in Figure 6D and E. The insertion loss is measured to be around 3–5 dB. The bandwidth exceeds 30 nm.

$Figure 6: Phase change material embedded silicon photonic metalens [88]. (A) Refractive index comparison for the nonvolatile PCM materials at inter-convertible states/phase. (B) Perspective view of the PCM metalens on silicon photonic platform. For parallel light input, the wavefront of the transmitted wave is convex for α-state and concave for β-state. Arrows are the propagation directions, and the curves are the wavefronts of the transmitted light. Inset: cross-sectional view of the device. (C) Electrical intensity distribution for a metalens with β-In2Se3, and (D) 100% α-In2Se3. The dashed curves are the wavefronts.$

Figure 6:

Phase change material embedded silicon photonic metalens [88]. (A) Refractive index comparison for the nonvolatile PCM materials at inter-convertible states/phase. (B) Perspective view of the PCM metalens on silicon photonic platform. For parallel light input, the wavefront of the transmitted wave is convex for α-state and concave for β-state. Arrows are the propagation directions, and the curves are the wavefronts of the transmitted light. Inset: cross-sectional view of the device. (C) Electrical intensity distribution for a metalens with β-In₂Se₃, and (D) 100% α-In₂Se₃. The dashed curves are the wavefronts.

6.2 Focal length switching with embedded phase change materials

Optical phase-change materials (PCMs) offer an appealing material solution for active metasurface devices with their large index contrast and non-volatile switching characteristics [88]. Distinct from the other reported PCMs, the refractive index of silicon is near the middle of the ones for In₂Se₃ in two interconvertible states (Figure 6A), which means the two states introduce phase shift with opposite signs in an In₂Se₃ integrated silicon photonic metasurface [89]. For a designed metalens based on such gradient hybrid metasurface, homogeneous refractive index change of In₂Se₃ leads to dramatic focusing length tuning (Figure 6B). The hybrid integration of In₂Se₃ and silicon photonic does not need any additional lithography step but simple material deposition and standardized planarization. The amplitude of the electric field distribution exhibits clear defocusing (Figure 6C) and focusing behavior (Figure 6D) at 100% β and α-state, respectively. The wavefronts are extracted from the phase distributions at different β and α-states.

As the percentage of α-state reduces from 100 to 70%, the focusing length extracted from the curvature is comparable to the one estimated from the cross-sectional optical intensity profile at y = 0. Near the focal point (x = 3.2 μm, y = 0), the normalized light intensity reduces to 40% as the percentage of α-state reduces from 100 to 70%. The refractive index difference to In₂Se₃/silicon structure can be tuned from −0.4 to +0.7, resulting in focusing length variation from ≈30 to >50 μm, and from <−50 to ≈−25 μm. The reconfigurable transformative optical components have the potential for applications in miniaturized sensors and chip-scale nonlinear optics.

7 Perspectives

7.1 Applications in mode multiplexing and demultiplexing

Within the past decades, advancements in the field of metasurface include (1) optimization and expanding the functionality of the metalens; (2) image processing and optical analog computing, including Fourier transform and differentiation; (3) chirality and polarization control; (4) reconfigurability [90]. In integrated optics, gradient metasurface along the waveguides control the mode-conversion and dispersion of guided waves [3, 91, 92]. Metasurfaces’ mode-conversion capabilities are useful in space division multiplexing for fiber and free-space communications [93]. Low insertion loss and broadband operation are of paramount importance. Arrays of dielectric metasurface devices have demonstrated spatial division multiplexing in fiber communications, which can simultaneously convert the two incomings orthogonally polarized LP01 signals into two different higher-order LP modes (LP11 and LP21), with operational bandwidth covering S-, C- and L-bands [93] (Figure 7A and B). A phase-only metasurface with a spiral phase pattern can convert the incoming Gaussian-like beam to a beam carrying orbital angular momentum (OAM). The metasurface-generated OAM beams have been utilized for optical free-space mode-division-multiplexing communications [94, 95]. Wavelength shift and parametric conversions are demonstrated with time-varying epsilon-near-zero metasurface [96–98]. The nonlinear response of the material introduces light-intensity dependent refractive index shift across and metasurface, resulting in ultrafast pulse shaping and wavefront engineering [97, 98].

Figure 7:

Metasurface generated OAM for the mode multiplexing in fiber and free-space optical communications. (A) Polarization modulation in metasurface. (B) SEM image of the metasurface [93, 94].

Increasing data throughput demands, especially in short-distance communications within computer racks (meters) and between datacenters (kilometers), spatial information multiplication techniques beyond wavelength division multiplication (WDM) [101–103]. Multimode fibers and few-mode fibers offer a few spatial modes and orders of magnitude higher data capacity. On the transmitter end, vertical-cavity surface-emitting lasers generate two-dimensional spatial information with WDM [104]. On the receiver end, silicon photonic mode-division de-multiplexers have attracted attention with their CMOS compatibility and small footprint [2, 105, 106]. The development of low loss multi-mode optical components is distinguished from conventional silicon photonic device designs based on single mode waveguides. The profile of multi-mode waveguides, convertors, bends and cross bars need to be tailored in the way that multiple supported modes can operate with low loss [2]. Through controlling the on-chip wavefront, the dielectric metasurface systems offer compact and low loss designs for converting mode size [6] and mode orders [4]. With machine learning assisted designs, compression of spatial and spectral data information is possible through a cascaded metasurface system [107–109]. Beside the works mentioned above, there are some recent progresses towards tunable metamaterials, such as the vortex nanosieves to generate the optical beams carrying multiplexed OAM both in free-space and in a plasmonic near field [110], the metasurface that can be controlled by brainwaves [111], and nanostructures used for thermally stable solar thermophotovoltaic systems [112]. In [113, 114], the authors review the hyperbolic metamaterials and intelligent metasurfaces.

The integrated metasurface systems can serve as photonic front end for data dimension reduction. As the data rate in multimode fibers are in the scale of Tb/s, the data dimension reduction can reduce the amount of effective data flow and thus the bandwidth requirement of subsequent optoelectronic conversion. With rapid advance of silicon photonics, the typical radio-frequency (RF) – optical bandwidth for the silicon photonic modulators and detectors can reach tens of Gb/s. Reduction of data volume in all-optical formats can ease the device cost and power consumption in the optoelectronic conversion.

7.2 Individual meta-atom/cell programed by laser writing

The small spot size of lasers and high local intensity makes laser-writing a widely used tool. Similar to conventional laser writing lithography, lasers create patterns on chalcogenide materials through non-volatile phase transition. Those devices are low power and reconfigurable in the way that the laser exposed area will remain high refractive index contrast after laser removal. A simple baking resets the patterned film to be homogenous. Delaney et al. demonstrated the feasibility of a laser programmed PCM device (Figure 8) [80]. The localized programming can be applied to reconfigurable nonvolatile integrated photonic devices with minimal thermal cross talk. As an example, C. Choi et al. demonstrated a hologram technique through hybrid state Ge₂Se₂Te₅ (GST) metasurface. Hybrid GST-Al metasurface structures are designed in the way that only at the specific middle states the holograph image is visible. The thermo-optical complexity in hybridized GST-Al metasurface allows the realization of secure cryptography [87].

Figure 8:

Perspective scheme for individual programming meta-atoms with vertically coupled laser. (A) Schematics of optical writing of patterns on a multimode interference router. (B) Numerically calculated electric field distribution in programmed device. The write area is the outline of the silicon device, and the white transparent patches represent the crystalline Sb₂Se₃ in amorphous background. (C) Optical microscope image of the laser programmed optical phase change material. Scale bars: 5 µm [81]. © 2021 The Authors.

8 Summary and perspectives

Given that on-chip dielectric metasurface offers a unique scheme to control degrees of freedom of the on-chip light field, as well as compatible with the standard CMOS fabrication process, it is expected to become one of the promising ways to realize on-chip integrated photonic computing chips with compact footprint, broadband, and low-loss on the basis of basic operations, like the convolution operation mentioned above, which is of great significance to promote the field of modern photonic technology in the future.

In summary, this paper reviews a variety of metasurface systems given its superior performance on spatial information processing with multiplication in spectral domains. As an arbitrary wavefront shaper, the low loss and broadband silicon photonic metasurface can serve as analog photonic accelerators and demultiplexer in MDM-DWDM optical communication. We review its role as Fourier optic mathematic operator for gradient metasurface. With machine learning accelerated reverse design, the metasurface system can perform advanced tasks such as image classification and hyperspectral imaging. Reconfigurability of such metasurface devices are still under development. With special designs and material selections, homogenous change of the refractive index can tune the focusing of the metalens. Individual unit can be reconfigured through local electrodes or laser exposure. Those active metasurface can play versatile and powerful roles in free space and fiber communications.

Corresponding author: Tingyi Gu, Department of Electrical and Computer Engineering, University of Delaware, Newark, DE 19711, USA, E-mail: tingyigu@udel.edu

Zi Wang, Yahui Xiao, Kun Liao, and Tiantian Li, contributed equally.

Funding source: DARPA

Award Identifier / Grant number: N660012114034

Funding source: Jiangsu Innovation Team, National Natural Science Foundation of China (Grant 61775094).

Award Identifier / Grant number: 61775094

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: This work was supported by DARPA (N660012114034) and Jiangsu Innovation Team, National Natural Science Foundation of China (Grant 61775094).
Conflict of interest statement: The authors declare no conflicts of interest regarding this article.

References

[1] L. Chrostowski and M. Hochberg, Silicon Photonics Design, Cambridge University Press, 2015.10.1017/CBO9781316084168Search in Google Scholar

[2] C. Li, D. Liu, and D. Dai, “Multimode silicon photonics,” Nanophotonics, vol. 8, pp. 227–247, 2018. https://doi.org/10.1515/nanoph-2018-0161.Search in Google Scholar

[3] Z. Li, M.-H. Kim, C. Wang, et al.., “Controlling propagation and coupling of waveguide modes using phase-gradient metasurfaces,” Nature Nanotechnol., vol. 12, pp. 675–683, 2017. https://doi.org/10.1038/nnano.2017.50.Search in Google Scholar PubMed

[4] H. Wang, Y. Zhang, Y. He, Q. Zhu, L. Sun, and Y. Su, “Compact silicon waveguide mode converter employing dielectric metasurface structure,” Adv. Opt. Mater., vol. 7, p. 1801191, 2018. https://doi.org/10.1002/adom.201801191.Search in Google Scholar

[5] A. Arbabi, Y. Horie, A. J. Ball, M. Bagheri, and A. Faraon, “Subwavelength-thick lenses with high numerical apertures and large efficiency based on high-contrast transmitarrays,” Nat. Commun., vol. 6, p. 7069, 2015. https://doi.org/10.1038/ncomms8069.Search in Google Scholar PubMed

[6] Z. Wang, T. Li, A. Soman, D. Mao, T. Kananen, and T. Gu, “On-chip wavefront shaping with dielectric metasurface,” Nat. Commun., vol. 10, p. 3547, 2019. https://doi.org/10.1038/s41467-019-11578-y.Search in Google Scholar PubMed PubMed Central

[7] R. Soref, “The past, present, and future of silicon photonics,” IEEE J. Select. Topics Quantum Electron., vol. 12, pp. 1678–1687, 2006. https://doi.org/10.1109/jstqe.2006.883151.Search in Google Scholar

[8] R. Soref, “Mid-infrared photonics in silicon and germanium,” Nat. Photonics, vol. 4, pp. 495–497, 2010. https://doi.org/10.1038/nphoton.2010.171.Search in Google Scholar

[9] K. Wada, H.-C. Luan, D. R. Lim, and L. C. Kimerling, “On-chip interconnection beyond semiconductor roadmap: silicon microphotonics,” Proc. SPIE 4870, Active and Passive Optical Components for WDM Communications II, 2002. https://doi.org/10.1117/12.475558.Search in Google Scholar

[10] B. Jalali and S. Fathpour, “Silicon photonics,” J. Lightwave Technol., vol. 24, pp. 4600–4615, 2006. https://doi.org/10.1109/jlt.2006.885782.Search in Google Scholar

[11] A. Vakil and N. Engheta, “Transformation optics using graphene,” Science, vol. 332, pp. 1291–1294, 2011. https://doi.org/10.1126/science.1202691.Search in Google Scholar PubMed

[12] G. Gok and A. Grbic, “Tailoring the phase and power flow of electromagnetic fields,” Phys. Rev. Lett., vol. 111, p. 233904, 2013. https://doi.org/10.1103/physrevlett.111.233904.Search in Google Scholar PubMed

[13] L. Feng, Y. L. Xu, W. S. Fegadolli, et al.., “Experimental demonstration of a unidirectional reflectionless parity-time metamaterial at optical frequencies,” Nat. Mater., vol. 12, pp. 108–113, 2013. https://doi.org/10.1038/nmat3495.Search in Google Scholar PubMed

[14] A. Y. Piggott, J. Lu, K. G. Lagoudakis, J. Petykiewicz, T. M. Babinec, and J. Vučković, “Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer,” Nat. Photonics, vol. 9, pp. 374–377, 2015. https://doi.org/10.1038/nphoton.2015.69.Search in Google Scholar

[15] B. Shen, P. Wang, R. Polson, and R. Menon, “An integrated-nanophotonics polarization beamsplitter with 2.4 × 2.4 μm2 footprint,” Nat. Photonics, vol. 9, pp. 378–382, 2015. https://doi.org/10.1038/nphoton.2015.80.Search in Google Scholar

[16] S. Sun, O. He, S. Xiao, Q. Xu, X. Li, and L. Zhou, “Gradient-index meta-surfaces as a bridge linking propagating waves and surface waves,” Nat. Mater., vol. 11, pp. 426–431, 2012. https://doi.org/10.1038/nmat3292.Search in Google Scholar PubMed

[17] W. Sun, Q. He, S. Sun, and L. Zhou, “High-efficiency surface plasmon meta-couplers: concept and microwave-regime realizations,” Light Sci. Appl., vol. 5, p. e16003, 2016. https://doi.org/10.1038/lsa.2016.3.Search in Google Scholar PubMed PubMed Central

[18] J. Lin, J. P. B. Mueller, Q. Wang, et al.., “Polarization-controlled tunable directional coupling of surface plasmon polaritons,” Science, vol. 340, pp. 331–334, 2013. https://doi.org/10.1126/science.1233746.Search in Google Scholar PubMed

[19] N. Yu and F. Capasso, “Flat optics with designer metasurfaces,” Nat. Mater., vol. 13, pp. 139–150, 2014. https://doi.org/10.1038/nmat3839.Search in Google Scholar PubMed

[20] S. Jahani and Z. Jacob, “All-dielectric metamaterials,” Nat. Nanotechnol., vol. 11, pp. 23–36, 2016. https://doi.org/10.1038/nnano.2015.304.Search in Google Scholar PubMed

[21] E. Arbabi, A. Arbabi, S. M. Kamali, Y. Horie, and A. Faraon, “Controlling the sign of chromatic dispersion in diffractive optics with dielectric metasurfaces,” Optica, vol. 4, p. 625, 2017. https://doi.org/10.1364/optica.4.000625.Search in Google Scholar

[22] A. Arbabi, Y. Horie, M. Bagheri, and A. Faraon, “Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission,” Nat. Nanotechnol., vol. 10, pp. 937–943, 2015. https://doi.org/10.1038/nnano.2015.186.Search in Google Scholar PubMed

[23] X. Chen, L. Huang, H. Mühlenbernd, et al.., “Dual-polarity plasmonic metalens for visible light,” Nat. Commun., vol. 3, p. 1198, 2012. https://doi.org/10.1038/ncomms2207.Search in Google Scholar PubMed PubMed Central

[24] D. Lin, P. Fan, E. Hasman, and M. L. Brongersma, “Dielectric gradient metasurface optical elements,” Science, vol. 345, pp. 298–302, 2014. https://doi.org/10.1126/science.1253213.Search in Google Scholar PubMed

[25] G. Zheng, H. Mühlenbernd, M. Kenney, G. Li, T. Zentgraf, and S. Zhang, “Metasurface holograms reaching 80% efficiency,” Nat. Nanotechnol., vol. 10, pp. 308–312, 2015. https://doi.org/10.1038/nnano.2015.2.Search in Google Scholar PubMed

[26] L. Huang, X. Chen, H. Mühlenbernd, et al.., “Three-dimensional optical holography using a plasmonic metasurface,” Nat. Commun., vol. 4, p. 2808, 2013. https://doi.org/10.1038/ncomms3808.Search in Google Scholar

[27] X. Ni, A. V. Kildishev, and V. M. Shalaev, “Metasurface holograms for visible light,” Nat. Commun., vol. 4, p. 2807, 2013. https://doi.org/10.1038/ncomms3807.Search in Google Scholar

[28] A. Silva, F. Monticone, G. Castaldi, V. Galdi, A. Alù, and N. Engheta, “Performing mathematical operations with metamaterials,” Science, vol. 343, pp. 160–163, 2014. https://doi.org/10.1126/science.1242818.Search in Google Scholar PubMed

[29] A. Pors, M. G. Nielsen, and S. I. Bozhevolnyi, “Analog computing using reflective plasmonic metasurfaces,” Nano Lett., vol. 15, pp. 791–797, 2015. https://doi.org/10.1021/nl5047297.Search in Google Scholar PubMed

[30] M. Khorasaninejad, W. T. Chen, J. Oh, and F. Capasso, “Super-dispersive off-Axis meta-lenses for compact high resolution spectroscopy,” Nano Lett., vol. 16, pp. 3732–3737, 2016. https://doi.org/10.1021/acs.nanolett.6b01097.Search in Google Scholar PubMed

[31] A. Yulaev, W. Zhu, C. Zhang, et al.., “Metasurface-integrated photonic platform for versatile free-space beam projection with polarization control,” ACS Photonics, vol. 6, no. 11, pp. 2902–2909, 2019. https://doi.org/10.1021/acsphotonics.9b01000.Search in Google Scholar

[32] F. Wang, Y. Xiao, T. Kananen, et al.., “Low-loss photonic crystal platform by foundry processing,” in Conference on Lasers and Electro-Optics, vol. 28, San Jose, CA, OSA, 2020, p. JTu2D.10.1364/CLEO_AT.2020.JTu2D.28Search in Google Scholar

[33] Y. Xiao, Z. Wang, F. Wang, et al.., “Engineering the light coupling between metalens and photonic crystal resonators for robust on-chip microsystems,” J. Opt. Microsyst., vol. 1, p. 024001, 2021. https://doi.org/10.1117/1.jom.1.2.024001.Search in Google Scholar

[34] T. D. Happ, M. Kamp, and A. Forchel, “Photonic crystal tapers for ultracompact mode conversion,” Opt. Lett., vol. 26, p. 1102, 2001. https://doi.org/10.1364/ol.26.001102.Search in Google Scholar PubMed

[35] P. Sanchis, J. Martí, J. Blasco, A. Martínez, and A. García, “Mode matching technique for highly efficient coupling between dielectric waveguides and planar photonic crystal circuits,” Opt. Express, vol. 10, p. 1391, 2002. https://doi.org/10.1364/oe.10.001391.Search in Google Scholar PubMed

[36] Y. Liu and H. W. M. Salemink, “Real-time dynamic sensing with an on-chip nanophotonic sensor,” Opt. Express, vol. 25, p. 17201, 2017. https://doi.org/10.1364/oe.25.017201.Search in Google Scholar PubMed

[37] N. Skivesen, A. Têtu, M. Kristensen, J. Kjems, L. H. Frandsen, and P. I. Borel, “Photonic-crystal waveguide biosensor,” Opt. Express, vol. 15, p. 3169, 2007. https://doi.org/10.1364/oe.15.003169.Search in Google Scholar PubMed

[38] R. Chandrasekar, Z. J. Lapin, A. S. Nichols, R. M. Braun, and A. W. Fountain, “Photonic integrated circuits for Department of Defense-relevant chemical and biological sensing applications: state-of-the-art and future outlooks,” Opt. Eng., vol. 58, p. 1, 2019. https://doi.org/10.1117/1.oe.58.2.020901.Search in Google Scholar

[39] M. Khorasaninejad, W. T. Chen, R. C. Devlin, J. Oh, A. Y. Zhu, and F. Capasso, “Metalenses at visible wavelengths: diffraction-limited focusing and subwavelength resolution imaging,” Science, vol. 352, pp. 1190–1194, 2016. https://doi.org/10.1126/science.aaf6644.Search in Google Scholar PubMed

[40] A. Arbabi, E. Arbabi, Y. Horie, S. M. Kamali, and A. Faraon, “Planar metasurface retroreflector,” Nat. Photonics, vol. 11, pp. 415–420, 2017. https://doi.org/10.1038/nphoton.2017.96.Search in Google Scholar

[41] Y. Zhou, W. Wu, R. Chen, W. Chen, R. Chen, and Y. Ma, “Analog optical spatial differentiators based on dielectric metasurfaces,” Adv. Opt. Mater., vol. 8, p. 1901523, 2020. https://doi.org/10.1002/adom.201901523.Search in Google Scholar

[42] N. Lassaline, R. Brechbühler, S. J. W. Vonk, et al.., “Optical fourier surfaces,” Nature, vol. 582, pp. 506–510, 2020. https://doi.org/10.1038/s41586-020-2390-x.Search in Google Scholar PubMed PubMed Central

[43] A. S. Backer, “Computational inverse design for cascaded systems of metasurface optics,” Opt. Express, vol. 27, p. 30308, 2019. https://doi.org/10.1364/oe.27.030308.Search in Google Scholar

[44] S. Zarei, M. Marzban, and A. Khavasi, “Integrated photonic neural network based on silicon metalines,” Opt. Express, vol. 28, p. 36668, 2020. https://doi.org/10.1364/oe.404386.Search in Google Scholar PubMed

[45] D. Marr and E. Hilldreth, “Theory of edge detection,” Proc. R. Soc. Lond. B, vol. 207, pp. 187–217, 1980. https://doi.org/10.1098/rspb.1980.0020.Search in Google Scholar PubMed

[46] Y. Zhou, H. Zheng, I. I. Kravchenko, and J. Valentine, “Flat optics for image differentiation,” Nat. Photonics, vol. 14, pp. 316–323, 2020. https://doi.org/10.1038/s41566-020-0591-3.Search in Google Scholar

[47] C. Guo, M. Xiao, M. Minkov, Y. Shi, and S. Fan, “Photonic crystal slab Laplace operator for image differentiation,” Optica, vol. 5, p. 251, 2018. https://doi.org/10.1364/optica.5.000251.Search in Google Scholar

[48] T. Zhu, Y. Zhou, Y. Lou, H. Ye, M. Qiu, Z. Ruan, et al.., “Plasmonic computing of spatial differentiation,” Nat. Commun., vol. 8, p. 15391, 2017. https://doi.org/10.1038/ncomms15391.Search in Google Scholar PubMed PubMed Central

[49] S. Abdollahramezani, O. Hemmatyar, and A. Adibi, “Meta-optics for spatial optical analog computing,” Nanophotonics, vol. 9, pp. 4075–4095, 2020. https://doi.org/10.1515/nanoph-2020-0285.Search in Google Scholar

[50] R. Yang, R. Yang, Y. Shi, C. Dai, S. Wan, and Z. Li, “On-chip metalenses based on one-dimensional gradient trench in the broadband visible,” Opt. Lett., vol. 45, p. 5640, 2020. https://doi.org/10.1364/ol.405446.Search in Google Scholar PubMed

[51] Y. Fan, X. Le Roux, A. Korovin, A. Lupu, and A. de Lustrac, “Integrated 2D-graded index plasmonic lens on a silicon waveguide for operation in the near infrared domain,” ACS Nano, vol. 11, pp. 4599–4605, 2017. https://doi.org/10.1021/acsnano.7b00150.Search in Google Scholar PubMed

[52] J. Guo, C. Ye, C. Liu, et al.., “Ultra-compact and ultra-broadband guided-mode exchangers on silicon,” Laser Photonics Rev., vol. 14, p. 2000058, 2020. https://doi.org/10.1002/lpor.202000058.Search in Google Scholar

[53] C. Yao, Y. Wang, J. Zhang, et al.., “Dielectric nanoaperture metasurfaces in silicon waveguides for efficient and broadband mode conversion with an ultrasmall footprint,” Adv. Opt. Mater., vol. 8, p. 2000529, 2020. https://doi.org/10.1002/adom.202000529.Search in Google Scholar

[54] S. Molesky, Z. Lin, A. Y. Piggott, W. Jin, J. Vucković, and A. W. Rodriguez, “Inverse design in nanophotonics,” Nat. Photonics, vol. 12, pp. 659–670, 2018. https://doi.org/10.1038/s41566-018-0246-9.Search in Google Scholar

[55] K. Yao, R. Unni, and Y. Zheng, “Intelligent nanophotonics: merging photonics and artificial intelligence at the nanoscale,” Nanophotonics, vol. 8, pp. 339–366, 2019. https://doi.org/10.1515/nanoph-2018-0183.Search in Google Scholar PubMed PubMed Central

[56] A. A. Dominguez, “History of the convolution operation [retrospectroscope],” IEEE Pulse, vol. 6, pp. 38–49, 2015. https://doi.org/10.1109/mpul.2014.2366903.Search in Google Scholar

[57] X. Xu, M. Tan, B. Corcoran, et al.., “11 TOPS photonic convolutional accelerator for optical neural networks,” Nature, vol. 589, pp. 44–51, 2021. https://doi.org/10.1038/s41586-020-03063-0.Search in Google Scholar PubMed

[58] J. Feldmann, N. Youngblood, M. Karpov, et al.., “Parallel convolutional processing using an integrated photonic tensor core,” Nature, vol. 589, pp. 52–58, 2021. https://doi.org/10.1038/s41586-020-03070-1.Search in Google Scholar PubMed

[59] K. Liao, T. Gan, X. Hu, and Q. Gong, “AI-assisted on-chip nanophotonic convolver based on silicon metasurface,” Nanophotonics, vol. 9, pp. 3315–3322, 2020. https://doi.org/10.1515/nanoph-2020-0069.Search in Google Scholar

[60] F. Tu, W. Wu, S. Yin, L. Liu, and S. Wei, “RANA: towards efficient neural acceleration with refresh-optimized embedded DRAM,” in 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), IEEE, 2018, pp. 340–352.10.1109/ISCA.2018.00037Search in Google Scholar

[61] T. Chen, Z. Du, N. Sun, et al.., “DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning,” SIGARCH Comput. Archit. News, vol. 42, pp. 269–284, 2014. https://doi.org/10.1145/2654822.2541967.Search in Google Scholar

[62] S. Zhang, Z. Du, L. Zhang, et al.., “Cambricon-X: An accelerator for sparse neural networks,” in 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), IEEE, 2016, pp. 1–12.10.1109/MICRO.2016.7783723Search in Google Scholar

[63] J.-F. Zhang, C.-E. Lee, C. Liu, Y. S. Shao, S. W. Keckler, and Z. Zhang, “SNAP: a 1.67—21.55TOPS/W sparse neural acceleration processor for unstructured sparse deep neural network inference in 16 nm CMOS,” in 2019 Symposium on VLSI Circuits, IEEE, 2019, pp. C306–C307.10.23919/VLSIC.2019.8778193Search in Google Scholar

[64] P. C. Knag, G. K. Chen, H. E. Sumbul, et al.., “A 617 TOPS/W all digital binary neural network accelerator in 10 nm FinFET CMOS,” in 2020 IEEE Symposium on VLSI Circuits, IEEE, 2020, pp. 1–2.10.1109/VLSICircuits18222.2020.9162949Search in Google Scholar

[65] R. Khaddam-Aljameh, M. Stanisavljevic, J. Fornt Mas, et al.., “HERMES Core – a 14 nm CMOS and PCM-based in-memory compute core using an array of 300 ps/LSB linearized CCO-based ADCs and local digital processing,” in 2021 Symposium on VLSI Circuits, IEEE, 2021, pp. 1–2.10.23919/VLSICircuits52068.2021.9492362Search in Google Scholar

[66] M. Cheng, L. Xia, Z. Zhu, et al.., “TIME: a training-in-memory architecture for memristor-based deep neural networks,” in Proceedings of the 54th Annual Design Automation Conference 2017, ACM, 2017, pp. 1–6.10.1145/3061639.3062326Search in Google Scholar

[67] B. Yan, J. Yang, Q. Wu, Y. Chen, and H. Li, “A closed-loop design to enhance weight stability of memristor based neural network chips,” in 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), IEEE, 2017, pp. 541–548.10.1109/ICCAD.2017.8203824Search in Google Scholar

[68] A. Ankit, I. E. Hajj, S. R. Chalamalasetti, et al.., “PUMA: a programmable ultra-efficient memristor-based accelerator for machine learning inference,” in Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, ACM, 2019, pp. 715–731.10.1145/3297858.3304049Search in Google Scholar

[69] A. Sayal, S. Fathima, S. S. T. Nibhanupudi, and J. P. Kulkarni, “14.4 all-digital time-domain CNN engine using bidirectional memory delay lines for energy-efficient edge computing,” in 2019 IEEE International Solid-State Circuits Conference – (ISSCC), IEEE, 2019, pp. 228–230.10.1109/ISSCC.2019.8662510Search in Google Scholar

[70] K. Yamamoto, K. Kawamura, K. Ando, et al.., “STATICA: a 512-spin 0.25 m-weight annealing processor with an all-spin-updates-at-once architecture for combinatorial optimization with complete spin–spin interactions,” IEEE J. Solid State Circ., vol. 56, pp. 165–178, 2021. https://doi.org/10.1109/jssc.2020.3027702.Search in Google Scholar

[71] Y.-D. Chih, P.-H. Lee, H. Fujiwara, et al.., “16.4 an 89TOPS/W and 16.3TOPS/mm2 all-digital SRAM-based full-precision compute-in memory macro in 22 nm for machine-learning edge applications,” in 2021 IEEE International Solid- State Circuits Conference (ISSCC), IEEE, 2021, pp. 252–254.10.1109/ISSCC42613.2021.9365766Search in Google Scholar

[72] F. N. Buhler, P. Brown, J. Li, T. Chen, Z. Zhang, and M. P. Flynn, “A 3.43TOPS/W 48.9 pJ/pixel 50.1 nJ/classification 512 analog neuron sparse coding neural network with on-chip learning and classification in 40 nm CMOS,” in 2017 Symposium on VLSI Circuits, IEEE, 2017, pp. C30–C31.10.23919/VLSIC.2017.8008536Search in Google Scholar

[73] Q. Liu, B. Gao, P. Yao, et al.., “33.2 A fully integrated analog ReRAM based 78.4TOPS/W compute-in-memory chip with fully parallel MAC computing,” in 2020 IEEE International Solid-State Circuits Conference – (ISSCC), IEEE, 2020, pp. 500–502.10.1109/ISSCC19947.2020.9062953Search in Google Scholar

[74] D. Saito, T. Kobayashi, H. Koga, et al.., “Analog in-memory computing in FeFET-based 1T1R array for edge AI applications,” in 2021 Symposium on VLSI Circuits, IEEE, 2021, pp. 1–2.10.23919/VLSICircuits52068.2021.9492479Search in Google Scholar

[75] Y.-H. Chen, J. Emer, and V. Sze, “Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks,” in 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), IEEE, 2016, pp. 367–379.10.1145/3007787.3001177Search in Google Scholar

[76] Z. Wang, L. Chang, F. Wang, T. Li, and T. Gu, “Integrated photonic metasystem for image classifications at telecommunication wavelength,” Nat. Commun., vol. 13, p. 2131, 2022. https://doi.org/10.1038/s41467-022-29856-7.Search in Google Scholar PubMed PubMed Central

[77] Y. Kwon, Y. Lee, and M. Rhu, “TensorDIMM: a practical near-memory processing architecture for embeddings and tensor operations in deep learning,” in Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, ACM, 2019, pp. 740–753.10.1145/3352460.3358284Search in Google Scholar

[78] B. Asgari, R. Hadidi, J. Cao, D. E. Shim, S.-K. Lim, and H. Kim, “FAFNIR: accelerating sparse gathering by using efficient near-memory intelligent reduction,” in 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), IEEE, 2021, pp. 908–920.10.1109/HPCA51647.2021.00080Search in Google Scholar

[79] A. N. Tait, T. F. de Lima, E. Zhou, et al.., “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep., vol. 7, p. 7430, 2017. https://doi.org/10.1038/s41598-017-07754-z.Search in Google Scholar PubMed PubMed Central

[80] M. Delaney, I. Zeimpekis, H. Du, et al.., “Nonvolatile programmable silicon photonics using an ultralow-loss Sb2Se3 phase change material,” Sci. Adv., vol. 7, p. eabg3500, 2021. https://doi.org/10.1126/sciadv.abg3500.Search in Google Scholar PubMed PubMed Central

[81] R. Osgood and X. Meng, Principles of Photonic Integrated Circuits: Materials, Device Physics, Guided Wave Design, Springer International Publishing, 2021.10.1007/978-3-030-65193-0Search in Google Scholar

[82] K. Kitayama, M. Notomi, M. Naruse, K. Inoue, S. Kawakami, and A. Uchida, “Novel Frontier of photonics for data processing—photonic accelerator,” APL Photonics, vol. 4, p. 090901, 2019. https://doi.org/10.1063/1.5108912.Search in Google Scholar

[83] Y. Shen, N. C. Harris, S. Skirlo, et al.., “Deep learning with coherent nanophotonic circuits,” Nat. Photonics, vol. 11, pp. 441–446, 2017. https://doi.org/10.1038/nphoton.2017.93.Search in Google Scholar

[84] C. Feng, J. Gu, H. Zhu, et al.., “Silicon photonic subspace neural chip for hardware-efficient deep learning,” arXiv preprint arXiv:2111.06705, 2021.Search in Google Scholar

[85] Y. Liu, X. Wang, Y. Yao, J. Du, Q. Song, and K. Xu, “Scalable 1 × N switch via on-chip tunable metasurface,” in Optical Fiber Communication Conference (OFC) 2021 Th4E.7, OSA, 2021.10.1364/OFC.2021.Th4E.7Search in Google Scholar

[86] J. Li, P. Yu, S. Zhang, and N. Liu, “Electrically-controlled digital metasurface device for light projection displays,” Nat. Commun., vol. 11, p. 3574, 2020. https://doi.org/10.1038/s41467-020-17390-3.Search in Google Scholar PubMed PubMed Central

[87] C. Choi, S. Mun, J. Sung, K. Choi, S. Lee, and B. Lee, “Hybrid state engineering of phase-change metasurface for all-optical cryptography,” Adv. Funct. Mater., vol. 31, p. 2007210, 2021. https://doi.org/10.1002/adfm.202007210.Search in Google Scholar

[88] N. Wang, T. Li, B. Sun, Z. Wang, L. Zhou, and T. Gu, “Focusing and defocusing switching of an indium selenide-silicon photonic metalens,” Opt. Lett., vol. 46, p. 4088, 2021. https://doi.org/10.1364/ol.427386.Search in Google Scholar

[89] T. Li, Y. Wang, H. Xing, et al.., “Nonvolatile switching in In2Se3-silicon microring resonators,” in Conference on Lasers and Electro-Optics SM4B.5, OSA, 2021.10.1364/CLEO_SI.2021.SM4B.5Search in Google Scholar

[90] D. Neshev and I. Aharonovich, “Optical metasurfaces: new generation building blocks for multi-functional optics,” Light Sci. Appl., vol. 7, p. 58, 2018. https://doi.org/10.1038/s41377-018-0058-1.Search in Google Scholar PubMed PubMed Central

[91] C. Wang, Z. Li, M.-H. Kim, et al.., “Metasurface-assisted phase-matching-free second harmonic generation in lithium niobate waveguides,” Nat. Commun., vol. 8, p. 2098, 2017. https://doi.org/10.1038/s41467-017-02189-6.Search in Google Scholar PubMed PubMed Central

[92] C. Wu, X. Yang, H. Yu, et al.., Harnessing Optoelectronic Noises in a Hybrid Photonic Generative Adversarial Network (GAN), 2021. Available at: https://www.researchsquare.com/article/rs-795091/v1.10.21203/rs.3.rs-795091/v1Search in Google Scholar

[93] S. Kruk, F. Ferreira, N. Mac-Suibhne, et al.., “Dielectric metasurfaces for optical communications and spatial division multiplexing,” in Conference on Lasers and Electro-Optics FW3H.4, OSA, 2018.10.1364/CLEO_QELS.2018.FW3H.4Search in Google Scholar

[94] Y. Ren, L. Li, Z. Wang, et al.., “Orbital angular momentum-based space division multiplexing for high-capacity underwater optical communications,” Sci. Rep., vol. 6, p. 33306, 2016. https://doi.org/10.1038/srep33306.Search in Google Scholar PubMed PubMed Central

[95] Z. Wang, Y. Yan, A. Arbabi, et al.., “Orbital angular momentum beams generated by passive dielectric phase masks and their performance in a communication link,” Opt. Lett., vol. 42, p. 2746, 2017. https://doi.org/10.1364/ol.42.002746.Search in Google Scholar PubMed

[96] K. Pang, M. Z. Alam, Y. Zhou, et al.., “Adiabatic frequency conversion using a time-varying epsilon-near-zero metasurface,” Nano Lett., vol. 21, pp. 5907–5913, 2021. https://doi.org/10.1021/acs.nanolett.1c00550.Search in Google Scholar PubMed

[97] C. Liu, M. Z. Alam, K. Pang, et al.., “Photon acceleration using a time-varying epsilon-near-zero metasurface,” ACS Photonics, vol. 8, pp. 716–720, 2021. https://doi.org/10.1021/acsphotonics.0c01929.Search in Google Scholar

[98] L. Wang, S. Kruk, K. Koshelev, I. Kravchenko, B. Luther-Davies, and Y. Kivshar, “Nonlinear wavefront control with all-dielectric metasurfaces,” Nano Lett., vol. 18, pp. 3978–3984, 2018. https://doi.org/10.1021/acs.nanolett.8b01460.Search in Google Scholar PubMed

[99] M. Miscuglio, Z. Hu, S. Li, et al.., “Massively parallel amplitude-only Fourier neural network,” Optica, vol. 7, p. 1812, 2020. https://doi.org/10.1364/optica.408659.Search in Google Scholar

[100] J. Chang, V. Sitzmann, X. Dun, W. Heidrich, and G. Wetzstein, “Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification,” Sci. Rep., vol. 8, p. 12324, 2018. https://doi.org/10.1038/s41598-018-30619-y.Search in Google Scholar PubMed PubMed Central

[101] B. G. Lee, D. M. Kuchta, F. E. Doany, et al.., “End-to-end multicore multimode fiber optic link operating up to 120 Gb/s,” J. Lightwave Technol., vol. 30, pp. 886–892, 2012. https://doi.org/10.1109/jlt.2012.2183853.Search in Google Scholar

[102] B. G. Lee, D. M. Kuchta, F. E. Doany, et al.., “Multimode transceiver for interfacing to multicore graded-index fiber capable of carrying 120-Gb/s over 100-m lengths,” in 2010 IEEE Photonic Society’s 23rd Annual Meeting, IEEE, 2010, pp. 564–565.10.1109/PHOTONICS.2010.5699012Search in Google Scholar

[103] D. J. Richardson, J. M. Fini, and L. E. Nelson, “Space-division multiplexing in optical fibres,” Nat. Photonics, vol. 7, pp. 354–362, 2013. https://doi.org/10.1038/nphoton.2013.94.Search in Google Scholar

[104] H. Chen, C. Li, N. K. Fontaine, et al.., “10-Mode-Multiplexed transmitter employing 2-D VCSEL matrix,” in 2021 European Conference on Optical Communication (ECOC), IEEE, 2021, pp. 1–4.10.1109/ECOC52684.2021.9605822Search in Google Scholar

[105] L.-W. Luo, N. Ophir, et al.., “WDM-compatible mode-division multiplexing on a silicon chip,” Nat. Commun., vol. 5, p. 3069, 2014. https://doi.org/10.1038/ncomms4069.Search in Google Scholar PubMed

[106] B. Stern, X. Zhu, C. P. Chen, et al.., “On-chip mode-division multiplexing switch,” Optica, vol. 2, p. 530, 2015. https://doi.org/10.1364/optica.2.000530.Search in Google Scholar

[107] Z. Wang, L. Chang, and T. Gu, “Integrated metasystem for image recognition,” in 2021 IEEE Photonics Society Summer Topicals Meeting Series (SUM), IEEE, 2021, pp. 1–2.10.1109/SUM48717.2021.9505827Search in Google Scholar

[108] J. Zhang, S. Chen, D. Wang, et al.., “Analog optical deconvolution computing for wavefront coding based on nanoantennas metasurfaces,” Opt. Express, vol. 29, p. 32196, 2021. https://doi.org/10.1364/oe.439106.Search in Google Scholar PubMed

[109] C. Chen, W. Qi, Y. Yu, and X. Zhang, “On-chip optical spatial-domain integrator based on Fourier optics and metasurface,” Nanophotonics, vol. 10, pp. 2481–2486, 2021. https://doi.org/10.1515/nanoph-2021-0137.Search in Google Scholar

[110] Z. Jin, D. Janoschka, J. Deng, et al.., “Phyllotaxis-inspired nanosieves with multiplexed orbital angular momentum,” eLight, vol. 1, no. 1, pp. 1–11, 2021. https://doi.org/10.1186/s43593-021-00005-9.Search in Google Scholar

[111] R. Zhu, J. Wang, T. Qiu, et al.., “Remotely mind-controlled metasurface via brainwaves,” eLight, vol. 2, no. 1, pp. 1–11, 2022. https://doi.org/10.1186/s43593-022-00016-0.Search in Google Scholar

[112] M. Abbas, J. Kim, A. S. Rana, I. Kim, B. Rehman, Z. Ahmad, et al.., “Nanostructured chromium-based broadband absorbers and emitters to realize thermally stable solar thermophotovoltaic systems,” Nanoscale, vol. 14, no. 17, pp. 6425–6436, 2022. https://doi.org/10.1039/d1nr08400c.Search in Google Scholar PubMed

[113] D. Lee, S. So, G. Hu, et al.., “Hyperbolic metamaterials: fusing artificial structures to natural 2D materials,” eLight, vol. 2, no. 1, pp. 1–23, 2022. https://doi.org/10.1186/s43593-021-00008-6.Search in Google Scholar

[114] L. Li, H. Zhao, C. Liu, L. Li, and T. J. Cui, “Intelligent metasurfaces: control, communication and computing,” eLight, vol. 2, no. 1, pp. 1–24, 2022. https://doi.org/10.1186/s43593-022-00013-3.Search in Google Scholar

Received: 2022-05-24

Accepted: 2022-06-30

Published Online: 2022-07-20

This work is licensed under the Creative Commons Attribution 4.0 International License.

Metasurface on integrated photonic platform: from mode converters to machine learning

Abstract

1 Introduction

2 Design principle of integrated meta-atom/metasurface cell

3 Ultra-compact, low loss and broadband mode converters by integrated metasurface

4 Integrated metasystem for mathematical operations

4.1 Fourier optics and mathematical operation

4.2 Broadband metasurface convolver

5 Metasystem for machine learning

5.1 Matrix operation in integrated metasurface system

5.2 Image classifier

6 Homogeneous multi-pixel switching schemes

6.1 Electro-thermal tunability

6.2 Focal length switching with embedded phase change materials

7 Perspectives

7.1 Applications in mode multiplexing and demultiplexing

7.2 Individual meta-atom/cell programed by laser writing

8 Summary and perspectives

References

Journal and Issue

Articles in the same Issue