Abstract
Computer generated optical volume elements have been investigated for information storage, spectral filtering, and imaging applications. Advancements in additive manufacturing (3D printing) allow the fabrication of multilayered diffractive volume elements in the micro-scale. For a micro-scale multilayer design, an optimization scheme is needed to calculate the layers. The conventional way is to optimize a stack of 2D phase distributions and implement them by translating the phase into thickness variation. Optimizing directly in 3D can improve field reconstruction accuracy. Here we propose an optimization method by inverting the intended use of Learning Tomography, which is a method to reconstruct 3D phase objects from experimental recordings of 2D projections of the 3D object. The forward model in the optimization is the beam propagation method (BPM). The iterative error reduction scheme and the multilayer structure of the BPM are similar to neural networks. Therefore, this method is referred to as Learning Tomography. Here, instead of imaging an object, we reconstruct the 3D structure that performs the desired task as defined by its input-output functionality. We present the optimization methodology, the comparison by simulation work and the experimental verification of the approach. We demonstrate an optical volume element that performs angular multiplexing of two plane waves to yield two linearly polarized fiber modes in a total volume of 128 μm by 128 μm by 170 μm.
1 Introduction
Optical volume elements (OVEs) can be defined as media whose transmission or reflection properties strongly depend on the spatial or spectral shape of the incident beam. Examples to OVEs consist of fiber Bragg gratings, volume holograms, photonic crystals, photonic lanterns and multiple layers in stratified fashion. Volume holographic elements in particular have been investigated for information storage and processing applications [1], [2], [3]. For example, multiple pages of information (i. e., two-dimensional spatial signal, analog or digital) or holograms can be stored or multiplexed in a volume element, and they can be accessed selectively either by utilizing different angles, wavelengths or phase distributions. Periodic structures in 3D media are widely used as optical filters in telecommunications and other applications [4].
The fabrication of such volume elements is done either by optical means (e. g., volume holography) or by lithographic deposition. In this paper, we present an approach to computationally design an OVE and fabricate it via additive manufacturing. Advancements in additive manufacturing based on two-photon polymerization (TPP) have made it practical to produce multilayered diffractive elements. TPP is a layer-by-layer technique and it offers great advantages in fabricating optically transparent arbitrary 3D micro/nanostructures [5].
There have been some studies to investigate the multilayer configuration of computer generated volume elements. Initially, computer generated 2-layer binary diffractive elements have been demonstrated experimentally with angular multiplexing for different diffraction patterns of letters [6]. Later, multilayer configurations have been shown for wavelength multiplexing of different letters [7], [8] and for spatial multiplexing of fiber modes [9]. Furthermore, a dynamic multilayer implementation utilizing a Spatial Light Modulator (SLM) has been shown for angular and wavelength multiplexing [10]. For the design process, an optimization scheme must be adopted. The above-mentioned studies used variants of generalized Gerchberg–Saxton algorithms [11]. Recently, algorithms that utilize the error backpropagation concept of deep neural networks have been used to calculate multi-layered diffractive elements to carry out different tasks including classification [12], [13]. The output of such algorithms is a stack of phase masks that should satisfy the thin element approach. These phase mask layers are physically implemented by thickness variations. Hence, there is a fundamental limit for the compactness of these designs, since the maximum thickness variation in each layer should be negligible when compared to the layer separation. Otherwise, there is a degradation in diffraction efficiency since the phase retardation that should be introduced by the mask and its representation by thickness variation start to differ significantly. We define this degradation as representation error, whose effect is shown in subsequent sections. To prevent representation error, the thickness variation of the layers should be taken into account during the optimization. In other words, the algorithm should optimize the thickness variation directly to realize a desired functionality of the 3D structure. Learning tomography (LT), which is a method to reconstruct 3D phase objects from experimental recordings of 2D projections of the 3D object [14], [15] is adapted by inverting its intended use to calculate the 3D index distribution of the OVE from examples of desired input-output pairs.
2 Methodology
A commercial Direct Laser Writing (DLW) system (Nanoscribe Photonic Professional GT) is used to fabricate a micro-scale optical element. This printer utilizes TPP, initiated by a laser beam highly focused into the volume of the photosensitive resin, yielding solidification. After the writing process of the desired structures inside the resin, the development step (washing out of the non-illuminated regions) follows, which yields the final polymerized material in the written 3D form. This binary nature of TPP constrains the design accordingly. In other words, the available refractive index values to achieve modulation are only
2.1 Optimization with LT
LT is a nonlinear optimization algorithm for optical diffraction tomography (ODT), which provides 3D RI distributions of transparent objects. The principle of ODT is to combine multiple 2D projections measured at different illumination angles to provide 3D distributions of RI [16]. In order to do so, the relationship between the 3D index distribution of a sample and the field scattered by the sample should be properly modeled. Unlike conventional reconstruction algorithms based on the first order approximations, LT is able to capture high orders of scattering by utilizing models such as the beam propagation method (BPM) [14] or split-step non-paraxial method (SSNP) [15]. The overall scheme of LT is to utilize one of such methods as the forward model in an iterative reconstruction scheme along with an intermediate step of regularization to impose our prior knowledge. This method has been referred to as Learning Tomography due to the multi-layer structure in the forward models, and depending on which model is used for the forward model, it can be further differentiated as LT-BPM or LT-SSNP. It has been shown [15] that LT-SSNP is the more accurate version of the mentioned methods. However, for small angles (i. e., multiplexing angles in OVE design and illumination angles for ODT) accuracy of LT-BPM and LT-SSNP are similar. In addition, LT-SSNP takes longer time for computation. Because of these reasons, we used LT-BPM in this work. In case of OVE design that utilizes high angles, LT-SSNP would be a better choice. This will be investigated in future work.
In this study, instead of experimental measurements obtained from ODT, LT is fed by the desired output field patterns. On the input side, these desired patterns have the plane wave illuminating fields at the corresponding incidence angles to establish angular multiplexing. Since 3D printing is a layer-by-layer technique, the computation volume is constrained to have RI change in a layer-by-layer fashion. The multilayer volume element and angular multiplexing schemes are demonstrated in the bottom part of Figure 1 for two input-two output case with three layers. For the desired fields, we arbitrarily decided to use linearly polarized (
Here, the most fundamental difference between our adapted use and typical use of LT is that for the volume element optimization, we are not looking for a unique solution. Any RI distribution that gives the desired output fields as intended is equally acceptable. We used the following cost function in LT to design OVE:
The above expression consists of only the data fidelity term that calculates the difference between the desired output
To design a multilayer OVE, the algorithm is initiated with layers of uniform thickness. Alternatively it can be initiated with any initial guess. Conversion of the optimized phase masks into varying thicknesses as initial guess can speed up the process if the phase masks are available. Each layer consists of many thin slices in BPM. The last slices of each layer consist of active voxels. They are active in the sense that the refractive indices of these voxels are allowed to change according to the error reduction scheme. When the refractive indices reach the upper or lower limits (i. e., the RI of the 3D printed polymer or the background medium), the active voxels are elongated forward or backward respectively. This approach is demonstrated in Figure 2. For simplicity, a representative sketch of one layer is shown. At the end of the iterations, we have a multilayer structure of varying thicknesses as shown previously in Figure 1. Note that optimization is not carried out in a binary manner since it prevents convergence of the algorithm.
The resolution of the simulation and the manufacturing are important parameters to consider. For an accurate simulation, the necessary resolution is higher than the chosen 3D printing method. With Nanoscribe, a pixel size of 2 µm by 2 µm is the standard for diffractive optical element applications where smaller pixel sizes suffer from edges, which start to dominate as we decrease the size further [18]. However, there is no constraint in axial direction except the resolution limit of the printer. For the forward model, we observed that a safe value for sampling interval is a quarter of the wavelength (λ/4), which implies that for a 2 µm by 2 µm lateral size in 3D printing, approximately 64 voxels are necessary in one axial slice of BPM (when λ = 1.03 µm). In our case, voxels of each slice are averaged in each iteration to take into account this lateral size limitation. In other words, the volume element is oversampled in the optimization process. The overall optimization scheme is summarized in Figure 3.
Another fundamental difference between the adapted version and the conventional version of LT is updating the RI. In conventional LT, the whole computation volume is updated whereas we update specific voxels (i. e., active voxels) to construct layer-by-layer volume elements. Another important difference to note is the synthetic generation of input-output pairs to establish the desired task. On the contrary, the output fields are experimentally recorded in tomography applications as depicted in Figure 1. Moreover, thickness limitations can be imposed on each layer if necessary.
2.2 Representation error
Representation error is the degradation of the fidelity in the reconstructed output fields after a multilayer OVE when the implemented layers do not satisfy the thin element approximation. To prevent representation error, topography (i. e., thickness variation) of the layers should be taken into account. BPM and SSNP are forward models that can account for the effects introduced by the topography of the layers since they represent these layers by having many sub-slices in the computation. We compare the performance obtained with and without LT in the propagation through the topographically encoded phase masks.
If we do not elongate the active voxels and allow the RI to vary continuously, then our approach reduces to a phase mask optimization approach. In each layer, the active voxels are kept on a plane (the last BPM slice of each layer) and RI (or equivalently the phase modulation) is updated in each step, which yields a phase mask at the end. If we consider the optimization steps in Figure 2, the phase mask optimization is equivalent to carry out the iterations as shown in Figure 2B and not carrying out the step shown in Figure 2C. A stack of phase masks can be converted to thickness variation simply by the following equation:
In Equation (2),
As seen from Figure 4, optimization with LT provides a better performance. Moreover, in Figure 5, one can compare the target field, the output obtained by phase mask stack, to the output obtained by thickness conversion element with
In Figure 6A, the optimized phase distributions of each layer of the phase mask stack are provided. Note that the phase mask stack yields the output fields in Figure 5B. In addition, the thickness conversion from these phase masks is provided in terms of the thickness map in Figure 6B, which gives the output fields in Figure 5C. Finally, we see the thickness map of volume element optimized by LT in Figure 6C, which gives the output fields in Figure 5D. For comparison, difference of thickness distributions of simply converted volume element and LT-optimized volume element are given in Figure 6D.
2.3 Full volume optimization
By changing writing parameters, it is possible to tune the RI to yield graded index (GRIN) materials with TPP [20]. However, the RI variation is quite low. Moreover, controllability and repeatability of the process are still challenging. If writing GRIN structures become more mature, then the true power of this optimization method can be further manifested for manufacturing of volume elements since the most significant advantage of this method is having access to all gradient values throughout the computation volume. By using GRIN writing, these volume elements become voxel-by-voxel encoded volume holograms, which should have higher diffraction efficiency in comparison with volume holograms that are recorded with optical means [21]. As a showcase, a volume of 64 μm by 64 μm by 120 μm is optimized to multiplex eight modes with RI variation of 0.15. Figure 7 depicts this GRIN element. The simulation results are given in Figure 8 in terms of amplitude of the output fields. Since our method optimizes with respect to electric field, we optimize both phase and amplitude. Section 3.1 of the Supplementary Material provides both phase and amplitude of reconstructed fields and desired fields. The inputs are plane waves with different incidence angles as indicated in Figure 8.
Even though the relation between the RI and the output fields is nonlinear, the relation between input and output fields is linear. A lossless linear optical system that is designed to produce a specified pattern for a given illumination field (e. g., angle of incidence of a plane wave) can also work in reverse producing the illumination field in response to the specified pattern. In other words, the OVE designed for angular multiplexing can also act as demultiplexer. Moreover, the OVE acts as a correlator. This correlation manifests itself as the strengths of the resulted plane waves with specific angles. With LT, it is possible to design an OVE that learns how to correlate variants of classes such as handwritten digits provided that it is trained with many examples (See Section 3.2 of the Supplementary Material).
3 Additive manufacturing of the 3-layer volume element and results
The 3-layer volume element for multiplexing two incident plane waves with +3° and −3° incidence angles to yield
The experiments for testing the multiplexing capabilities of the printed structure were performed by using an optical setup in which a spatial light modulator (SLM, Pluto-NIR2, Holoeye) is used to change the angle of incidence beam to the volume element. As the light source, Amplitude Laser – Satsuma generating pulses at 1030 nm is used. After the volume element, a 4f imaging system is used to record the output field. In the experiments, we observed an increase in the high-frequency noise compared to simulations. One source of error is 3D printing in terms of the shrinkage of the polymer after development, edge effects and slight RI deviation. The effect of shrinkage can be observed in Figure 9C and D. In 9C, we see that the sides of the given layer are curved, which implies that the structure shrunk. In addition, size of the structure is less than the design value of 128 μm. In Figure 9D, we can see that transverse dimensions of the voxels are slightly less than the design value of 2 μm for this sample. Another source of error is the optical setup in terms of the non-ideal illumination beam shape and slight misalignments. To prevent high-frequency noise, we put an iris in the Fourier plane of the imaging 4f system to block the high frequencies. The output intensity measurements corresponding both specified illumination angles are given in Figure 10.
We have tried oversampling for both 2 µm by 2 µm and 1 µm by 1 µm lateral dimensions. Having a smaller voxel size is advantageous since it increases degrees of freedom in a fixed volume and enables modulating higher frequencies. However, the disadvantage is degradation due to edge effects which is depicted in Figure 11 with high aspect ratio pillars (i. e., high
4 Conclusions
In this study, we have proposed a new optimization scheme to calculate OVEs and investigated angular multiplexing property. Practical constraints enforce a multilayer volume element with the binary RI. Our proposed method can adapt to these constraints and optimize an OVE directly in 3D. In other words, there is no intermediate step of optimizing the stack of phase masks and converting the phase values to corresponding thickness values for physical implementation. We showed that this approach improves the performance especially when the layer separation becomes comparable with the thickness variation of each layer. Moreover, our proposed method can reconstruct a continuous RI variation in the whole volume to yield a computer generated, voxel-by-voxel encoded volume hologram, which might show improved efficiency and storage capacity per volume. Additive manufacturing allows reaching every voxel individually in the printing process, which means that we can translate the degrees of freedom that we have computationally to physical implementation if the printed RI values are varied in a manageable manner. Further investigation for ways of manufacturing such an element is planned, using GRIN that shows more promising results.
For multilayer configuration, we used additive manufacturing, specifically TPP, to experimentally verify our method. We observed that the OVE perform angular multiplexing and reconstruct the desired output fields as intended. However, we see more high-frequency noise than what we have in our simulations, which can be due to additive manufacturing imperfections and optical setup imperfections as mentioned. In our optical setup, high-frequency noise is filtered in the Fourier plane so that satisfactory outputs are recorded at the expense of high-frequency information. Another factor is that our forward model (BPM) does not count for reflections and this can create a discrepancy. More accurate forward models are computationally too heavy as soon as the computation volume becomes larger than
Conflict of interest statement: The authors declare no conflicts of interest regarding this article.
References
[1] D. Gabor, “Associative holographic memories,” IBM J. Res. Dev., vol. 13, pp. 156–9, 1969, https://doi.org/10.1147/rd.132.0156.Search in Google Scholar
[2] E. N. Leith, A. Kozma, J. Upatnieks, J. Marks, and N. Massey, “Holographic data storage in three-dimensional media,” Appl. Opt., vol. 5, pp. 1303–11, 1966, https://doi.org/10.1364/ao.5.001303.Search in Google Scholar
[3] D. Psaltis, and F. Mok, “Holographic memories,” Sci. Am., vol. 273, pp. 70–6, 1995, https://doi.org/10.1038/scientificamerican1195-70.Search in Google Scholar
[4] G. J. Steckman, W. Liu, R. Platz, D. Schroeder, C. Moser, and F. Havermeyer, “Volume holographic grating wavelength stabilized laser diodes,” IEEE J. Sel. Top in Quantum Electron, vol. 13, no. 3, pp. 672–8, 2007, https://doi.org/10.1109/jstqe.2007.896060.Search in Google Scholar
[5] X. Zhou, Y. Hou, and J. Lin, “A review on the processing accuracy of two-photon polymerization,” AIP Advances, vol. 5, 2015, Art no. 030701.10.1063/1.4916886Search in Google Scholar
[6] S. Borgsmüller, S. Noehte, C. Dietrich, T. Kresse, and R. Männer, “Computer-generated stratified diffractive optical elements,” Appl. Opt., vol. 42, pp. 5274–83, 2003, https://doi.org/10.1364/ao.42.005274.Search in Google Scholar
[7] T. Kämpfe, E. B. Kley, A. Tünnermann, and P. Dannberg, “Design and fabrication of stacked, computer generated holograms for multicolor image generation,” Appl. Opt., vol. 46, pp. 5482–8, 2007, https://doi.org/10.1364/ao.46.005482.Search in Google Scholar
[8] T. Kämpfe, E. B. Kley, and A. Tünnermann, “Designing multiplane computer-generated holograms with consideration of the pixel shape and the illumination wave,” J. Opt. Soc. Am A., vol. 25, pp. 1609–22, 2008, https://doi.org/10.1364/josaa.25.001609.Search in Google Scholar
[9] G. Labroille, B. Denolle, P. Jian, P. Genevaux, N. Treps, and J. F. Morizur, “Efficient and mode selective spatial mode multiplexer based on multi-plane light conversion,” Opt. Express, vol. 22, pp. 15599–607, 2014, https://doi.org/10.1364/oe.22.015599.Search in Google Scholar
[10] H. Wang, and R. Piestun, “Dynamic 2D implementation of 3D diffractive optics,” Optica, vol. 5, pp. 1220–8, 2018, https://doi.org/10.1364/optica.5.001220.Search in Google Scholar
[11] R. Gerchberg, and W. Saxton, “A practical algorithm for the determination of the phase from image and diffraction plane pictures,” Opt. Jena, vol. 35, pp. 237–46, 1972.Search in Google Scholar
[12] X. Lin, Y. Rivenson, N. T. Yardimci, et al., “All-optical machine learning using diffractive deep neural networks,” Science, vol. 361, pp. 1004–8, 2018, https://doi.org/10.1126/science.aat8084.Search in Google Scholar
[13] Y. Luo, D. Mengu, N. T. Yardimci, et al., “Design of task-specific optical systems using broadband diffractive neural networks,” Light Sci. Appl., vol. 8, pp. 112, 2019, https://doi.org/10.1038/s41377-019-0223-1.Search in Google Scholar
[14] U. S. Kamilov, I. N. Papadopoulos, M. H. Shoreh, et al., “Learning approach to optical tomography,” Optica, vol. 2, pp. 517–22, 2015, https://doi.org/10.1364/optica.2.000517.Search in Google Scholar
[15] J. Lim, A. B. Ayoub, E. E. Antoine, and D. Psaltis, “High-fidelity optical diffraction tomography of multiple scattering samples,” Light Sci. Appl., vol. 8, pp. 82, 2019, https://doi.org/10.1038/s41377-019-0195-1.Search in Google Scholar
[16] E. Wolf, “Three-dimensional structure determination of semi-transparent objects from holographic data,” Opt. Commun., vol. 1, no. 4, pp. 153–6, 1969, https://doi.org/10.1016/0030-4018(69)90052-2.Search in Google Scholar
[17] A. Beck, and M. Teboulle, “Fast gradient-based algorithms for constrained total variation image denoising and deblurring prolems,” IEEE Trans. Image Process, vol. 18, pp. 2419–34, 2009, https://doi.org/10.1109/tip.2009.2028250.Search in Google Scholar
[18] Diffractive Optical Elements (DOE) and Image Printing, Nanoscribe GmbH, 2020 (Accessed January 26, 2020 at: https://support.nanoscribe.com/hc/en-gb/articles/360000530993).Search in Google Scholar
[19] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process, vol. 13, pp. 600–12, 2004, https://doi.org/10.1109/tip.2003.819861.Search in Google Scholar
[20] A. Zukauskas, I. Matulaitiene, D. Paipulas, G. Niaura, M. Malinauskas, and R. Gadonas, “Tuning the refractive index in 3D direct laser writing lithography: Towards GRIN microoptics,” Laser Photon Rev., vol. 9, pp. 706–12, 2015, https://doi.org/10.1002/lpor.201500170.Search in Google Scholar
[21] G. Barbastathis, and D. Psaltis, “Volume holographic multiplexing methods,” in Holographic Data Storage, H. J. Coufal, D. Psaltis, and G. T. Sincerbox, Eds., Springer Berlin Heidelberg: Springer, 2000, pp. 21–62.10.1007/978-3-540-47864-5_2Search in Google Scholar
Supplementary material
The online version of this article offers supplementary material (https://doi.org/10.1515/nanoph-2020-0196).
© 2020 Niyazi Ulas Dinc et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.