Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Reconfigurable heterogeneous integration using stackable chips with embedded artificial intelligence

Nature Electronics volume 5, pages 386–393 (2022)Cite this article

Subjects

Abstract

Artificial intelligence applications have changed the landscape of computer design, driving a search for hardware architecture that can efficiently process large amounts of data. Three-dimensional heterogeneous integration with advanced packaging technologies could be used to improve data bandwidth among sensors, memory and processors. However, such systems are limited by a lack of hardware reconfigurability and the use of conventional von Neumann architectures. Here we report stackable hetero-integrated chips that use optoelectronic device arrays for chip-to-chip communication and neuromorphic cores based on memristor crossbar arrays for highly parallel data processing. With this approach, we create a system with stackable and replaceable chips that can directly classify information from a light-based image source. We also modify this system by inserting a preprogrammed neuromorphic denoising layer that improves the classification performance in a noisy environment. Our reconfigurable three-dimensional hetero-integrated technology can be used to vertically stack a diverse range of functional layers and could provide energy-efficient sensor computing systems for edge computing applications.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Integration technologies of sensor computing systems for edge computing.
Fig. 2: Components of stackable hetero-integrated neuromorphic chips.
Fig. 3: Replaceable and stackable hetero-integrated neuromorphic chips and their robust kernel operations.
Fig. 4: Application of stackable neuromorphic chips: insertion of a denoising functional layer in a noisy environment.

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding authors upon reasonable request.

References

  1. Zhang, W. et al. Neuro-inspired computing chips. Nat. Electron. 3, 371–382 (2020).

    Article  Google Scholar 

  2. Lin, P. et al. Three-dimensional memristor circuits as complex neural networks. Nat. Electron. 3, 225–232 (2020).

    Article  Google Scholar 

  3. Chen, W.-H. et al. CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors. Nat. Electron. 2, 420–428 (2019).

  4. Hills, G. et al. Modern microprocessor built from complementary carbon nanotube transistors. Nature 572, 595–602 (2019).

    Article  Google Scholar 

  5. Bishop, M. D. et al. Fabrication of carbon nanotube field-effect transistors in commercial silicon manufacturing facilities. Nat. Electron. 3, 492–501 (2020).

    Article  Google Scholar 

  6. Mukhopadhyay, S. et al. Heterogeneous integration for artificial intelligence: challenges and opportunities. IBM J. Res. Dev. 63, 4:1 (2019).

  7. Kum, H. S. et al. Heterogeneous integration of single-crystalline complex-oxide membranes. Nature 578, 75–81 (2020).

    Article  Google Scholar 

  8. Ohara, Y. et al. Chip-based hetero-integration technology for high-performance 3D stacked image sensor. In 2012 2nd IEEE CPMT Symposium Japan 1–4 (IEEE, 2012).

  9. Amir, M. F., Ko, J. H., Na, T., Kim, D. & Mukhopadhyay, S. 3-D stacked image sensor with deep neural network computation. IEEE Sens. J. 18, 4187–4199 (2018).

    Article  Google Scholar 

  10. Sabry Aly, M. M. et al. The N3XT approach to energy-efficient abundant-data computing. Proc. IEEE 107, 19–48 (2019).

    Article  Google Scholar 

  11. Bhansali, S. et al. 3D heterogeneous sensor system on a chip for defense and security applications. In Proc. SPIE 5417, Unattended/Unmanned Ground, Ocean, and Air Sensor Technologies and Applications VI 5417, 413 (2004).

    Google Scholar 

  12. Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).

    Article  Google Scholar 

  13. Ng, K. W. et al. Unconventional growth mechanism for monolithic integration of III-V on silicon. ACS Nano 7, 100–107 (2013).

    Article  Google Scholar 

  14. Koma, A. Van der Waals epitaxy for highly lattice-mismatched systems. J. Cryst. Growth 201, 236–241 (1999).

    Article  Google Scholar 

  15. Liau, Z. L. & Mull, D. E. Wafer fusion: a novel technique for optoelectronic device fabrication and monolithic integration. Appl. Phys. Lett. 56, 737–739 (1990).

    Article  Google Scholar 

  16. Benwadih, M., Coppard, R., Bonrad, K., Klyszcz, A. and Vuillaume, D. High mobility flexible amorphous IGZO thin-film transistors with a low thermal budget ultra-violet pulsed light process. ACS Appl. Mater. Interfaces 8, 34513–34519 (2016).

  17. Vinet, M. et al. 3D monolithic integration: technological challenges and electrical results. Microelectron. Eng. 88, 331–335 (2011).

    Article  Google Scholar 

  18. Bao, S. et al. A review of silicon-based wafer bonding processes, an approach to realize the monolithic integration of Si-CMOS and III–V-on-Si wafers. J. Semicond. 42, 023106 (2021).

  19. Lee, S. M. et al. High performance ultrathin GaAs solar cells enabled with heterogeneously integrated dielectric periodic nanostructures. ACS Nano 9, 10356–10365 (2015).

    Article  Google Scholar 

  20. Shulaker, M. M. et al. Carbon nanotube computer. Nature 501, 526–530 (2013).

    Article  Google Scholar 

  21. Le Gallo, M. et al. Mixed-precision in-memory computing. Nat. Electron. 1, 246–253 (2018).

    Article  Google Scholar 

  22. Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).

    Article  Google Scholar 

  23. Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).

    Article  Google Scholar 

  24. Wang, S. et al. Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci. Rev. 8, nwaa172 (2021).

    Article  Google Scholar 

  25. Wang, C. et al. Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array. Nat. Nanotechnol. 16, 1079–1085 (2021).

    Article  Google Scholar 

  26. Shulaker, M. M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74–78 (2017).

    Article  Google Scholar 

  27. Choi, M. H., Koh, H. J., Yoon, E. S., Shin, K. C. and Song, K. C. Self-aligning silicon groove technology platform for the low cost optical module. In Proc. 49th Electronic Components and Technology Conference (Cat. No. 99CH36299) 1140–1144 (IEEE, 1999).

  28. Barwicz, T. et al. Integrated metamaterial interfaces for self-aligned fiber-to-chip coupling in volume manufacturing. IEEE J. Sel. Topics Quantum Electron. 25, 1–13 (2018).

    Article  Google Scholar 

  29. Yeon, H. et al. Alloying conducting channels for reliable neuromorphic computing. Nat. Nanotechnol. 15, 574–579 (2020).

    Article  Google Scholar 

  30. Ferrari, G., Gozzini, F., Molari, A. & Sampietro, M. Transimpedance amplifier for high sensitivity current measurements on nanodevices. IEEE J. Solid-State Circuits 44, 1609–1616 (2009).

    Article  Google Scholar 

  31. Gurun, G., Hasler, P. & Degertekin, F. L. Front-end receiver electronics for high-frequency monolithic CMUT-on-CMOS imaging arrays. IEEE Trans. Ultrason., Ferroelectr., Freq. Control 58, 1658–1668 (2011).

    Article  Google Scholar 

  32. Rosenstein, J. K., Wanunu, M., Merchant, C. A., Drndic, M. & Shepard, K. L. Integrated nanopore sensing platform with sub-microsecond temporal resolution. Nat. Methods 9, 487–492 (2012).

    Article  Google Scholar 

  33. Dodge, S. and Karam, L. A study and comparison of human and deep learning recognition performance under visual distortions. In 2017 26th International Conference on Computer Communication and Networks (ICCCN) 1–7 (IEEE, 2017).

  34. Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D. and Barron, J. T. Unprocessing images for learned raw denoising. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 11036–11045 (IEEE, 2019).

  35. Charte, D., Charte, F., García, S., del Jesus, M. J. & Herrera, F. A practical tutorial on autoencoders for nonlinear feature fusion: taxonomy, models, software and guidelines. Inf. Fusion 44, 78–96 (2018).

    Article  Google Scholar 

  36. Xu, L. et al. Extracting and composing robust features with denoising autoencoders. In Proc. 25th International Conference on Machine Learning 1096–1103 (IEEE, 2008).

  37. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).

    Article  Google Scholar 

  38. Wang, C.-Y. et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci. Adv. 6, eaba6173 (2020).

    Article  Google Scholar 

  39. Pan, C. et al. Reconfigurable logic and neuromorphic circuits based on electrically tunable two-dimensional homojunctions. Nat. Electron. 3, 383–390 (2020).

    Article  Google Scholar 

  40. Zhou, T. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photon. 15, 367–373 (2021).

    Article  Google Scholar 

Download references

Acknowledgements

J.-H.K. acknowledges financial support from the Ministry of Trade, Industry and Energy (MOTIE), South Korea, under the Fostering Global Talents for Innovative Growth Program (P0008749) by the Korea Institute for Advancement of Technology (KIAT). The team at MIT acknowledge financial support from the Korea Institute of Science and Technology (KIST) through 2E31550 and the Samsung Global Research Outreach (GRO) Program.

Author information

Author notes

  1. These authors contributed equally: Chanyeol Choi, Hyunseok Kim, Ji-Hoon Kang, Min-Kyu Song.

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA

    Chanyeol Choi

  2. Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA, USA

    Chanyeol Choi, Hyunseok Kim, Ji-Hoon Kang, Min-Kyu Song, Hanwool Yeon, Celesta S. Chang, Jun Min Suh, Jiho Shin, Kuangye Lu, Bo-In Park, Yeongin Kim, Han Eol Lee, Doyoon Lee, Sang-Hoon Bae, Hyun S. Kum, Peng Lin & Jeehwan Kim

  3. Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA

    Hyunseok Kim, Ji-Hoon Kang, Min-Kyu Song, Hanwool Yeon, Celesta S. Chang, Jun Min Suh, Jiho Shin, Kuangye Lu, Bo-In Park, Yeongin Kim, Han Eol Lee, Doyoon Lee, Subeen Pang, Sang-Hoon Bae, Hyun S. Kum, Peng Lin & Jeehwan Kim

  4. School of Materials Science and Engineering, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea

    Hanwool Yeon

  5. Department of Electrical Engineering and Computer Science, University of Cincinnati, Cincinnati, OH, USA

    Yeongin Kim

  6. Division of Advanced Materials Engineering, Jeonbuk National University, Jeonju, Republic of Korea

    Han Eol Lee

  7. School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA

    Jaeyong Lee

  8. Department of Radiology, Massachusetts General Hospital/Harvard Medical School, Boston, MA, USA

    Ikbeom Jang

  9. Department of Radiology, Stanford University, Stanford, CA, USA

    Kanghyun Ryu

  10. Department of Mechanical Engineering and Materials Science, Washington University in St. Louis, St. Louis, MO, USA

    Sang-Hoon Bae

  11. Materials Science Division, Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Yifan Nie

  12. Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Republic of Korea

    Hyun S. Kum

  13. Post-silicon Semiconductor Institute, Korea Institute of Science and Technology, Seoul, Republic of Korea

    Min-Chul Park, Suyoun Lee & Hyung-Jun Kim

  14. Institute of Microelectronics, Tsinghua University, Beijing, China

    Huaqiang Wu

  15. College of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Peng Lin

  16. Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA

    Jeehwan Kim

Authors
  1. Chanyeol Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyunseok Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji-Hoon Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min-Kyu Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hanwool Yeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Celesta S. Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jun Min Suh
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jiho Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kuangye Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Bo-In Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Yeongin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Han Eol Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Doyoon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jaeyong Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Ikbeom Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Subeen Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Kanghyun Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Sang-Hoon Bae
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Yifan Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Hyun S. Kum
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Min-Chul Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Suyoun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Hyung-Jun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Huaqiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Peng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Jeehwan Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.C. and P.L. conceived this work and designed the experiments. J.K. directed the team. C.C., H.K., P.L., M.-K.S, H.Y. and J.K. prepared the manuscript. C.C., M.-K.S., H.Y., J.M.S., J.L. and D.L. performed the device fabrication. H.K. grew the thin films for optoelectronic devices. C.C., J.-H.K. and P.L. conducted the crossbar array measurement. C.C. designed and built the optical setup. C.C. and H.Y. performed the sensor array measurement. C.C. conducted the neural network simulations and implementations using MATLAB, Python and PyTorch. C.C., S.P. and K.R. discussed the results of the neural networks simulations. All the authors contributed to the discussion and analysis of the results.

Corresponding authors

Correspondence to Huaqiang Wu, Peng Lin or Jeehwan Kim.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Electronics thanks Rui Yang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–12.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, C., Kim, H., Kang, JH. et al. Reconfigurable heterogeneous integration using stackable chips with embedded artificial intelligence. Nat Electron 5, 386–393 (2022). https://doi.org/10.1038/s41928-022-00778-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41928-022-00778-y

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing