Huge advancements have been made over the years in terms of modern image-sensing hardware and visual computing algorithms (e.g., computer vision, image processing, and computational photography). However, to this day, there still exists a current gap between the hardware and software design in an imaging system, which silos one research domain from another. Bridging this gap is the key to unlocking new visual computing capabilities for end applications in commercial photography, industrial inspection, and robotics. In this survey, we explore existing works in the literature that can be leveraged to replace conventional hardware components in an imaging system with software for enhanced reconfigurability. As a result, the user can program the image sensor in a way best suited to the end application. We refer to this as software-defined imaging (SDI), where image sensor behavior can be altered by the system software depending on the user’s needs. The scope of our survey covers imaging systems for single-image capture, multi-image, and burst photography, as well as video. We review works related to the sensor primitives, image signal processor (ISP) pipeline, computer architecture, and operating system elements of the SDI stack. Finally, we outline the infrastructure and resources for SDI systems, and we also discuss possible future research directions for the field.

中文翻译：

---

软件定义成像：调查

多年来，现代图像传感硬件和视觉计算算法（例如，计算机视觉、图像处理和计算摄影）取得了巨大进步。然而，时至今日，成像系统中的硬件和软件设计之间仍然存在差距，这使得一个研究领域与另一个研究领域形成孤岛。弥合这一差距是为商业摄影、工业检测和机器人技术的终端应用解锁新的视觉计算能力的关键。在本次调查中，我们探索了文献中的现有作品，这些作品可以用来用软件替换成像系统中的传统硬件组件，以增强可重构性。因此，用户可以以最适合最终应用的方式对图像传感器进行编程。我们将此称为软件定义成像 (SDI)，系统软件可以根据用户需求更改图像传感器的行为。我们的调查范围涵盖用于单图像捕捉、多图像和连拍摄影以及视频的成像系统。我们回顾了与传感器基元、图像信号处理器 (ISP) 管道、计算机体系结构和 SDI 堆栈的操作系统元素相关的工作。最后，我们概述了 SDI 系统的基础设施和资源，我们还讨论了该领域未来可能的研究方向。图像信号处理器 (ISP) 流水线、计算机体系结构和 SDI 堆栈的操作系统元素。最后，我们概述了 SDI 系统的基础设施和资源，我们还讨论了该领域未来可能的研究方向。图像信号处理器 (ISP) 流水线、计算机体系结构和 SDI 堆栈的操作系统元素。最后，我们概述了 SDI 系统的基础设施和资源，我们还讨论了该领域未来可能的研究方向。