Deep learning-based super-resolution (SR) has gained tremendous popularity in recent years because of its high image quality performance and wide application scenarios. However, prior methods typically suffer from large amounts of computations and huge power consumption, causing difficulties for real-time inference, especially on resource-limited platforms such as mobile devices. To mitigate this, we propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. The inference speed is directly taken into the optimization along with the SR loss to derive SR models with high image quality while satisfying the real-time inference requirement. Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence. With the proposed framework, we achieve real-time SR inference for implementing 720p resolution with competitive SR performance (in terms of PSNR and SSIM) on GPU/DSP of mobile platforms (Samsung Galaxy S21).

中文翻译：

---

用于移动实时超分辨率的编译器感知神经架构搜索

基于深度学习的超分辨率（SR）由于其高图像质量性能和广泛的应用场景，近年来获得了极大的普及。然而，现有方法通常会受到大量计算和巨大功耗的影响，导致实时推理困难，尤其是在移动设备等资源有限的平台上。为了缓解这种情况，我们提出了一个编译器感知的 SR 神经架构搜索 (NAS) 框架，该框架使用自适应 SR 块进行深度搜索和每层宽度搜索。在满足实时推理要求的同时，将推理速度与 SR 损失一起直接纳入优化，得到具有高图像质量的 SR 模型。而不是在搜索过程中每次迭代测量移动设备上的速度，利用与编译器优化相结合的速度模型来预测具有各种宽度配置的 SR 块的推理延迟，以实现更快的收敛。通过所提出的框架，我们实现了实时 SR 推理，以在移动平台（三星 Galaxy S21）的 GPU/DSP 上实现具有竞争 SR 性能（在 PSNR 和 SSIM 方面）的 720p 分辨率。