Edge video analytics is becoming the solution to many safety and management tasks. Its wide deployment, however, must first address the tension between inference accuracy and resource (compute/network) cost. This has led to the development of video analytics pipelines (VAPs), which reduce resource cost by combining DNN compression/speedup techniques with video processing heuristics. Our measurement study on existing VAPs, however, shows that today's methods for evaluating VAPs are incomplete, often producing premature conclusions or ambiguous results. This is because each VAP's performance varies substantially across videos and time (even under the same scenario) and is sensitive to different subsets of video content characteristics. We argue that accurate VAP evaluation must first characterize the complex interaction between VAPs and video characteristics, which we refer to as VAP performance clarity. We design and implement Yoda, the first VAP benchmark to achieve performance clarity. Using primitive-based profiling and a carefully curated benchmark video set, Yoda builds a performance clarity profile for each VAP to precisely define its accuracy/cost tradeoff and its relationship with video characteristics. We show that Yoda substantially improves VAP evaluations by (1) providing a comprehensive, transparent assessment of VAP performance and its dependencies on video characteristics; (2) explicitly identifying fine-grained VAP behaviors that were previously hidden by large performance variance; and (3) revealing strengths/weaknesses among different VAPs and new design opportunities.

中文翻译：

---

提升边缘视频分析的性能

边缘视频分析正在成为许多安全和管理任务的解决方案。但是，其广泛的部署必须首先解决推理准确性和资源（计算/网络）成本之间的矛盾。这导致了视频分析管道（VAP）的发展，该技术通过将DNN压缩/加速技术与视频处理启发式技术相结合来降低资源成本。但是，我们对现有VAP的测量研究表明，当今评估VAP的方法并不完整，常常会得出不成熟的结论或含糊不清的结果。这是因为每个VAP的性能会随视频和时间的不同而有很大差异（即使在相同的情况下），并且对视频内容特征的不同子集敏感。我们认为，准确的VAP评估必须首先表征VAP与视频特征之间的复杂交互，这就是我们所说的VAP性能清晰度。我们设计并实现了Yoda，这是第一个VAP基准测试，旨在实现性能的清晰度。Yoda使用基于原始的性能分析和精心策划的基准视频集，为每个VAP构建了性能清晰度配置文件，以精确定义其准确性/成本权衡以及与视频特性的关系。我们证明，尤达（1）通过提供对VAP性能及其对视频特性的依赖性的全面，透明的评估，大大提高了VAP评估。（2）明确识别以前由于较大的性能差异而隐藏的细粒度VAP行为；