This paper proposes a novel framework to evaluate fluid simulation methods based on crowd-sourced user studies in order to robustly gather large numbers of opinions. The key idea for a robust and reliable evaluation is to use a reference video from a carefully selected real-world setup in the user study. By conducting a series of controlled user studies and comparing their evaluation results, we observe various factors that affect the perceptual evaluation. Our data show that the availability of a reference video makes the evaluation consistent. We introduce this approach for computing scores of simulation methods as visual accuracy metric. As an application of the proposed framework, a variety of popular simulation methods are evaluated.

中文翻译：

---

液体模拟方法的感官评估

本文提出了一个新的框架，用于基于众包用户研究评估流体模拟方法，以稳健地收集大量意见。进行可靠而可靠的评估的关键思想是在用户研究中使用经过精心选择的实际设置中的参考视频。通过进行一系列受控用户研究并比较他们的评估结果，我们观察到影响感知评估的各种因素。我们的数据表明，参考视频的可用性使评估保持一致。我们引入这种方法来计算模拟方法的分数作为视觉准确性指标。作为所提出框架的应用，评估了各种流行的仿真方法。