Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios with human-robot interaction, such as autonomous driving. In turn, there has been a significant amount of interest and research in trajectory forecasting, resulting in a wide variety of approaches. Common to all works, however, is the use of the same few accuracy-based evaluation metrics, e.g., displacement error and log-likelihood. While these metrics are informative, they are task-agnostic and predictions that are evaluated as equal can lead to vastly different outcomes, e.g., in downstream planning and decision making. In this work, we take a step back and critically evaluate current trajectory forecasting metrics, proposing task-aware metrics as a better measure of performance in systems where prediction is being deployed. We additionally present one example of such a metric, incorporating planning-awareness within existing trajectory forecasting metrics.

中文翻译：

---

重新思考轨迹预测评估

预测其他代理的行为是现代机器人自主堆栈的一个组成部分，尤其是在人机交互的安全关键场景中，例如自动驾驶。反过来，人们对轨迹预测产生了大量的兴趣和研究，从而产生了各种各样的方法。然而，所有作品的共同点是使用相同的少数基于准确性的评估指标，例如位移误差和对数似然。虽然这些指标信息丰富，但它们与任务无关，并且被评估为相同的预测可能导致截然不同的结果，例如，在下游规划和决策制定方面。在这项工作中，我们退后一步，批判性地评估当前的轨迹预测指标，在部署预测的系统中，提出任务感知指标作为更好的性能衡量标准。我们还提供了此类度量的一个示例，将规划意识纳入现有轨迹预测度量中。