Benchmarking is a common practice employed to quantify the performance of various approaches toward the same task. By maintaining a consistent test environment, the inherent behaviours between methods may be distinguished—which is key to progressing a research field. In robotic manipulation research there is a current lack of standardisation, making it challenging to fairly assess and compare various approaches throughout literature. This paper proposes new criteria in conjunction with a benchmarking platform to measure the effectiveness of a grasping pipeline. The proposed benchmarking template offers a testing platform for 2-fingered, vision-based grasp synthesis methodologies. A prototype system was constructed. The prototype was shown to serve as a suitable benchmarking platform for the deployment of various grasp synthesis methodologies. 4000 trials were conducted to evaluate the differing approaches. Results showed that the proposed metrics provide useful insights into the quality of grasp poses produced by a grasp synthesis methodology. Moreover, such metrics provide more comprehensive insights into grasp outcome than traditional methods used to quantify performance of a methodology and present a fair baseline for comparison between different approaches.

基准测试是一种常见的做法，用于量化针对同一任务的各种方法的性能。通过保持一致的测试环境，可以区分方法之间的固有行为——这是推进研究领域的关键。在机器人操作研究中，目前缺乏标准化，这使得在整个文献中公平地评估和比较各种方法具有挑战性。本文结合基准测试平台提出了新标准，以衡量抓取管道的有效性。拟议的基准测试模板为基于视觉的两指抓取综合方法提供了一个测试平台。构建了原型系统。该原型被证明可以作为一个合适的基准平台，用于部署各种抓取综合方法。进行了 4000 次试验来评估不同的方法。结果表明，所提出的指标提供了对抓握综合方法产生的抓握姿势质量的有用见解。此外，与用于量化方法性能的传统方法相比，此类指标提供了对抓取结果的更全面的见解，并为不同方法之间的比较提供了公平的基线。