• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms.
Anthony Francis, Claudia Perez-D'Arpino,
Chengshu Li, Fei Xia, Alexandre Alahi, Rachid Alami1, Aniket Bera, Abhijat Biswas,
Joydeep Biswas, Rohan Chandra, Hao-Tien Lewis Chiang, Michael Everett, Sehoon Ha,
Justin Hart, Jonathan P. How, Haresh Karnan,
Tsang-Wei Edward Lee, Luis J. Manso, Reuth Mirksy, Soren Pirk,
Phani Teja Singamaneni, Peter Stone, Ada
V. Taylor, Peter Trautman, Nathan Tsoi, Marynel Vazquez, Xuesu Xiao, Peng
Xu, Naoki Yokoyama, Alexander Toshev, and and Roberto Martin-Martin.
ACM Transactions on Human-Robot Interaction (THRI),
14(2), February 2025.
Official version on publisher's website
A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation. While the field of social navigation has advanced tremendously in recent years, the fair evaluation of algorithms that tackle social navigation remains hard because it involves not just robotic agents moving in static environments but also dynamic human agents and their perceptions of the appropriateness of robot behavior. In contrast, clear, repeatable, and accessible benchmarks have accelerated progress in fields like computer vision, natural language processing and traditional robot navigation by enabling researchers to fairly compare algorithms, revealing limitations of existing solutions and illuminating promising new directions. We believe the same approach can benefit social navigation. In this article, we pave the road toward common, widely accessible, and repeatable benchmarking criteria to evaluate social robot navigation. Our contributions include (a) a definition of a socially navigating robot as one that respects the principles of safety, comfort, legibility, politeness, social competency, agent understanding, proactivity, and responsiveness to context, (b) guidelines for the use of metrics, development of scenarios, benchmarks, datasets, and simulators to evaluate social navigation, and (c) a design of a social navigation metrics framework to make it easier to compare results from different simulators, robots, and datasets.
@Article{25thri,
author="Anthony Francis and Claudia Perez-D'Arpino and Chengshu Li and Fei Xia and Alexandre Alahi and Rachid Alami1 and Aniket Bera and Abhijat Biswas and Joydeep Biswas and Rohan Chandra and Hao-Tien Lewis Chiang and Michael Everett and Sehoon Ha and Justin Hart and Jonathan P. How and Haresh Karnan and Tsang-Wei Edward Lee and Luis J. Manso and Reuth Mirksy and Soren Pirk and Phani Teja Singamaneni and Peter Stone and Ada V. Taylor and Peter Trautman and Nathan Tsoi and Marynel Vazquez and Xuesu Xiao and Peng Xu and Naoki Yokoyama and Alexander Toshev and and Roberto Martin-Martin",
title="Principles and Guidelines for Evaluating Social Robot Navigation Algorithms",
journal="ACM Transactions on Human-Robot Interaction (THRI)",
volume="14",
number="2",
article="34",
month="February",
year="2025",
abstract={
A major challenge to deploying robots widely is navigation
in human-populated environments, commonly referred to as
social robot navigation. While the field of social
navigation has advanced tremendously in recent years, the
fair evaluation of algorithms that tackle social
navigation remains hard because it involves not just
robotic agents moving in static environments but also
dynamic human agents and their perceptions of the
appropriateness of robot behavior. In contrast, clear,
repeatable, and accessible benchmarks have accelerated
progress in fields like computer vision, natural language
processing and traditional robot navigation by enabling
researchers to fairly compare algorithms, revealing
limitations of existing solutions and illuminating
promising new directions. We believe the same approach can
benefit social navigation. In this article, we pave the
road toward common, widely accessible, and repeatable
benchmarking criteria to evaluate social robot
navigation. Our contributions include (a) a definition of
a socially navigating robot as one that respects the
principles of safety, comfort, legibility, politeness,
social competency, agent understanding, proactivity, and
responsiveness to context, (b) guidelines for the use of
metrics, development of scenarios, benchmarks, datasets,
and simulators to evaluate social navigation, and (c) a
design of a social navigation metrics framework to make it
easier to compare results from different simulators,
robots, and datasets.},
wwwnote={<a href="https://dl.acm.org/doi/10.1145/370059">Official version</a> on publisher's website},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Mar 12, 2025 00:14:46