Authors

* External authors

Venue

Date

Share

Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

Zifan Xu*

Bo Liu*

Xuesu Xiao*

Anirudh Nair*

Peter Stone

* External authors

ICRA 2023

2023

Abstract

Deep reinforcement learning (RL) has broughtmany successes for autonomous robot navigation. However,there still exists important limitations that prevent real-worlduse of RL-based navigation systems. For example, most learningapproaches lack safety guarantees; and learned navigationsystems may not generalize well to unseen environments.Despite a variety of recent learning techniques to tackle thesechallenges in general, a lack of an open-source benchmarkand reproducible learning methods specifically for autonomousnavigation makes it difficult for roboticists to choose whatlearning methods to use for their mobile robots and for learningresearchers to identify current shortcomings of general learningmethods for autonomous navigation. In this paper, we identifyfour major desiderata of applying deep RL approaches forautonomous navigation: (D1) reasoning under uncertainty, (D2)safety, (D3) learning from limited trial-and-error data, and (D4)generalization to diverse and novel environments. Then, weexplore four major classes of learning techniques with thepurpose of achieving one or more of the four desiderata:memory-based neural network architectures (D1), safe RL (D2),model-based RL (D2, D3), and domain randomization (D4). Bydeploying these learning techniques in a new open-source large-scale navigation benchmark and real-world environments, weperform a comprehensive study aimed at establishing to whatextent can these techniques achieve these desiderata for RL-based navigation systems

Related Publications

Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning

AAAI, 2024
Zizhao Wang*, Caroline Wang*, Xuesu Xiao*, Yuke Zhu*, Peter Stone

Two desiderata of reinforcement learning (RL) algorithms are the ability to learn from relatively little experience and the ability to learn policies that generalize to a range of problem specifications. In factored state spaces, one approach towards achieving both goals is …

Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents

AAAI, 2024
Arrasy Rahman*, Jiaxun Cui*, Peter Stone

Robustly cooperating with unseen agents and human partners presents significant challenges due to the diverse cooperative conventions these partners may adopt. Existing Ad Hoc Teamwork (AHT) methods address this challenge by training an agent with a population of diverse tea…

Learning Optimal Advantage from Preferences and Mistaking it for Reward

AAAI, 2024
W. Bradley Knox*, Stephane Hatgis-Kessell*, Sigurdur Orn Adalgeirsson*, Serena Booth*, Anca Dragan*, Peter Stone, Scott Niekum*

We consider algorithms for learning reward functions from human preferences over pairs of trajectory segments---as used in reinforcement learning from human feedback (RLHF)---including those used to fine tune ChatGPT and other contemporary language models. Most recent work o…

  • HOME
  • Publications
  • Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

JOIN US

Shape the Future of AI with Sony AI

We want to hear from those of you who have a strong desire
to shape the future of AI.