RxR-Habitat Competition

The RxR-Habitat competition is a challenge that requires Vision-and-Language Navigation (VLN) agents to follow RxR navigation instructions in Matterport3D environments using the Habitat simulator. Compared to the standard RxR competition, in which paths are defined in a navigation graph, the continuous environments in Habitat emphasize the need for robust control using low-level actions, as illustrated in the demo video. Results are reported on the RxR Test-Challenge split. For scoring, participants must upload the paths generated by their agents in the Habitat simulator. As with the standard RxR competition, navigation instructions are in English, Hindi and Telugu.

For fair comparison, all entries to the CVPR 2023 Challenge must use a standardized configuration for the VLN agent's observation and action space. This configuration, which is the same as the last two years, is based on a 30° turn angle, 30° look up / look down action, 0.25m step size and 480x640 RGBD vision. Precise details of the configuration are provided with the starter code (see below). CVPR Challenge entrants will need to certify (via a checkbox) that their agent uses this configuration when submitting to the leaderboard. Any submission received between 20 March and 15 May 2023 using the standardized configuration will be considered an entry to the CVPR 2023 Challenge. Submissions that do not use the standardized configuration (e.g., using panoramic vision, or larger step sizes) are welcome at any time but will not be part of the CVPR 2023 Challenge. These submissions should answer 'No' to the question asking if the agent was trained using the standardized challenge config.

We provide PyTorch starter code for an RxR-Habitat agent on github. In the starter code, all Habitat simulator settings (i.e., agent height and width, sensor configuration and action space configuration) are set to the standardized configuration. Paths in the RxR dataset were ported to the Habitat simulator using the VLN-CE codebase. Note that some paths could not be ported to Habitat so the RxR-Habitat dataset is slightly smaller than the original RxR dataset.

• 17 March RxR-Habitat competition launch
• 20 March Leaderboard submissions open
• 15 May Leaderboard submission deadline at 23:59:59 UTC
• 19 June Winners announced at the CVPR 2023 Embodied AI Workshop

The organizers of the RxR-Habitat competition are:

• Jacob Krantz, Oregon State University
• Alex Ku, Google Research
• Oleksandr Maksymets, Facebook AI Research
• Stefan Lee, Oregon State University
• Peter Anderson, Google Research

The RxR-Habitat competition was first proposed as part of the CVPR 2021 Embodied AI Workshop, and repeated at CVPR 2022 and CVPR 2023. The video below provides further details of the task, starter code, and results from the first workshop: