Back to Top
Andrzej Pronobis
UW KTH

InfoBot - Information Gathering Robot Answering Natural Language Questions About the Environment

About

Advances in mobile robotics have enabled robots to autonomously operate in human-populated environments. Although primary tasks for such robots are typically fetching, delivery, or escorting, they present and opportunity to assist human users in their daily tasks by gathering and reporting up-to-date knowledge about the environment which they co-inhabit.

In this project, we first sought to understand the requirements for such information gathering robots (InfoBots) from the perspective of the user requesting the information. To this end, we conducted user studies and Wizard-of-Oz deployment of an InfoBot in two office buildings with the inhabitants of the buildings asking questions to an InfoBot through a web-based interface. These studies allowed us to characterize the types of information that InfoBots can provide for their users.

Second, we implemented an autonomous, end-to-end InfoBot framework that enables a mobile robot to answer natural language questions about the state of a large-scale, dynamic environment. The system parses the question from a user and estimates an initial viewpoint that is likely to contain information for answering the question based on prior environment knowledge. Then, it autonomously navigates towards the viewpoint while dynamically adapting to changes and new information. Finally, the system captures images of the most relevant part of the environment in order to deliver the answer to the user through the web interface. We evaluated our approach with a custom mobile robot deployed in a university building, with questions collected from occupants of the building.

Related Publications