Robotic perception needs to go beyond object detection/recognition. Robots working in a human environment face the challenge of recognizing a wide variety of visual characteristics and affordances during the execution of a task, and answer queries that potentially go beyond what is directly perceivable (e.g. is this object a container, or does this object have a lid). We address this problem by extending RoboSherlock to enable knowledge-based reasoning about which of its perceptual experts it should run, given a task description, and also reason about the objects that it has recognized, to further examine them.Example queries implemented can be tried out here
. Currently only reasoning capabilities are shown, live execution of RoboSherlock on logged data will be added shortly.