Navigation
UCSC's text spotting code has been implemented and tested on a mobile robot at UCM, with the objective of demonstrating text discovery in an indoor environment. The robot is a Pioneer P3AT equipped with a SICK LMS-200 range finder and a stereocamera (Bumblebee 2). The control software is written in C++ and is based on Player and OpenCV.
The robot is equipped with the occupancy map of the environment, the text to be searched, and a set of approximate prior areas for the location of the text (defined as rectangular areas in global coordinates). The robot's initial position is unknown; it starts by attempting to localize itself based on laser range data using the AMCL algorithm (particle filter) provided in Player. Once it localizes itself, the robot begins moving towards the closest area in the set of priors. When it reaches the area, it begins to randomly explore the environment.
While the robot is running, an image processing pipeline is executed, which turns the stereo camera data into goal candidates. Images are initially processed by the text spotter, which produces bounding boxes for potential text regions. These regions are merged together based on depth information and turned into candidate goals, characterized by the orientation and position of the potential region of text. Existing candidates are updated by subsequent text spotter iterations; they are rejected if they do not re-appear (which typically indicates that the candidate did not represent a valid region of text). Candidates are used by the goal seeking process only after they have been seen in a certain number of consecutive image frames.
Once the robot detects a candidate inside the prior area towards which it is moving, it stops random exploration and begins navigating towards the candidate. At this point the main thread begins running OCR on the captured images. OCR uses only the regions provided by the text spotter for computational efficiency, and merges results from multiple images to minimize error and determine whether the candidate matches the current label. The robot navigates to a position in front of the label at a distance of approximately 2 meters, and decides whether the text is a match based on the results of OCR. If the text is is equal to the known target text, the robot assumes the label has been found. Otherwise, the robot labels the prior area as visited and restarts the proces moving towards the closest unvisited prior area. This video shows an example of navigation with text discovery.
