Bee — Using optical flow for obstacle detection and avoidance

This experiments tests the viability of optical flow for obstacle detection and avoidance. Some time ago, I read an article about how bees use the optical flow while flying. The trick is simple: Keep the optical flow on the left and on the right same. This alone keeps the bee in the middle between obstacles. It has also been shown, that bees use the optical flow for distance estimation. Various flying insects seem to employ optical flow for safe landing. Simple time-to-collision analysis reveals a distance to an obstacle.

There have been many (successful) attempts on using optical flow in robotics. Vision just seems appealing to us, humans. A combination of computer vision and simple, robust and tested-by-nature algorithm is simply irresistible. But stating that optical flow can be better than sonars in obstacle detection, that is just courageous ... or is it?

Optical flow

Optical flow is a difference between consecutive image frames caused by the motion of the observer and the observed objects. There are many methods how to estimate the optical flow. While testing with the Lukas-Kanade method, I realized that smooth (even partially) surfaces cause a strong non-homogeneity of the optical flow. Henceforth I focused on sparse representations of the flow, which tell me where to focus directly. The "sparse" approach is seemingly simple too: Find corresponding points of interest in the consecutive frames and consider their displacement to be the optical flow. Of course, "corresponding points of interest" is the crucial non-trivial part. Again, I started with a simple approach: I used the corner detector in the OpenCV. The following correspondence detection proved to be difficult, though. Depending on the sensitivity level, the scene contains either too many or too few points of interest. Finally, I turned to SURF (Speeded Up Robust Features) detector provided by the OpenSURF library. In the current svn snapshot of OpenCV, I believe, SURF detection should be included. With SURFs I am satisfied.

Experiments with a robot

For the experiments, I have set up the Mob2 robotic platform, which we use for robotic lectures at our faculty. The robot is controlled by a netbook (Asus EEE 901). The only used sensor is the built-in webcam in the chassis of the netbook. This webcam gives a very bad image, indeed. OK, the second sensor are the encoders used for the velocity control, but that is not important.

The control algorithm behaves as described above. SURFs give a sparse representation of an optical flow. The ratio between optical flow in the left half of the image and in the right half is used to determine the steering angle. The forward speed is inversely proportional to the steering angle, thus the robot does not try to move forward while turning. If a time-to-collision of some point of interest falls below a threshold, it is considered to be an obstacle ahead and the robot moves away from it. The ultimate goal is just to move forward and not to crash.

Gotchas

There is one big gotcha, not dealt with anywhere I have seen. The time-to-collision estimation assumes the robot to be moving straightly forward. That is not the case with a real robot, which turns around all the time. The optical flow induced by the turning of the robot causes many phantom obstacles near ahead. To avoid this effect, I applied a simple image stabilization technique to reduce the optical flow caused by rotation. According to the experiments, it seems that this stabilization should be performed after determining the steering angle.

The second gotcha is that the grounded robot is not a flying insect. A highly textured ground would induce a strong optical flow just in front of the robot, giving a false feeling of an obstacle. This is the reason why you see smooth non-textured ground surface in all my videos.

The third gotcha is the narrow viewing angle of the camera. Together with the purely reactive behavior of the algorithm this causes problems. As soon as the (near) obstacle falls out of view, it is forgotten and possibly crashed into while turning. One possible solution is to use a wide-angle camera. Not having one, I incorporated a simple short-term memory. This memory also filters out the noisy obstacles appearing in just one or two image frames.

Video

Indoors	Outdoors
ahead.avi (1.3MB; 10s):	out1.avi(4.0MB; 36s):
corner.avi (3.7MB; 34s):	out2.avi(15.1MB; 2m24s):
back_and_forth.avi (9.7MB; 1m31s):

Conclusion

Not having a robot with sonars, I cannot compare whether optical flow is or is not better. However, I do have some sonar readings available. These data made me look for other ways of obstacle detection. The optical flow approach surprised me pleasantly. I consider it a very viable alternative to sonars (wide angle, low quality) and infrared sensors (narrow angle, high quality) now. The available articles do not exaggerate as much as I expected :-)