FRIEND::Image Processing - ROVIS
Block diagram of ROVIS, the robust vision architecture for care-providing robot FRIEND.
The main problem with service robotic systems such as the care-providing robot FRIEND is that they have to operate in dynamic surroundings where the state of the environment is unpredictable and changes stochastically, hence two main problems have been encountered when developing image processing systems for service robotics: unstructured environment and variable illumination conditions. They have to cope with a large amount of visual information and for the implementation of the vision system a high degree of complexity is necessary. A second major problem in robot vision is the wide spectrum of illumination conditions that appear during the on-line operation of the machine vision system, since colors are one important attribute in object recognition. The human visual system has the ability to compute color descriptors that stay constant even in variable illumination conditions, which is not the case for machine vision systems. A key requirement in this field is the reliable recognition of objects in the robot's camera image, extraction of object features from the images and, based on the extracted features, subsequent correct object localization in a complex 3D environment so that these information can be used for reliable object grasping and manipulation.
In order to cope with the above described problems in the care-providing robot FRIEND the special vision system ROVIS (RObust machine VIsion for Service robotics)   was developed. The structure of ROVIS is depicted in the figure. There are two main ROVIS components: hardware and object recognition and reconstruction chain. The connection between ROVIS and the overall robot control system is represented by the World Model where ROVIS stores the processed visual information. At the initialization phase of ROVIS the extrinsic camera parameters needed for 3D object reconstruction and the transformation between stereo camera and manipulator which is necessary for vision based object manipulation are calculated by the Camera Calibration module. The object recognition and reconstruction chain consists of robust algorithms for object recognition and 3D reconstruction for the purpose of reliable manipulation motion planning and grasping in unstructured environments and variable illumination conditions. Therefore an accuracy of 5mm for the estimated 3D pose is necessary which enforces very good and precise algorithms. In ROVIS, robustness must be understood as the capability to the system to adapt to varying operational conditions and is realized through the inclusion feedback mechanisms at the image processing level and also between different hardware and software components of the vision system . A core part of the system is the automatic, closed-loop calculation of an image Region of Interest (ROI) on which vision methods are applied. By using a ROI the performance of object recognition and 3D reconstruction can be improved since the scene complexity is reduced.
Within ROVIS there are several methods used for object recognition, e.g. robust region based color segmentation  and robust edge detection. The first one is for objects with uniform color and without texture (e.g. bottle, glass, handles) and the second one for objects with textures (e.g. books). In order to recognize objects in FRIEND's environment robustly special feature are extracted which are used to identify objects and to determine their pose . For big objects like refrigerator or microwave an improved SIFT (Scale Invariant Feature Transform) algorithm is used, which was developed at the Institute of Automation at the University of Bremen .