Data fusion of images and 3D range data

A robot is a machine that embodies decades of research and development. Born as a simple mechanical devices, these machines evolved together with our technology and knowledge, reaching levels of automation never imagined before. The modern dream is represented by the cooperative robotics, where the robots do not just work for the people, but together with the people. Such result can be achieved only if these machines are able to acquire knowledge through perception, in other words they need to collect sensor measurements from which they extract meaningful information of the environment in order to adapt their behavior. This thesis speaks about the topic of the autonomous object recognition and picking for Automated Guided Vehicles, AGVs, robots employed nowadays in the automatic logistic plants. The development of a technology capable of achieving such task would be a significant technological improvement compared to the structure currently used in this field: rigid, strongly constrained and with a very limited human machine interaction. Automating the process of picking by making such vehicles more smart would open to many possibilities, both in terms of organization of the plants, both for the remarkable economic implications deriving from the abatement of many of the associated fixed costs. The logistics field is indeed a niche, in which the costs of the technology represent the true limit to its spread, costs due mainly to the limitations of the current technology. The work is therefore aimed at creating a stand-alone technology, usable directly on board of the modern AGVs, with minimal modifications in terms of hardware and software. The elements that made possible such development are the multi-sensor approach and data-fusion. The thesis starts with the analysis of the state of the art related of the field of the automated logistic, focusing mostly on the most innovative applications and researches on the automatization of the load/unload of the goods in the modern logistic plants. What emerges form the analysis it is that there is a technological gap between the world of the research and the industrial reality: the results and solutions proposed by the first seem not match the requirements and specification of the second. The second part of the thesis is dedicated to the sensors used: industrial cameras, planar 2D safety laser scanners and 3D time of flight cameras (TOF). For every device a specific (and independent) process is developed in order to recognize and localize Euro pallets: the information that AGVs require in order to perform the picking of an object are the three coordinates that define its pose in the 2D space, $[x,y,\theta]$, position and attitude. The focus is addressed both on the maximization of the reliability of the algorithms and both on the capability in providing a correct estimation of uncertainty of the results. The information content that comes from the uncertainty represents a key aspect for this work, in which the probabilistic characterization of the results and the adoption of the guidelines of the measurement field are the basis for a new approach to the problem. That allowed both the modification of state of the art algorithms both the development of new ones, developing a system that in the final implementation and tests has shown a reliability in the identification process sufficiently high to fulfill the industrial standards, 99\% of positive identifications. The third part is devoted to the calibration of system. In order to ensure a reliable process of identification and picking it is indeed fundamental to evaluate the relations between the sensing devices, sensor-sensor calibration, but also to relate the results obtained with the machine, sensor-robot calibration. These calibrations are critical steps that characterize the measurement chain between the target object and the robot controller. From that chain depends the overall accuracy in performing the forking procedure and, more important, the safety of such operation. The fourth part represents the core element of the thesis, the fusion of the identifications obtained from the different sensors. The multi-sensor approach is a strategy that allows the overcome of possible operational limits due to the measurement capabilities of the individual sensors, taking the best from the different devices and thus improving the performance of the entire system. This is particularly true in the case in which there are independent information sources, these, once fused, provide results way more reliable than the simple comparison of the data. Because of the different typology of the sensors involved, Cartesian ones like the laser and the TOF, and perspective ones like the camera, a specific fusion strategy is developed. The main benefit that the fusion provides is a reliable rejection of the possible false positives, which could cause very dangerous situations like the impact with objects or worst. A further contribution of this thesis is the risk prediction for the maneuver of picking. Knowing the uncertainty in the identification process, in calibration and in the motion of the vehicle it is possible to evaluate the confidence interval associated to a safe forking, the one that occurs without impact between the tines and the pallet. That is critical for the decision making logic of the AGV in order to ensure a safe functionality of the machine during all daily operations. Last part of the thesis presents the experimental results. The aforementioned topics have been implemented on a real robot, testing the behavior of the developed algorithms in various operative conditions.