The TLM machines with keen eyesight

The new 3D camera system brings to life the concept of fully automatic “bin-picking” for complex products in any position. Whether unsorted, shifted or even lying on top of each other, the 3D image processing system’s four cameras and two light pattern projectors accurately recognise fed products of all shapes and colours in terms of position, height and orientation. This means that the robot tool can precisely pick up components that fed in an unformatted pattern, eliminating the need for complex sorting and separating systems. After each robot intervention, the objects are again recorded three-dimensionally, and the position and orientation of the next product to be picked up is calculated. For this reason, the product intake also functions reliably if the arrangement of the other products changes due to the removal of one product.

The new powerful camera system generates very high-resolution 3D images within fractions of a second. This makes it possible to quickly recognise the height and quality of an object in addition to its position and orientation, which ensures that the robots can pick up products in any position in three-dimensional space. The two camera pairs offer the best all-round view and fewer gaps in the image than conventional single-shot 3D camera systems. Additionally, there is a projector between each camera, which throws a random light pattern onto the products. Light pattern projection and stereo image processing are intelligently combined to create optimal point clouds. The height and surface control is a central function in the recognition of products in 3D space and in their qualitative evaluation.

The vision system’s image data and the height profile are displayed on the operator terminal. Each product to be detected is characterised by a corresponding set of parameters, which is accessible to the plant operator via an associated menu. The type and number of parameters are application-dependent and can be individually adjusted. Depending on the task, they are defined in advance and stored in the control system. No adjustments are necessary during operation.

The image processing experts at Schubert are also currently researching an intelligent system for quality assurance. Artificial intelligence (AI) and deep learning are the buzzwords here: neural networks (in particular CNN, Convolutional Neural Networks) are to be trained in advance with several thousand product images and can then recognise whether a certain product meets the quality criteria and should be further processed. “Baked goods exhibit a high variance in shape and surface. Additionally, there are a variety of possible defects per product class. These two factors make the mathematical description of the shape and quality characteristics a challenge. Neural networks are being used increasingly as a powerful instrument to accomplish such complex tasks,” said Abdelmalek Nasraoui. As with the 3D scanner and the new 3D camera system, the implementation of neural networks in Schubert’s packaging machines represents further milestones in the company’s own image processing.