HR-RECYCLER Project moves on

HR-RECYCLER Project moves on

Since 2018 Sadako participates in the European Project HR-Recycler (Hybrid Human-Robot Recycling plant for electrical and electronic equipment), a new opportunity for the company to expand the impact of its technology in the waste and recycling industry.

HR-Recycler targets the development of a ‘hybrid human-robot recycling plant for WEEE (Waste Electrical and Electronic Equipment).

The overall goal of HR-Recycler is to create a hybrid collaboration environment, where humans and robots will harmoniously share and undertake at the same time different processing and manipulation tasks, targeting the industrial application case of WEEE recycling.

As an AI and computer vision company, the role of Sadako in the HR-Recycler project is to provide the crucial vision algorithms to the robotic systems, allowing the perception of the environment in which the machines evolve.

Because of the wide variety of vision-dependent systems being employed in this project (different types of robot arms and Automatically Guided Vehicles, central factory planner), many different challenges are encountered which need each their own specific solution. All of the designed solutions rely on the use of RGB-D cameras and computers capable of Neural Network inference and are a good illustration of the many capabilities and applications of computer vision.

Until now, Sadako has developed solutions for the Classification and Disassembly stages of the recycling process, as well as for the AGV navigation. These solutions consist fundamentally in the detection of WEEE objects and of human pose and gestures. The 3 main development areas have been:

  1. Classification: 3D perception and object recognition for bin picking
  2. Disassembly: Video-inference for human action recognition, needed for Human-Robot collaboration and task status management
  3. AGV’s: Human location perception, needed for optimal AGV path planning and human safety

 

  1. Classification: 3D perception and object recognition for bin picking

This stage is the first of the recycling process, where WEEE arrives mixed in baskets and must be sorted by type of object (Microwave, Flat Panel Display, PC Tower, Emergency Lamp).

We have developed Deep Learning vision algorithms and point cloud processing that identify the type and 3D orientation of each object lying in the basket and provide the robotic arm (a COMAU NJ-220) with a picking decision.

For that, we have previously created a relevant Dataset through several recording sessions in our lab and in project partners end-users (WEEE recycling plants) premises.

 

 

The bin picking functionality has been tested with the robot on a prototype classification cell on COMAU’s premises, as can be seen in the picture below.

 

 

  1. Disassembly: Video-inference for pose and action recognition

The second process step, Disassembly, is where the previously classified are dismantled and their parts are sorted by material to be sent to their final recycling location. The dismantling is made by an operator, with the support of a collaborative robotic arm.

In order to achieve collaboration with the robot, Sadako has created a vision system that comprises pose and action detection with 2 main functionalities: that the worker can interact with the workbench robot through a predetermined set of actions (gestures and tasks), and that the robot can change its behavior as a function of the human-robot distance.

To that aim, Sadako has developed AI-powered software capable of detecting actions in a video, in contrast with standard computer vision that detects objects in an image.

Software integration has been successfully tested with IBEC, on Sadako premises.

 

 

  1. AGV’s: Human location perception

AGVs are used to transport WEEE objects between each stage of the dismantling process, reducing high-strain and repetitive physical work for the operators. To ensure that the AGV trajectory does not interfere with the movement of humans around the factory, the central system must be aware of their position.

To achieve this, Sadako has developed a solution based on Openpose, a software that extracts the position of the joints of humans on an image, crossed with depth information provided by the camera, thus allowing to perceive the position and orientation of operators in three dimensions. This software can be deployed at the Disassembly workstation as well as onboard the AGVs.

 

 

HR-Recycler project moves on! Now towards the piloting phase, Sadako is staying strongly active and working on it in its compromise of developing technology to improve the world and reduce health risks endured by humans in the waste industry.

 

 

HR-Recycler is a project funded by the European Commission.

This project has received funding from the European Union’s Horizon 2020 Research and Innovation program under Grant Agreement n°820742.