AI methods for controlling digital gantry robot twins

The challenges facing the manufacturing industry in the course of Industry 4.0 require new solution concepts in many areas. Within the framework of the project KI-Verfahren zur Steuerung von Digitalen Portalroboterzwillingen (KISPo, engl. AI methods for controlling digital gantry robot twins), an innovative solution concept for the area of conveyor technology in production is presented. With KISPo, autonomous and self-learning control software for so-called gantry systems is to be developed on the basis of machine learning and simulation methods. In gantry systems, fully automated trolleys move horizontally along an axis to production machines for loading or unloading. The vertical movement towards the machine is performed by the so-called gripper. Several machines can be approached by one portal.

According to the current state of the art, gantry systems are controlled with the help of simple priority rules that are often set once based on experience and then rigidly stored in the control system. With the help of such rules, however, it is no longer possible to meet the increasing demands on production control that arise in the context of Industry 4.0 and ever greater pressure to adapt in the manufacturing sector. Although replacing rule-based controls with systems based on the machine learning method Reinforcement Learning (RL) is a generally recognised solution strategy, there is currently no commercially available control software for gantries with this technology. On the contrary, the German Engineering Federation (VDMA) writes in its guide Self-learning production processes - implementation strategy for reinforcement learning in industrial practice: "In particular, the area of reinforcement learning for autonomous control of production processes is still little or not at all developed in industry."

The aim of the KISPo project is to fill this gap and to design an RL-based software agent and develop a corresponding prototype. This is to be coupled with simulation models of portal systems in order to enable autonomous adaptation of the portal control to newly arising situations in the production environment and in this way to meet the challenges of production companies with a solution module in the sense of Industry 4.0.

The RL agent is based on a Deep-Q-Network (DQN) approach. PlantSimulation from Siemens and Anylogic are used as simulation systems.