Temperature Control for Automated Tape Laying with Infrared Heaters Based on Reinforcement Learning

The use of fiber-reinforced lightweight materials in the field of electromobility offers great opportunities to increase the range of electric vehicles and also enhance the functionality of the components themselves. In order to meet the demand for a high number of variants, flexible production technologies are required which can quickly adapt to different component variants and thereby avoid long setup times of the required production equipment. By applying the formflexible process of automated tape laying (ATL), it is possible to build lightweight components in a variant-flexible way. Unidirectional (UD) tapes are often used to build up lightweight structures according to a predefined load path. However, the UD tape which is used to build the components is particularly sensitive to temperature fluctuations due to its low thickness. Temperature fluctuations within the production sites as well as the warming of the tape layer and the deposit surface over longer process times have an impact on the heat flow which is infused to the tape and make an adaptive control of the tape heating indispensable. At present, several model-based control strategies are available. However, these strategies require a comprehensive understanding of the ATL system and its environment and are therefore difficult to design. With the possibility of model-free reinforcement learning, it is possible to build a temperature control system that learns the common dependencies of both the process being used and its operating environment, without the need to rely on a complete understanding of the physical interrelationships. In this paper, a reinforcement learning approach based on the deep deterministic policy gradient (DDPG) algorithm is presented, with the aim to control the temperature of an ATL endeffector based on infrared emitters. The algorithm was adapted to the thermal inertia of the system and trained in a real process environment. With only a small amount of training data, the trained DDPG agent was able to reliably maintain the ...