Monitoring manufacturing systems using AI: A method based on a digital factory twin to train CNNs on synthetic data

Modern cyber–physical production systems provide advanced solutions to enhance factory throughput and efficiency. However, monitoring its behaviour and performance becomes challenging as the complexity of a manufacturing system increases. Artificial Intelligence (AI) provides techniques to manage not only decision-making tasks but also to support monitoring. The integration of AI into a factory can be facilitated by a reliable Digital Twin (DT) that enables knowledge-based and data-driven approaches. While computer vision and convolutional neural networks (CNNs) are crucial for monitoring production systems, the need for extensive training data hinders their adoption in real factories. The proposed methodology leverages the Digital Twin of a factory to generate labelled synthetic data for training CNN-based object detection models. Regarding their position and state, the focus is on monitoring entities in manufacturing systems, such as parts, components, fixtures, and tools. This approach reduces the need for large training datasets and enables training when the actual system is unavailable. The trained CNN model is evaluated in various scenarios, with a real case study involving an industrial pilot plant for repairing and recycling Printed Circuit Boards (PCBs).