Transfer learning for Renaissance illuminated manuscripts: starting a journey from classification to interpretation

In recent years illuminated manuscripts have been extensively digitized, providing an unprecedented amount of material for computer vision research and the creation of better performing neural networks. In addition to character recognition, which has long been the main application field, these new resources have allowed the adoption of machine learning to detect decoration and miniatures that usually concern only a few pages of a codex but attract great attention especially from non-academic public. The paper presents ongoing research that aims to demonstrate the possible adoption of transfer learning for digitized artworks to improve a pretrained deep neural network ability to recognize handwritten pages, identify the layout elements and, specifically, figurative miniatures in Renaissance manuscripts to be used for the creation of an immersive interface for consulting and comparing images based on their iconography. After a brief introduction to contextualize the changes brought by the massive cultural heritage digitization, we will present some of the most interesting research conducted on both manuscripts and artworks. Next, the dataset built to train the model will be described, focusing on its composition and the image classification system adopted. In conclusion, we will then expose the training strategy chosen to minimize human effort by dividing the dataset into three groups before concluding with the first results obtained so far and the prospects for future development.