Augmented reality experience with physical artefacts

This technical report presents a system to improve the engagement of the user in a virtual reality experience using economic, physical copies of real artefacts, made with cheap 3D fabrication technologies. Based on a combination of hardware and software components, the proposed system gives the user the possibility to interact with the physical replica in the virtual environment and to see the appearance of the original artefact. In this way, we overcome the current limitation of the cheap 3D fabrication technologies: a faithful appearance reproduction. Moreover using a consumer device for the real-time hand tracking and a custom electronic controller for the capacitive touch sensing, the system permits the creation of virtual experiences where the user with his hand can change the virtual appearance of the object using a set of personalization actions selectable from a physical 3D printed palette.