Virtual Dubbing: a new concept for the regeneration of voice and acoustic ambience in cinematographic sound tracks

Cinematographic archive material poses strong research and technological challenges for its preservation as a cultural heritage and also for its utilization in new productions characterized by high quality postproduction environments. The UE research program IST RACINE-S is aimed to the development of innovative hardware and software for the high quality reconstruction of highly damaged or even missing pieces of archive cinematographic material. The Acoustics Laboratory of the FSSG-CNR at Venice contributes in particular to the development of innovative algorithms for the audio reconstruction of damaged films. In this framework, a method for dialogues and acoustic ambience reconstruction called Virtual Dubbing has been proposed. Virtual Dubbing was conceived for converting the voice of a professional dubbing speaker into a target missing voice. The information about the missing voice is taken from sections of the film previous or following the damaged part. State of the art LPC voice transformation algorithms have been developed at a research level and further implemented in a DSP hardware platform. Moreover, a non-linear perceptive model working in parallel with the LPC method has been developed. This non-linear model sorts high level perceptive parameters which can be used for the analysis of the voice signals and for high level control of the LPC synthesis step. On the other hand, acoustical ambience reconstruction has been based on a new intensimetric approach which allows a true physical characterization of the sound field.