Joint non-rigid registration and restoration of recto-verso ancient manuscripts

Ancient manuscripts written on both pages of the sheet are frequently affected by ink bleeding from the reverse side, which produces a significant degradation of the text. Effective digital image restoration techniques may require the use of the content of both document sides, thus needing their perfect alignment. Usually, recto and verso are not aligned either for rigid misalignments occurring during acquisition, or for non-rigid deformations of the sheet. In this paper we propose a novel method to jointly register and restore color recto-verso manuscript images in a piecewise manner, by subdividing the images into sub-images that exhibit apparent, different deformations of one with respect to the other. For each pair of corresponding sub-images, a specific projective transformation is computed, the two sub-images are registered, and then restored with a pixel-by-pixel algorithm that returns free of interferences versions of the images in their original acquisition layout. The projective transformation is estimated exploiting the precise computation of the shifts of a large number of small corresponding recto and verso patches, via correlation of their gradients. The experiments show that this combined procedure of local registration plus restoration can provide an excellent removal of bleed-through, while leaving unaltered the salient features of the original manuscripts.