Twenty-Five Years of Pansharpening: A Critical Review and New Developments

Pansharpening is a branch of data fusion, more specifically of image fusion, which is receiving an ever-increasing attention from the remote sensing community. New-generation spaceborne imaging sensors operating in a variety of ground scales and spectral bands provide huge volumes of data having complementary spatial and spectral resolutions. Constraints on the signal-to-noise ratio (SNR) impose that the spatial resolution must be lower, if the desired spectral resolution is larger. Conversely, the highest spatial resolution is obtained whenever no spectral diversity is required. The trade-off of spectral and spatial resolution makes it desirable to perform a spatial resolution enhancement of the lower-resolution multispectral (MS) data or, equivalently, to increase the spectral resolution of the data set having a higher ground resolution, but a lower spectral resolution; as a limit case, constituted by a unique panchromatic image (Pan) bearing no spectral information. To pursue this goal, an extensive number of methods have been proposed in the literature over the last two decades. Most of them follow a general protocol, that can be summarized in the following two key points: (1) extract high-resolution geometrical information of the scene, not present in the MS image, from the Pan image; (2) incorporate such spatial details into the low-resolution MS bands, interpolated to the spatial scale of the Pan image, by properly modeling the relationships between the MS bands and the Pan image. The main objective of this chapter is to propose a comprehensive framework encompassing earlier classification attempts of pansharpening methods, which makes it possible to categorize, compare, and evaluate existing image fusion methods, as well as to develop new ones. The almost totality of methods can be accommodated into one of two classes. Such classes uniquely differ by the way spatial details are extracted from the Pan image. According to the new approach, all methods belonging to either of the two main classes possess complementary and predictable characteristics of spectral and spatial quality of fusion products, as well as typical behaviors in the presence of specific anomalies in the data, like aliasing of MS bands, mis-registration between MS and Pan and temporal changes between MS and Pan observations. A thorough experimental section with comparisons of several methods on very high-resolution (VHR) MS + Pan data highlight the assets of the new classification approach.