The Cost Function of the Data Fusion Process and its Application

Remote sensing observations of the atmosphere are performed with instruments operating on space-borne and airborne platforms, as well as from ground-based stations and vertical profiles of atmospheric variables are often obtained with an inversion procedure from the observed radiances. When an instrument or several instruments observe more times the same portion of atmosphere, the information obtained from the different measurements can be combined in order to obtain a single vertical profile of improved quality with respect to that of the profiles retrieved from the single observations. The most comprehensive way to combine different measurements of the same quantity is considered to be the simultaneous retrieval, in which all the observations are used as inputs of a single retrieval algorithm that produces a single profile. However, recently a new method, referred to as Complete Data Fusion, was proposed that, with simple implementation requirements, provides products of quality equivalent to that of the simultaneous retrieval products. The use of the Complete Data Fusion highlighted a problem that we believe to be common to simultaneous retrieval and data fusion. The measurements that we wish to fuse often present some inconsistencies due to three causes: (i) the profiles to be fused (in the following referred to as fusing profiles) are represented on different vertical grids, (ii) a variability is present in the observed species and the fusing profiles refer to different times and space locations and (iii) the fusing profiles are affected by different forward model errors. These inconsistencies may spoil the quality of the fused profile. In order to apply the Complete Data Fusion method to inconsistent measurements without a degradation of the product, it is necessary to add to the error covariance matrix of each fusing profile a covariance matrix that takes into account the inconsistencies. Therefore, the main problem in the fusion of inconsistent measurements is the realistic estimate of these inconsistency covariance matrices. The value of the cost function, which is minimized in the Complete Data Fusion, depends on the inconsistency covariance matrices and can be used to establish some constraints on their amplitude. To this purpose, we have analytically calculated the expected value and the variance of the cost function and used these quantities to define a procedure that estimates the inconsistency covariance matrices. Modelling the inconsistency covariance matrices with one parameter, we determine the value of this parameter that makes the reduced cost function equal to its expected value and use the variance to assign an error to this determination. The use of the Complete Data Fusion will be particularly relevant for the analysis of the future atmospheric Sentinel missions of the Copernicus program. The amount of data that will be available from these missions will pose technical challenges to most applications and the Complete Data Fusion can be used to reduce the number of products while maintaining the information content of the full datasets. For this reason, we test the proposed procedure on simulated measurements of ozone profiles as they could be obtained by the Infrared Sounder operating in the thermal infrared on board the Meteosat Third Generation satellite in the framework of the Sentinel 4 mission. In this context, the procedure is used to estimate the coincidence covariance matrices that take into account the variability of ozone when the fusing profiles refer to different times and space locations. The merits of this new procedure in the case of this specific dataset are presented and discussed.