Time profile surrogates: A new method to validate the performance of seizure prediction algorithms.

RATIONALE: In a growing number of publications it is claimed that epileptic seizures can be predicted by the analysis of the EEG using different characterizing measures. Many of these studies suffer from a lack of statistical validation. Only rarely results are passed to a statistical test and verified against some null hypothesis in order to quantify their significance. To address this issue we recently introduced the method of seizure time surrogates (Andrzejak et al., Phys Rev E,67, 010901, 2003). Here we propose a new and complementary approach, the method of time profile surrogates. METHODS: We investigate the predictive performance of bivariate measures of synchronization (e.g., a measure for phase synchronization and the recently proposed event synchronization). These measures were applied to quasi-continuous multi-channel EEG recordings (typically lasting some days) of epilepsy patients using a moving window technique. From the time profiles rendered we first generate an ensemble of surrogates by a constrained randomization. Using the method of simulated annealing we impose the amplitude distribution as well as the autocorrelation function from the original time profiles on the surrogates. Then we apply our seizure prediction algorithm (a simple discrimination test for amplitude distributions of pre-ictal and inter-ictal intervals) to the original time profile as well as to the surrogates. Given the existence of detectable changes before seizure onset, highest discriminative values should be obtained for the true time profiles. RESULTS: Applying the discrimination test to the original time profiles lead to highly non-uniform results. While for most channel combinations the pre-ictal and inter-ictal amplitude distributions proved to be indistinguishable, a high degree of discrimination could be obtained for some channel combinations. This might reflect the existence of a detectable pre-ictal state but it could also be the spurious result of statistical fluctuations. This ambiguity could be resolved by the method of time profile surrogates. Indeed, different results were obtained for different measures. The null hypothesis could be rejected with varying levels of significance. CONCLUSIONS: We could show, that the performance value of a seizure prediction algorithm is not sufficient to answer the question whether epileptic seizures can be predicted using a certain measure. Additionally a test for its statistical validity is needed. For this aim we proposed the method of time profile surrogates, in which results are tested against a null hypothesis and a level of significance is assigned. We would like to emphasize that it was not our aim to prove or disprove the existence of a pre-ictal state with another sophisticated seizure prediction algorithm but rather to supply a new and general means to reliably evaluate the statistical validity of the performance of such an algorithm.