Impact assessment is usually based on direct surveys of treated individuals. The net impact should be estimated observing untreated individuals which are very similar to the treated group. However, the high cost of implementation does not allow to use large samples, and therefore it is difficult to carry out statistically significant comparison at the local level. The open access to databases which were created for different purposes (e.g., tax, administrative, monitoring) could overcome these limitations with many advantages. In this article we present a validation exercise based on data records provided by the Italian law whenever a change in an employment contract occurs, i.e. the so called compulsory communications (COB). This was possible because of an exceptional access to two different data sources, with the aim to assess the impact in terms of employment of vocational training policies in Regione Piemonte (northwest Italy), that is the COB database and a direct survey performed on students one year after the course. We describe the major differences between indicators calculated on survey data and indicators calculated on COBs. Discrepancies are observed for 20% individuals in the sample analyzed. In addition, the change of source distorts the results of the net impact evaluation. In fact, both the determinants of the employment probability and the importance of the net impact do significantly vary across sources. Whenever an administrative database is used as a source for socio-economic analysis, it is essential to be cautious and critic, validating its reliability by comparison with different data sources. In the absence of such a preliminary validation process, which should take place in close collaboration with regional and national authorities managing the informational systems, researchers do not only risk to uncritically accept information that provide systematic distortions, but they prevent corrective procedures which are fundamental for the development and improvement of the whole system.
I dati amministrativi per la valutazione delle politiche: riscontri dall'esperienza piemontese sul FSE
Ragazzi E;Sella L
2014
Abstract
Impact assessment is usually based on direct surveys of treated individuals. The net impact should be estimated observing untreated individuals which are very similar to the treated group. However, the high cost of implementation does not allow to use large samples, and therefore it is difficult to carry out statistically significant comparison at the local level. The open access to databases which were created for different purposes (e.g., tax, administrative, monitoring) could overcome these limitations with many advantages. In this article we present a validation exercise based on data records provided by the Italian law whenever a change in an employment contract occurs, i.e. the so called compulsory communications (COB). This was possible because of an exceptional access to two different data sources, with the aim to assess the impact in terms of employment of vocational training policies in Regione Piemonte (northwest Italy), that is the COB database and a direct survey performed on students one year after the course. We describe the major differences between indicators calculated on survey data and indicators calculated on COBs. Discrepancies are observed for 20% individuals in the sample analyzed. In addition, the change of source distorts the results of the net impact evaluation. In fact, both the determinants of the employment probability and the importance of the net impact do significantly vary across sources. Whenever an administrative database is used as a source for socio-economic analysis, it is essential to be cautious and critic, validating its reliability by comparison with different data sources. In the absence of such a preliminary validation process, which should take place in close collaboration with regional and national authorities managing the informational systems, researchers do not only risk to uncritically accept information that provide systematic distortions, but they prevent corrective procedures which are fundamental for the development and improvement of the whole system.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
