Individual disadvantage and policy performance: The makings of "model-based" composite indicators

In evaluation activity of (regional) policy, it is a very hard task to represent its multiple dimensions and the targets it affects. Indeed, a policy impact generally involves a combination of socio-economic aspects that are difficult to represent, and then to measure. In our specific case study, regional training policies are aimed at recovering the gaps in employability and social inclusion of weak unemployed individuals. Hence, a precise measurement of the their initial gap is an essential part of the evaluation exercise. But previous counterfactual evaluation of the net impact of training policies in Piedmont (a region in North-West Italy) show the mess to observe and take into account the manifold aspects of individual weakness. In fact, the target population consists of very disadvantaged individuals, who experienced hard situations in the labour market, such as youth with past school failures, low-educated adults, immigrants. Probit models confirm the existence of an initial disadvantage among trainees, but all attempts to represent individual weakness by describing the family and the living context were ineffective. To overcome this shortfalls, this paper proposes an evaluation model that considers the impact of the trainees' socio-economic conditions on the policy outcome itself, compared with a control group. Policies, individual attitudes, and eventual weakness do represent the explanatory variables (input) affecting labour insertion (output), which in turn affects the phenomenon of labour market inclusion (outcome). Clearly, all these phenomena can not be measured by a single indicator, but rather their several dimensions are expressed by manifest variables and elementary indicators (EI). A synthesis can be reached by composite indicators (CI), that weight different dimensions on the basis of their relative importance. Since the traditional data-centric weighting approach has many limitations, this paper proposes an approach based on CIs calculated by a structural equation model (SEM) estimated with Partial Least Squares method. It will be applied to training policies evaluation. In this context, a CI is seen as a latent variable (LV) of several dimensions that can not be measured directly but is linked to several EIs (manifest variables) by ties of either formative or reflective type (external model). Each CI can be linked to other CIs in a systemic vision (internal/structural model), represented by a set of interdependent linear regressions. The makings of SEM are applied to the example of training policies in Piedmont. The model will represent the LVs playing a central role in the case of training policies and their relationships. In similar previous studies, these models were applied to analyze the effect of the pedagogical approach on the employability of vocational training students in compulsory education. They proved effective to isolate the effects of initial conditions on either an individual and a background level.