A framework for the validation and testing of access control systems

Context: In modern pervasive applications, it is important to validate access control mechanisms that are usually defined by means of the standard XACML language. Mutation analysis has been applied on access control policies for measuring the adequacy of a test suite. Objective: The paper targets two goals: 1) providing an automatic framework for test strategy assessment; and 2) providing an environment for running controlled experiment replications. In particular, the framework aims both the measurement of fault detection capability of the different testing strategies and the testing of the Policy Decision Point component of the access control systems. Method: We conducted a controlled experiment considering nine real-world access control policies aiming to evaluate two selected test strategies: a combinatorial and a model based testing approach. The assessment has been performed by using mutation analysis and computing the mutation scores reached by each strategy. The rate of fault detection has been considered as assessment metric. The description of the components of the proposed framework and their role during the controlled experiment are also provided. Results: The preliminary results show that model based test strategy achieves the same fault-detection effectiveness reached by combinatorial one in almost all cases, but employing a smaller number of test requests. Conclusion: We conclude that the test cases generated by the model based test strategy can be used usefully under budget constraints.