PLASTIC Deliverable D4.1: test framework specification and architecture. PLASTIC (Providing Lightweight and Adaptable Service Technology for pervasive Information and Communication) (IST-2005-26955). Deliverable PLASTIC IST-2005-26955 - D4.1, 2007.

This document is the first deliverable of PLASTIC Work Package 4 (WP4): Service Validation Methodology and Tools. It provides the high-level specifications of the "Test framework specification and architecture" aimed at validating the PLASTIC applications. Generally speaking, the validation of B3G applications requires novel advanced technologies to face the high complexity of the systems considered, in terms of distribution, mobility, heterogeneity, dynamism, context-awareness, and the stringent requirements for quality of service and dependability. Given the blend of functional and extra-functional properties that need to be validated, in WP4 we aim at a holistic approach which includes analytical and empirical techniques to be employed both during development and at runtime. The framework has been organized around three main stages that we have identified as suitable for the validation of B3G services. A first main distinction concerns whether the services are tested in a fake/simulated environment before deployment, or rather after deployment, by validating their behaviour while they execute in the real environment. We name the two kinds of testing as off-line and on-line, respectively, and we claim that the validation of PLASTIC applications should stem from a suitable combination of both these kinds of testing. In particular, in addition to off-line testing, we adopt on-line testing because the high run-time dynamism of service-based systems makes it impossible to foresee all possible contexts in which the services will be used. Both off-line and on-line validation should consider the functional properties and the extra-functional properties. With regard to off-line validation, we propose advanced techniques such as simulation-based testing and model-based testing on symbolic state machine, and introduce support for distributed experimentation and automated test harness generation. In relation to on-line validation, we introduce a further distinction between i) on-line testing before publication, by which a service undergoes a sort of qualification exam, called Audition, before it can be "officially" recognized as having adequate quality for being included in the PLASTIC environment, and ii) monitoring of the service behaviour during real usage, in which case we provide for an aspect-oriented approach to monitoring service compositions, as well as for the verification that the contractually agreed extra-functional specifications are fulfilled. As a result of the rich variety of challenges, in WP4 we introduce a set of technologies. The proposed techniques used altogether provide a powerful multilateral approach to the validation of B3G applications. As their usage however requires some effort and implies the provision of adequate basic technology, they have been conceived so that each can be used independently from the others, and also they can be variously combined to obtain a more comprehensive approach. An integrated overview of the resulting architecture, in terms of a unified process, is given in the concluding part of this document.