Software certification by demand: an experience from a third-party laboratory

There is still a sort of ambiguity surrounding the expression "software certification". The dated, common-sense IEEE definition [IEEE/729] "a written guarantee that a system or computer program complies with its specified requirements" does not provide useful guidance for organizing an effective certification service. Besides, a certificate issued according to this interpretation is inevitably mistaken for a "guarantee" of product adequacy, that which can definitely not be risked on at the state of the art. As an independent laboratory which has been offering this kind of service for some years, on request from Public Administrations, users and producers, we have been becoming more and more aware of this. Thus, we have conceived and applied a process in which a defined set of characteristics of both software product and developing project undergo a defined Validation Suite. This process ends with a formal declaration illustrating the actions performed and their results. This is what we mean by software certification. The Validation Suite must be designed for a class of product criticality and may be adjusted for each application field. In practice, our laboratory had to perform both the tasks of putting together Validation Suites for a number of cases and of carrying out the certification processes. If we would refer our work to some proposed certification schemes (like the European Community's), this would not be exactly the role of a certification body, which is not supposed to work directly in definition of standards. However, lack of established, commonly agreed rules, along with pressure from the demand side, convinced the laboratory to take on both aspects of the job. The paper describes some critical aspects faced by the laboratory during the preparation and application of the Validation Suite to actual software products certification. In particular, the concept of certifiability (that determines the feasibility of applying a Validation Suite leading to a positive certificate) is added to the yet conspicuous set of-ilities qualifying a product. Techniques used in the Validation Suite are picked up from defined standards and recommendations for designing "good" software, and provide a way to deal with imperfect user specifications. Such techniques include inspection, testing and tracing specifications to code and vice-versa. Problems rising from the peculiar experience of independent certification, such as dealing with manufacturer's privacy and interfacing software products to external inspectors, are also presented and a trace for their solution is proposed.