Tackling the Equivalent Mutant Problem in Real-Time Systems: The 12 Commandments of Model-Based Mutation Testing

Mutation testing can effectively drive test generation to reveal faults in software systems. However, it faces a typical efficiency issue as it can produce many mutants that are equivalent to the original system, making it impossible to generate test cases. We consider this problem when model-based mutation testing is applied to real-time system product lines, represented as timed automata. We define novel, time-specific mutation operators and formulate the equivalent mutant problem in the frame of timed refinement relations. Further, we study in which cases a mutation yields an equivalent mutant. Our theoretical results provide guidance to system engineers, allowing them to eliminate mutations from which no test case can be produced. Our evaluation, based on a proof-of-concept tool and an industrial case from the automotive domain, confirms the validity of our theory and demonstrates that our approach can eliminate many of the equivalent mutants (88% in our case study).