Methods of predicting mutation testing results

DOI: 10.31673/2412-9070.2024.050854

Abstract

Mutation testing is a widely recognized technique for assessing the effectiveness of test suites in detecting faults in software systems. Despite the effectiveness of this method, performing mutations requires significant computational expenses, which hinders its widespread adoption. In order to tackle the issue of efficiency, researchers have suggested different approaches such as selective, weakened, and predictive mutation testing. While various optimizations are utilized to cut costs, the current techniques and methods become unfeasible when dealing with an evaluation set that is not a static collection of tests but instead constantly evolving. This challenge arises due to the utilization of mutation testing as a metric to gauge the quality of a test suite developed through fuzzing. Both mutation testing and fuzzing are computationally intensive processes, creating a significant hurdle requiring innovative solutions. In light of these complexities, the optimization of computational resources becomes a key area of focus, aiming to balance efficiency without compromising the accuracy and depth of the testing methodologies employed. This intricate interplay between balancing costs, computational power, and testing effectiveness underscores the need for strategic optimization methods tailored to dynamic and resource-demanding evaluation scenarios. Finding approaches that allow for a quick and accurate assessment of mutation testing efficiency remains an open problem because mutation reduction techniques do not outweigh the random sampling method of mutants. 
One way to address this issue is by utilizing prediction methods that forecast outcomes without executing mutations. These prediction methods for mutation testing represent a new research direction in the field of mutation testing optimization methods. The discussed methods can be classified based on the type of machine learning (supervised and unsupervised learning), types of features (dynamic and static), level of prediction (at the mutant level and test suite level), and the type of mutations used (first-order mutations and higher-order mutations).

Keywords: mutation testing, machine learning, deep learning, model, optimization, framework, data analysis.