Ensuring high object interoperability is a goal of integration testing for object-oriented (OO) software. When messages are sent, objects that receive them should respond as intended. Ensuring this is especially difficult when software uses components that are developed by different vendors, in different languages, and the implementation sources are not all available. A finite state machines model of inter-operating OO classes was presented in a previous paper. The previous paper presented details of the method and empirical results from an automatic tool. This paper presents additional details about the tool itself, including how test sequences are generated, how several difficult problems were solved and the introduction of new capabilities to help automate the transformation of test specifications into executable test cases. Although the test method is not 100% automated, it represents a fresh approach to automated testing. It follows accepted theoretical procedures while operating directly on OO software specifications. This yields a data flow graph and executable test cases that adequately cover the graph according to classical graph coverage criteria. The tool supports specification-based testing and helps to bridge the gap between theory and practice.
A program mode is a regular trajectory of the execution of a program that is determined by the values of its input variables. By exploiting program modes, we may make worst-case execution time (WCET) analysis more precise. This paper presents a novel method to automatically find program modes and calculate the WCET estimates of programs. First, the modes of a program will be identified automatically by mode-relevant program slicing, and the precondition will be calculated for each mode using a path-wise test data generation method. Then, for each feasible mode, we show how to calculate its WCET estimate for modern reduced instruction set computer (RISC) processors with caches and pipelines and for traditional complex instruction set computer (CISC) processors. We also present a method to obtain the symbolic expression for each mode for CISC processors. The experimental results show the effectiveness of the method.
Test case generation based on design specifications is an important part of testing processes. In this paper, Unified Modeling Language activity diagrams are used as design specifications. By setting up several test adequacy criteria with respect to activity diagrams, an automatic approach is presented to generate test cases for Java programs. Instead of directly deriving test cases from activity diagrams, this approach selects test cases from a set of randomly generated ones according to a given test adequacy criterion. In the approach, we first instrument a Java program under testing according to its activity diagram model, and randomly generate abundant test cases for the program. Then, by running the instrumented program we obtain the corresponding program execution traces. Finally, by matching these traces with the behavior of the activity diagram, a reduced set of test cases are selected according to the given test adequacy criterion. This approach can also be used to check the consistency between the program execution traces and the behavior of activity diagrams.
During component-based and object-oriented software development, software classes exhibit relationships that complicate integration, including method calls, inheritance and aggregation. Classes are integrated and tested in specific orders, where each class is added and tested one by one to see if it integrates successfully. A difficulty arises when cyclic dependencies exist—the functionality that is used by the first class to be tested must be mimicked by creating ‘stubs’ (sometimes called ‘mock objects’), an expensive and error-prone operation. This problem is generally called the class integration and test order (CITO) problem, and solutions must fully be automated for integration and testing to proceed smoothly and efficiently. This paper describes new techniques and algorithms to solve the CITO problem. New results include improved edge weights to more precisely model the cost of stubbing, and the use of node weights, which allows more information to be used. These weights are derived from quantitative measures of couplings between the integrated and the stubbed classes. Also, a new algorithm for computing the integration and test orders is presented. The technique is compared with an existing approach and found to be cheaper, get the same results when using edge weights exclusively, and yield better results when using node weights.
Generation of adequate test cases is difficult and expensive, especially for testing software systems whose input is structurally complex. This paper presents an approach called data mutation to generating a large number of test data from a few seed test cases. It is inspired by mutation testing methods, but differs from them in the aim and the way that mutation operators are defined and used. While mutation testing is a method for measuring test adequacy, data mutation is a method of test case generation. In traditional mutation testing, mutation operators are used to transform the program under test. In contrast, mutation operators in our approach are applied on input data to generate test cases, hence called data mutation operators. The paper reports a case study with the method on testing an automated modelling tool to illustrate the applicability of the proposed method. Experiment data clearly demonstrate that the method is adequate and cost effective, and able to detect a large proportion of faults.
Test coverage is sometimes used to measure how thoroughly software is tested and developers and vendors sometimes use it to indicate their confidence in the readiness of their software. This survey studies and compares 17 coverage-based testing tools primarily focusing on, but not restricted to, coverage measurement. We also survey features such as program prioritization for testing, assistance in debugging, automatic generation of test cases and customization of test reports. Such features make tools more useful and practical, especially for large-scale, commercial software applications. Our initial motivations were both to understand the available test coverage tools and to compare them to a tool that we have developed, called eXVantage (a tool suite that includes code coverage testing, debugging, performance profiling and reporting). Our study shows that each tool has some unique features tailored to its application domains. The readers may use this study to help pick the right coverage testing tools for their needs and environment. This paper is also valuable to those who are new to the practice and the art of software coverage testing, as well as those who want to understand the gap between industry and academia.
