transparent gif


Ej inloggad.

Göteborgs universitets publikationer

Empirical evaluations on the cost-effectiveness of state-based testing: An industrial case study

Författare och institution:
Nina E. Holt (-); Lionel Briand (-); Richard Torkar (Institutionen för data- och informationsteknik (GU))
Publicerad i:
Information and Software Technology, 56 ( 8 ) s. 890-910
Artikel, refereegranskad vetenskaplig
Sammanfattning (abstract):
Context: Test models describe the expected behavior of the software under test and provide the basis for test case and oracle generation. When test models are expressed as UML state machines, this is typically referred to as state-based testing (SBT). Despite the importance of being systematic while testing, all testing activities are limited by resource constraints. Thus, reducing the cost of testing while ensuring sufficient fault detection is a common goal in software development. No rigorous industrial case studies of SBT have yet been published. Objective: In this paper, we evaluate the cost-effectiveness of SBT on actual control software by studying the combined influence of four testing aspects: coverage criterion, test oracle, test model and unspecified behavior (sneak paths). Method: An industrial case study was used to investigate the cost-effectiveness of SBT. To enable the evaluation of SBT techniques, a model-based testing tool was configured and used to automatically generate test suites. The test suites were evaluated using 26 real faults collected in a field study. Results: Results show that the more detailed and rigorous the test model and oracle, the higher the fault-detection ability of SBT. A less precise oracle achieved 67% fault detection, but the overall cost reduction of 13% was not enough to make the loss an acceptable trade-off. Removing details from the test model significantly reduced the cost by 85%. Interestingly, only a 24–37% reduction in fault detection was observed. Testing for sneak paths killed the remaining eleven mutants that could not be killed by the conformance test strategies. Conclusions: Each of the studied testing aspects influences cost-effectiveness and must be carefully considered in context when selecting strategies. Regardless of these choices, sneak-path testing is a neces- sary step in SBT since sneak paths are common while also undetectable by conformance testing.
Ämne (baseras på Högskoleverkets indelning av forskningsämnen):
Data- och informationsvetenskap ->
State-based testing, UML, Cost effectiveness, Automated testing, Empirical evaluation, Industrial case study
Postens nummer:
Posten skapad:
2014-05-16 17:00
Posten ändrad:
2014-07-10 10:57

Visa i Endnote-format

Göteborgs universitet • Tel. 031-786 0000
© Göteborgs universitet 2007