By Matthias Ratert
In all projects we test SW by executing automated test scripts and/or running many test cases stored in a Test Management Tool such as HP Quality Center. We check whether all requirements are implemented correctly. By performing Regression Testing we ensure they are working as expected when shipping a new software version to the customer.
To find new issues not discovered by Regression Testing, Exploratory Testing (ET) is one of the largest hypes within the testing community in recent years. The tester has to uncover situations that could negatively impact the end user in terms of reliability, stability and usability.
Trying ET was also helpful for us: We discovered new issues and it was fun to test. However it became increasingly difficult to come up with new and creative test ideas and too many areas of our complex system remained untested.
IST (incremental Scenario Testing), a Teleca methodology
In order to provide a testing alternative better suited to our needs, we developed Incremental Scenario Testing (IST). IST and its implementation, the IST Tool, were developed over several iterations. Soon we called it the creative director of our Exploratory Test sessions.
Testers are guided through a session by scenarios which are composed of test items such as preconditions, states and events. These test items provide the scenario framework; how to reach each specific test item is up to the tester. The expected result is not specified and has to be judged by the tester – as in ET.
The test scenarios are generated based on the occurrence probabilities of each test item. Scenarios evolve over time as previous test results are an important factor when generating new test scenarios:
• Failed test scenarios are re-tested whereas passed test scenarios are not re-tested regularly.
• Failed test items are used more often and passed test items are used less.
• The probability of unexecuted test scenarios being run increases over time.
This guarantees that each test session will identify new issues, independent of the SW development phase. The issues are validated by the IST Tool based on the tester’s experience. Reported issues are re-tested within the same test session and fixes are verified with a new SW build.
IST involves SW developers in the testing process. Modified or new functionality can be prioritized for testing. As the test subject can be exchanged easily and whenever needed, these SW changes are tested directly when integrated.
IST allows for flexible, adaptable and efficient test sessions that are suited perfectly for agile software development.
Testers are guided through a session by scenarios which are composed of test items such as preconditions, states and events. These test items provide the scenario framework; how to reach each specific test item is up to the tester. The expected result is not specified and has to be judged by the tester – as in ET.
The test scenarios are generated based on the occurrence probabilities of each test item. Scenarios evolve over time as previous test results are an important factor when generating new test scenarios:
• Failed test scenarios are re-tested whereas passed test scenarios are not re-tested regularly.
• Failed test items are used more often and passed test items are used less.
• The probability of unexecuted test scenarios being run increases over time.
This guarantees that each test session will identify new issues, independent of the SW development phase. The issues are validated by the IST Tool based on the tester’s experience. Reported issues are re-tested within the same test session and fixes are verified with a new SW build.
IST involves SW developers in the testing process. Modified or new functionality can be prioritized for testing. As the test subject can be exchanged easily and whenever needed, these SW changes are tested directly when integrated.
IST allows for flexible, adaptable and efficient test sessions that are suited perfectly for agile software development.
IST as open source
Finally our latest announcement: we decided that the IST Tool should be published as Open Source to provide this great methodology to the testing community. To be 100% sure that the implementation of IST is on the right level we first want to execute a pilot project. I’m looking for a partner right now – I hope I will find him on the Agile Testing Days in Berlin beginning of October. I will keep you informed ;-)
Finally our latest announcement: we decided that the IST Tool should be published as Open Source to provide this great methodology to the testing community. To be 100% sure that the implementation of IST is on the right level we first want to execute a pilot project. I’m looking for a partner right now – I hope I will find him on the Agile Testing Days in Berlin beginning of October. I will keep you informed ;-)
Posts
No comments:
Post a Comment