PNSQC 2009-Presentation
I had a fantastic time at PNSQC to the point I'm just worn out! I could not sit down and behave myself despite having recently had surgery, so it took me a few days, however, I have now prepared the reducing test case bloat technical paper along with the reducing test case bloat slides including audience suggestions! Soon I plan to group and reply to the suggestions, but here are the highlights.
Most of the suggestions are in the actual paper, but unfortunately NOT so much in my talk as I ran out of time and there wasn't time for the participants to read my paper before the talk, so I'm pretty sure most of them didn't realize how much work and research I did before speaking on the topic or how much of this stuff I've done.
I learned that I forgot to mention several things:
1. We need a paper for how to prevent creating bloat when writing test cases. That is not this paper, but I agree, good topic! I'd like to see some data on having an expiration date set for test cases.
2. I had not included the "start over" idea because I assumed that your test cases had more value than THAT, but that was an assumption that may be very invalid. By all means, if you are dealing with test cases of little value, starting over may be the only way to proceed.
3. The person who suggested to cut any case they couldn't automate?? You scare me so much.
4. I didn't go into detail about "remove duplicates" because from my findings so far, this is not the lion's share of the bloat, but it is worth mentioning.
The most creative ideas I heard back were to prevent bloat in the first place.
Most of the suggestions are in the actual paper, but unfortunately NOT so much in my talk as I ran out of time and there wasn't time for the participants to read my paper before the talk, so I'm pretty sure most of them didn't realize how much work and research I did before speaking on the topic or how much of this stuff I've done.
I learned that I forgot to mention several things:
1. We need a paper for how to prevent creating bloat when writing test cases. That is not this paper, but I agree, good topic! I'd like to see some data on having an expiration date set for test cases.
2. I had not included the "start over" idea because I assumed that your test cases had more value than THAT, but that was an assumption that may be very invalid. By all means, if you are dealing with test cases of little value, starting over may be the only way to proceed.
3. The person who suggested to cut any case they couldn't automate?? You scare me so much.
4. I didn't go into detail about "remove duplicates" because from my findings so far, this is not the lion's share of the bloat, but it is worth mentioning.
The most creative ideas I heard back were to prevent bloat in the first place.
Posted by Testy Redhead at 29 Oct 2009 
Categories: uncategorized
Tags: career conference test case bloat presenting pnsqc testing speaking
Categories: uncategorized
Tags: career conference test case bloat presenting pnsqc testing speaking
Lanette,
Great article.
I knew Matt Heusser thought highly of you but didn't know specifically why. You're interested in the same topics in testing that I am. Thanks for helping to get the word out about the benefits of pairwise and combinatorial testing. These powerful techniques are not nearly as well known as they should be given how powerful they are.
Please also see similar themed blog posts and presentations I've put together on pairwise and combinatorial testing methods. I've posted a few on my blog - http://hexawise.wordpress.com
I've also included a video explanation and tool walk through at http://hexawise.com/intro_movie
- Justin
Founder of Hexawise (a pairwise and combinatorial testing tool with free and commercial versions)
Twitter: http://twitter.com/hexawise
Reply to this
Hi Justin,
I'm glad that you understood from my paper that in those cases where test cases have been designed to cover every possible option why I suggest some tool that uses combinatorial pairs first as this is a well-respected way to reduce tests by many engineers and is fairly pain free to negotiate as a way to cover fewer functional tests. I like the idea of covering the functional tests as easily, early, fast, and low risk as possible so that we can move on to doing exploratory testing and other more user focused tests which humans are better at doing. I a big supporter of a balanced test strategy, which is using automation and tools to find those bugs which could block testing (Broken builds, regressions, functionality breaking behind us), and using humans to report on bugs that are harder to automate for, such as problems in the user experience, finding what is missing from the code, or even reporting that something feels slow or is frustrating to use.
Thanks for reading my paper and sharing your experience. While I don't often design tests with full functional coverage, when I do I start with the results of all pairs as the "full set" and reduce from there if needed. In some cases I can run all of the pairs, it depends on the time and resourcing of the project.
Thanks,
Lanette
Reply to this