I'm meeting up with a long time colleague next week to discuss my ideas for reducing test case bloat. I thought I'd share my ideas with you all in hopes that you'll share your thoughts with me as well!

Here are my ideas starting with the most common and ending with my favorite ideas.

1. Pairwise testing-Make sure to screen with equivalency cases.
2. Tiered by importance-Create a frequency subset based on risk.
3. Change based-Do a second pass and add more to lower priority tiers based on low risk due to code change.
4. Timing based-Ignore some test cases until major code changes are made, so schedule WHEN the testing will have maximum impact. If it isn't that time window, plan to ignore those tests.
5. Performance of the test case -How many bugs did that test case find last time? Tier accordingly, if low risk and low impact, ditch it.
6. Complexity-How many code paths does this test case cover? I prefer one testcase that covers maximum lines of code over a simple case.
7. Model Based-Draw model using UML. Map out which test cases cover what. Based on risk and planned code changes, assign tiered priority that way. Lower priority/low risk goes into "if time allows" bucket.
8. This is what I actually do-Use customer surveys and ask product marketing and project managers what their "top features" are for this version. Use actual usage data to prioritize tests-How many customers will this impact? If very few, ditch the test. If many, automate as BVT. Try to automate at least the top 5 for testability of basics. Set out in test plan what new features and what legacy workflows you are protecting and what you are ignoring because they aren't important enough. Recognize the RISK and get the guts to not test the low priority areas and get people to sign off on it. If the test doesn't fall into one of the most used workflows or one of the top priority experiences that the software is delivering, it can't be a tier 1 test. Tier 1 means that the test failure will stop shipment of the product. Tier 2 should be ran until RC submit. It means that if there is time, most likely it will be fixed if the test fails. Tier 3 means it may or may not be fixed, but would be good to know about.  All of this testing should be complete and abandoned (as finished for the cycle) before RC submit.


So, when you have infinite paths which could be tested, what makes sense for selecting test cases? In my situation, testing between products, my assignment is too large to do model based testing, or I would. I think it is a fantastic way to arrange tests. I hope that there are other ways to prioritize and reduce bloat that I missed here that some of you can share with me and I also hope some of these help you get some new ideas.

I tend to agree with James Bach when he says that the best test case is the one that hasn't been thought of yet. Never let your old test cases take priority over the new. Just like the girl scout motto, but a bit altered. Make new friends, but keep the BEST of the old. One is silver, the other is gold, the last one is growing mold.