If you saw me present at PNSQC, this is version 2.0 of this topic. I'll be going in to more detail about each suggestion and I'm also going to get into the topic of "what happened" as a result and explain why I don't give the metrics. I'll explain the data I have and why it isn't complete and you can draw your own conclusions from it.

When people giving presentations show a ___% reduction in customer issues and take all of the credit for it? Or a ___% increase in sales? Those people and I are not in agreement. I am part of a team. My team protects quality and provides information. When we write bugs, others make a choice about how to respond. The product quality is a combination of the design, the execution of the design, the testing done by any number of people, and decisions made based on the data from test, and then a number of factors beyond our control, such as the market the product is released in. I am a part of the product quality, but as an individual the results of the overall product do not exist in a scientific environment where the results can be reproduced the same way every time even if we did the same exact things for bug finding as the time before. When you compare software version over version without taking into account the factors you don't control you compare oranges to apples and quite often you are misleading yourself. So, I could get up there tomorrow and report that despite the changes in legacy test case coverage n% fewer customer calls were logged in this version vs the last version. I would also be a misleading you and taking credit for work that isn't mine. I would not respect myself if I did that. The fact that if you check into the revenue of the company that is available in public you can make assumptions of sales versus the last version, and you'd realize that I'd puff myself up based on selective numbers. So, there will be none of that. I'll tell you my experience, what I did, and what happened, but when we are talking about the work of many teams I'll tell you what I see as the trends, what the risks and rewards were, and what my perspective is. I can't tell you "the truth", I can only share my own perspective with you in a way that allows you to draw your own conclusions for how it turned out.

I'm excited to share with you some survival tips when you have way more legacy test cases than you can possibly test. I survived a seemingly impossible situation using this strategy and even if the story isn't pretty, it's real and it is an exciting one. There is one tip that I share with absolutely anyone who is testing in tough times, and you'd better believe I'm putting my money where my mouth is right now. That is to keep testing. No matter what. Keep testing the product. Do not let the gap between reality and what the perceived quality of the product is grow further apart on your watch. Even if you are spending the whole day working on your automation, even if you are doing test planning, at least know the status of the software. Even if only 15 minutes was spent verifying that you still don't have a build. As a tester, know the status of the software. Even if you are a test manager, KNOW what is going on. Why? Because the people who don't? You don't want to be like them. They are too far from the heart of testing software. If you love it. I mean if you love software testing and you love quality, you will feel better just knowing how the build is today no matter what else is going on around you. That is the minimum viable tester to me. Do you even know what is going on with the product beyond what people tell you? Because in a crisis people lie sometimes. Not on purpose, but they are scared. Take the time to get in touch with the truth just a little bit.

Ok, I'm off of my soapbox now, and back to testing, but come say hello at Microsoft if you are there tomorrow. Your questions are welcome!

Also, I was looking for a speaker bio photo and I wanted to post this in my blog as this picture pretty much sums it up. I'm a big dork. Here's what I wish my author photo was. The reason is? This is from the fair this year. I had so much fun there! I have this much fun when I speak about testing.