Measure the Quality?
Measure everything of significance.
"I swear this is true. Anything that is measured and watched, improves." -Bob Parsons
I think the opposite also has to be considered. Be careful what you measure to make sure it is the MOST important because you are robbing the unwatched areas to put focus on the area being measured.
I believe that there are only 4 important measures of quality in software.
1. Customer Satisfaction.
2. Sales verses the last release and vs the competition.
3. Costs of creating and maintaining the software overall.
4. Total cost of tech support.
You must raise number 1 and 2 without sending 3 and 4 higher, or you can reduce 3 and 4. It's a choice much like you get with food and ladies. Healthy, cheap, and tasty. Pick any 2. Pretty, Smart, or Laidback. Pick any 2. I am sure there are some exceptions where you can raise 1 and 2 WHILE reducing 3 and 4, but in general it's tough to do.
I am pretty sure there are hundreds of ways to measure if not more. How do you measure? I have reasons why and abuses of measures I've seen for each of the items I think are harmful to measure. I am hoping that you can convince me otherwise. Explain to me why these items are worth the time, effort, and expense to track?
p.s. Whomever found this blog by searching for testyredhead and boyfriend, I've got one, but you should read my blog anyways! I swear I am qualified to make a testing blog, boyfriend or no. I'm a very difficult woman and certainly am easier to talk about testing with than to date.
"I swear this is true. Anything that is measured and watched, improves." -Bob Parsons
I think the opposite also has to be considered. Be careful what you measure to make sure it is the MOST important because you are robbing the unwatched areas to put focus on the area being measured.
I believe that there are only 4 important measures of quality in software.
1. Customer Satisfaction.
2. Sales verses the last release and vs the competition.
3. Costs of creating and maintaining the software overall.
4. Total cost of tech support.
You must raise number 1 and 2 without sending 3 and 4 higher, or you can reduce 3 and 4. It's a choice much like you get with food and ladies. Healthy, cheap, and tasty. Pick any 2. Pretty, Smart, or Laidback. Pick any 2. I am sure there are some exceptions where you can raise 1 and 2 WHILE reducing 3 and 4, but in general it's tough to do.
| Common Measures | My Opinion |
| Test Cases Passed/Failed | Waste of time |
| Number of bugs written | Total Waste of time |
| Number of bugs fixed | Can be useful if you know where and use this as a way to judge risk |
| Lines of Code Covered | Can be useful IF used wisely with known numbers for 1-4 above. Those trying to get to 100% deserve rebuke. |
| Percentage of test cases Automated | Total waste of time. I realize that people measuring this way just don't understand and aren't really evil, but I have to remind myself. |
| Duplicates | Only of value if used to determine how many testers noticed this bug. |
| Number of crashes | Somewhat useful if taken in context. |
| Days/Hours/Weeks/Months/Percentage of up time or mean time to crash. | Very useful if this is critical for the workflow of the users. A good measure taken in context of items 1-4 above. |
I am pretty sure there are hundreds of ways to measure if not more. How do you measure? I have reasons why and abuses of measures I've seen for each of the items I think are harmful to measure. I am hoping that you can convince me otherwise. Explain to me why these items are worth the time, effort, and expense to track?
p.s. Whomever found this blog by searching for testyredhead and boyfriend, I've got one, but you should read my blog anyways! I swear I am qualified to make a testing blog, boyfriend or no. I'm a very difficult woman and certainly am easier to talk about testing with than to date.
Comments