Testing Nightmares

Several nights ago I awoke from a terrifying dream. I was in a classroom where they were training testers. Large banners of mission statements were hanging near the front and acronyms were posted around the room. The teacher upfront was sort of like an Amway spokesperson and would call out, "What are our A.B.C.'s?" This eager clean faced young Skippy replies, "Always Be Certain!" Everyone in the class agrees parroting back their learned acronyms. The guest speaker arrives and it was this developer I used to work with who I really respected, someone very smart but soft spoken. The slides come up and the presentation is a very technical and engineering focused slide show explaining nothing about testing in any way. As the presentation ends I start to ask questions about what testing risk do the changes introduce to try to get some discussion going about that. Silence and dirty looks. I try again and ask what development challenges were faced and get a vague answer. I give feedback that I'd like to learn more about the topic as it impacts testing, or at least follow up with someone who knows about the testing aspects of the technology and was looked at like I was crazy. Knowingly, the people looked at each other in the classroom. I was somehow shameful and an embarrassment.

I'm afraid that testing itself is becoming antiquated in the opinion of some people who are smart people, but simply do not see the point of testing beyond getting to complete code coverage. I worry that at the core there are some business people who do not understand that software testing has value because they are trying to engineer their way around doing any testing. Please note that this isn't about automated tools or test automation, because test automation when done well still should be about solving testing problems, and accomplishing tasks which result in higher software quality.

Assumptions that I fear:
1. Testing tasks can be done by anyone when using an incremental development model. Testing should be done by anyone on the team who has time in order to be efficient and "agile". In fact, by not having designated testers, you can possibly keep a faster velocity because everyone will do everything, including everyone coding on the project at the start of a sprint and everyone testing the features to get to done at the end of a sprint.
2. Testing tasks should be all done by developers. In fact, a CS degree and a testing certification are worth more than years of experience.
3. Testing qualifications are based on certification and coding expertise. Test knowledge is coding knowledge. Knowing how software is built in theory is far superior to testing because any non-code testing is "blind", "needlessly time consuming", and "manual" and is only done by the poor peons who do not know how to code well enough to use the better (automated) way of testing.
4. Good development methodology produces quality code without testing, therefore testing is obsolete as a practice and profession.
5. It is possible to move all testing upstream so far that it is no longer relevant to test completed and integrated and installed software in the environment it will be deployed on.
6. The ability to make good tools for other people to test with is equivalent to being good at testing as a profession.
7. While there is proof that it is cheaper to test than fix bugs once they have been released, if we just call our efforts for better development, such as unit tests and peer review and automated build verification and sanity tests our "testing efforts", then we have saved money and can safely assume no other testing is needed.
8. It is perfectly acceptable to sit in an office and write a pet project of code and not test because testing is not important, only coding is important. If the tool you are making is good enough other people will use it to test for you (what?), because there are so many testers left just waiting for another tool to do testing.

You know the most terrifying thing? I know that everything listed above is false. But why is it untrue? I'm aware that I can not PROVE that I know anything. I have no way to argue against those points and that is a failing of mine, not an incorrect assumption of someone else. I should be able to offer something besides my opinion and experience. Why don't I know a dozen references and studies and experts who can prove otherwise? Why is it acceptable that my knowledge has such a huge gap? I  mean, I'm not new at testing, nor am I some person on the fringe who never reads anything published in the industry. I seek out information, yet I clearly don't know enough about testing and I have nightmares about it.  The assumptions above can't be false just because I believe it to be false. The thing is, assumptions that are attractive that people are motivated to make because they save money in tight economic times do not need to be proven to be acted on. All it takes is a few words to a few people who are willing to believe it. A few whispers of "We don't need testing anymore. We've replaced it with robots and it's cheaper." to the right people, then what happens? If I am not prepared to stand up for testing with a well stated logical argument based on fact, I don't think I'm really good enough as a tester. I need to correct this.

I am looking for ideas, for science, for research, for proof that skilled human testing saves money. I believe that the majority of people in software today want to do what is best for the users while balancing out the difficult choices that need to be made in a tough economy to remain a viable business. While fear is irrational, I think it would be useful for me to have more facts and knowledge to back up my opinion that testing provides a better experience for the user. How does it save the company money long term? How can we measure how much and make sure that the value added is increasing as we improve our methodology and modernize our test practices? It is one thing to make sure that your test strategy is great at protecting user interests. Is it possible to also know that your testing is effectively contributing to lowering costs and providing value for shareholders? For projects that are for a "for profit" company, these questions are important and fair to ask. My nightmare may be that testing goes away and is replaced with nothing but smoke and mirrors, but to those higher up in the company, perhaps their nightmare is that the company is gone in a puff of smoke. Balancing the needs of the users with the needs of the company can be tricky and difficult, but I think that it is one area of test leadership that I don't know enough about. In order to grow I should have more knowledge and practice and understanding in this area. In the past I've avoided it because I felt that everyone always wanted to do what was best for the user, because what was best for the user was of course best for the company.

As a test leader, what ways do you express the value that testing adds to the stakeholders who are most interested in the results of testing based on money savings? How do you measure the ways that testing saves the company money?

 

What did you think of this article?




Trackbacks
  • No trackbacks exist for this post.
Comments
  • No comments exist for this post.
Leave a comment

Submitted comments are subject to moderation before being displayed.

 Enter the above security code (required)

 Name (required)

 Email (will not be published) (required)

 Website

Your comment is 0 characters limited to 3000 characters.