One of the testers who's writing and work I'm getting to know, Chris McMahon wrote about Testing Innocence. Wow-the link interface doesn't like this long of a link. Testing, what? Ok, workaround time-Copy/Paste the following longest link ever www.stickyminds.com/sitewide.asp?ObjectId=15405&Function=DETAILBROWSE&ObjectType=COLsqry=*Z(SM)*J(COL)*R(createdate)*K(colarchive)*F(~)*&sidx=3&sopp=10&sitewide.asp?sid=1sqry=*Z(SM)*J(COL)*R(createdate)*K(colarchive)*F(~)*&sidx=3&sopp=10

One thing that sucks about being a tester is I never get to USE software. I'm constantly finding bugs and I'm driven to report them to see them fixed. It annoys other people. If you collaborate with me, ask me to be a speaker, ask me to write something, I'll almost always report a bug or two about the software we are using. It isn't on purpose. I'm really trying to use the software, but I continue to have the tester eyes as after so long it is just part of who I am. I would say I've always been this way, but it has gotten worse the more time I've spent testing. After reading Chris' reasons why we should all read the code, I realized I have much to say on that, so I wrote the following.

Understanding how the code works is one good data point, but it is not the only important data to consider. In fact, I would argue that what is missing (or NOT in the code) is where the most important bugs lurk. In my work I have had endless frustration trying to convince testers and test managers that code coverage is NOT test coverage, that functional testing is not "the testing" and that testing for the user experience is vital and our automation isn't very good at it. My concern is that rather than a balanced approach where the code is reviewed and understood and considered as one important piece to data that informs the ever evolving test strategy, it is seen as the ONLY important input. I think understanding the code and testing from that alone is far easier than testing the software with a balanced approach. It requires less thinking. It is possible to be more innocent because you can pretend "If it isn't in the code it doesn't exist, or if it does exist it doesn't matter." If you focus your mind in only on the code, the scope of anything seems more defined. It may be comforting, but because I love puns I must point out that denial is more than a river in Egypt.

First, let's look at what you mean by "understand the code" or "read the code":
1. You understand the design? Usually not. The code is not the design.

2. You understand the psuedocode. (You'd better! If you can't understand how it works, what are you testing?)

3. You can map/model the code. (Yes, I see this is critical).

4. You can write the psuedocode for the part of the product you are testing. (I think if you are pretty accurate here that would be a good exercise, but the map/model is more useful for testing)

5. You can read and have read all of the code for the entire system under test (Got a lifetime if you test Windows? LOL)

6. You can and have read the code changes and bug fixes that go into the product under test (Ok, this makes sense to me sortof-more than curling up with the code for an entire product does. I'm envisioning a long beard and flowing hair before you know one feature in the case of the products I work on).

7. You can write some useful code to test your product. (Yeah--I can see this matters.)

8. You can write code on your own product. (Hmmm. The lines between tester and developer are blurring. As much as I'm shamed to admit that my own dev counterpart fixed my Python because he needed it sooner than I could fix it, he helps me sometimes, why do I not help him? Well, I assume he wants the core product to work even sooner, so maybe I should? I still think if we went head to head at testing I'm winning in terms of important bugs found, but who knows? Maybe his understanding of the code would win? We may never find out.)

9. You are "on the bench" as a developer and your whole goal is to become a developer. (Why is it that some devs think we want to be them? I think some testers do have dev envy and want that job, but as I stated above, that isn't what I'm talented at. I like to find issues and see them fixed. I have natural talent for it. I'm faster and better at that than I am at coding. Like I said, for me, testing is natural, coding is painstaking for me. It's like digging a hole with a teaspoon and the whole thing is like a practice in patience. How long can I do it before I have a meltdown like a toddler. Reading someone elses' code? Much easier, but still not what I'm going to sit down with on a Sunday afternoon by the fire for fun.)


Now deemed as unimportant in the face of the almighty code is all usage data that can be found (actual and user study information), integration problems, user scenarios(real world and predictive), past issues with the software(including top support issues), risks, code changes (areas of activity), and prioritized environments along with quality goals and company goals for the software. Is there anyone left employed at this point who still thinks understanding the code isn't important? I think all of those people have mostly been laid off at this point. My question is, why haven't those who can't see or think beyond the code been talked to about bringing more balance into their testing. The code alone is not enough to test the overall quality of the software. It is certainly an important part of a good test strategy, but I think we are severely low on vitamin U and lots of software sucks to use because of it. Yes, we need a user focus supplement of Vitamin U and it needs to be better than just "let the user test it for us and mark it as Beta for 3 years". Well, that strategy has worked well for Google, but I don't think it will work that well for all software companies.

I don't think my view is innocent or misinformed. In fact, only one who has suffered the sugar crash of eating cotton candy for dinner could appreciate such a boring thing as the need for a balanced testing diet.

So, I pose a question to you. How do you keep balance in your test strategy while using the code? How do you keep the user in mind? How does the code better help you create test cases?