Testing in Isolation
What is isolation? In testing we find a bug and we think first about isolating it. We want to break it down into the fewest steps possible to recreate the issue. The exact scope required to make the bug happen. If we can find out exactly what is causing the problem, all the better. It is much easier to route the problem to the correct programmer for a fix and have the bug evaluated by the right "bug deciding group".
Isolation also means you are testing one component at a time. Maybe instead of testing a core technology integrated into a product, you have a custom interface that allows you to test it by itself. This way you can tell if the application integration is the problem, or the code of the component itself. Perfect.
Unit tests are my example of isolation. A clean machine is another example of isolation. You don't want other software interfering. One application at a time. Perfect isolation. One tester working alone in an office with the door shut. Or one tester total for the entire team and consider your team lucky to have one at all. The lone ranger. The tester in complete isolation. Or you don't test with any deviation at all. Test driven development meaning pre-defined tests that we code TO pass. A great way to start and when they pass it is wonderful to know you can START testing (although I'd run a manual sanity test first as well, but I digress).
So what is up with this trend of tester isolation? I see these conferences and there is no one else from my company there. Not ONE soul. They are busy writing more tests and taking classes to test in more advanced "isolated" ways.
In object oriented programming, it is ideal to create things in a modular, or isolated fashion. It really helps to keep the code clean, logical, and maintainable. I believe that testers who have had this rammed into their brain for years as developers first assume that it must be that testing is this way as well. The more logical, clean, and modular our tests, the better. For unit tests and the basic code coverage automation this is absolutely true. This is NOT enough. Testing in isolation is a huge problem because users do not work in isolation with one application at a time running on their hardware.
Testing in isolation is like eating a meal where you don't let any of your food items touch. You eat all of one food until it is gone, drink some water, wash all your utensils, then eat another item. It is unnatural, unenjoyable, and not a very good replica of what users will do. If our tests are designed to avoid potential problems, they are designed to avoid finding potential bugs as well. What makes good code doesn't always make good tests. I am not saying that our automation has to have haphazard and unmaintainable code, but only that we have to recognize the limitations of testing in isolation, be it automated or manual and know when it is time to move beyond those constraints.
What can you do if you are the only tester on your team? One thing you can do is test with your developer. Literally schedule a meeting with them and show them your tests charters. Don't throw some document over the wall and hope they read it. Instead, show them how you plan to test it. You also can test with testers on another team. Not constantly, but for variety you can exchange ideas. Ask to check out how they are testing their product and pair with them. Is this a waste of time if they are the more junior tester? Of course not! You get to reinforce your testing skills by explaining them to another person.
Test together. No tester is an island. If you are Tester Island, try building a bridge so at least you aren't totally testing in isolation. Ship in data. Ship yourself off to a conference. Ship in this blog on a hand truck if you must like they do in Venice, just don't convince yourself that isolation is an unchangeable fact about testing.
Loner cat don't need no stinking pairing. He will do it self.
Isolation also means you are testing one component at a time. Maybe instead of testing a core technology integrated into a product, you have a custom interface that allows you to test it by itself. This way you can tell if the application integration is the problem, or the code of the component itself. Perfect.
Unit tests are my example of isolation. A clean machine is another example of isolation. You don't want other software interfering. One application at a time. Perfect isolation. One tester working alone in an office with the door shut. Or one tester total for the entire team and consider your team lucky to have one at all. The lone ranger. The tester in complete isolation. Or you don't test with any deviation at all. Test driven development meaning pre-defined tests that we code TO pass. A great way to start and when they pass it is wonderful to know you can START testing (although I'd run a manual sanity test first as well, but I digress).
So what is up with this trend of tester isolation? I see these conferences and there is no one else from my company there. Not ONE soul. They are busy writing more tests and taking classes to test in more advanced "isolated" ways.
In object oriented programming, it is ideal to create things in a modular, or isolated fashion. It really helps to keep the code clean, logical, and maintainable. I believe that testers who have had this rammed into their brain for years as developers first assume that it must be that testing is this way as well. The more logical, clean, and modular our tests, the better. For unit tests and the basic code coverage automation this is absolutely true. This is NOT enough. Testing in isolation is a huge problem because users do not work in isolation with one application at a time running on their hardware.
Testing in isolation is like eating a meal where you don't let any of your food items touch. You eat all of one food until it is gone, drink some water, wash all your utensils, then eat another item. It is unnatural, unenjoyable, and not a very good replica of what users will do. If our tests are designed to avoid potential problems, they are designed to avoid finding potential bugs as well. What makes good code doesn't always make good tests. I am not saying that our automation has to have haphazard and unmaintainable code, but only that we have to recognize the limitations of testing in isolation, be it automated or manual and know when it is time to move beyond those constraints.
What can you do if you are the only tester on your team? One thing you can do is test with your developer. Literally schedule a meeting with them and show them your tests charters. Don't throw some document over the wall and hope they read it. Instead, show them how you plan to test it. You also can test with testers on another team. Not constantly, but for variety you can exchange ideas. Ask to check out how they are testing their product and pair with them. Is this a waste of time if they are the more junior tester? Of course not! You get to reinforce your testing skills by explaining them to another person.
Test together. No tester is an island. If you are Tester Island, try building a bridge so at least you aren't totally testing in isolation. Ship in data. Ship yourself off to a conference. Ship in this blog on a hand truck if you must like they do in Venice, just don't convince yourself that isolation is an unchangeable fact about testing.
Loner cat don't need no stinking pairing. He will do it self.
Posted by Testy Redhead at 7 May 2010 
Categories: uncategorized
Tags: methodology thinking isolation automation agile pairing testing teamwork
Categories: uncategorized
Tags: methodology thinking isolation automation agile pairing testing teamwork
I was the solo tester or nearly 4 years on a project and I did all of the above. I've met other solo testers and talked about this very subject. We agreed that the only way for solo testing to work is if there is transparency and collaboration on the team.
Unfortunately, because business can very misguidedly see testing as "no value add," there are many solo testers. These are testers who work in impossible conditions with less than cutting-edge tools. They must find creative ways to test if their testing is to mean anything and have the strength to defend what they've done, possibly without anyone standing behind them.
Although it is a tough position to be in, solo-testing can have it's rewards too. I admire those testers who turn the solo testing gig into more of a blessing than a curse.
I hope to see conferences, and writing-about-testing address solo testing more than they have.
Reply to this
Me too! I think these solo testers should present some and show us how they have success and overcome the difficulties of being the only tester.
Each conference I attend I'm realizing there are more and more solo testers on teams. I hope one of them writes a good book.
Reply to this
Did I read that you are coming to the end of your project? If so, we are looking for another tester so that I am not alone, holding down that fort. We have a great team doing .net dev and really want to do some strategic automated testing, though most the work is manual testing. Interested?
Reply to this