Testing, it really isn't about process.
This week I've been reading about software process. Agile, Performance, Waterfall, Lean, Six Sigma. It is assumed that testing is a part of these software development approaches, models, or paradigms. Is it? We hear "Test Methodology" or the "Software Testing Process". Is it really a process?
I don't know of thinking of it as a process helps. I'd like to think of it instead as an action, a goal, and important thing "to do". What is the "to do" of testing? Well, mainly to gather relevant information, with the goal of improving the overall quality. I am not one of those testers who believes the limit SHOULD be that testers provide information. Sometimes that is the limit of our power, but I believe testers can and often do coach and pair with developers to contribute to quality of the product up front. Even just by asking questions early on, even just by listening and adding in tests, even just by quickly providing feedback that developers can count on every build. If they know each day they are going to hear sanity pass/fail and benchmarks faster/slower that is feedback they can rely on. It is a reason to be careful, it is a reason to finish up so that the tests won't fail. It isn't about being tricky or "getting those developers", anymore than it is about them sneaking a bug past us. Giving reliable quality feedback to the team often can build trust. In the same manner, when developers who have a tricky fix to check in are willing to walk me through it, give me their opinion on what to look out for, and show me that they CARE about the testing, they build trust with me. They have a bank account of credibility with me as a tester. If they did this on every little tool tip spelling change I'd be annoyed and wonder if they were paranoid or what, but if on a complex or late change they take time to do this I think it is very cool. In the same way, in a situation where we need to do less testing than is ideal, I'll write a brief test strategy email and send it just to the few devs involved. Then I'll meet with them in person if I can, or just on IM or phone to see if they have suggestions. Ok, not JUST to see if they have suggestions, but also to see if it triggers any warnings in their mind when they think about that fix. If I did this for every single fix it would be a total waste of time, but I think it builds trust if they know I'm really serious about getting good coverage in high risk situations.
I bring this up because this pairing is inefficient. If process and savings are your top goal at every point in the project, our team should have been penalized for this communication. However, if you are rewarding the success of the product, the quality of the product, and the value the team delivers by meeting the goals as a team, this process makes perfect sense. That is why I want to talk more about how to do testing as a verb in different environments, in different business structures, and despite whatever process is in place to deliver value. I think process can be harmful and come between the team and the goal rather than guide the team to reaching the goal. When that happens, usually the team will hide the real process the use, and pretend to use the prescribed process, when actually they aren't following it. One example of this faux process I see is people who claim they have "no manual tests". What they mean is that they don't document or admit to doing any tests, but people DO test the software. They just don't talk about it, or it is when they are "doing other things", like verifying a bug fix, fixing automation, or isolating a potential issue.
My last project was the closest I ever have been to having a process support the real work we were doing. We used scrum. That is why I like agile. However, I am not an agile zealot. I want to work on a team where we spend our time and energy meeting the goals the software has, and working well as a team, not forcing an unnatural process. It is joyful and rewarding to see the software getting better as a result of testing and bug fixes during a sprint. That sort of reward makes it not feel like work. Being productive is an unbelievable feeling. Knowing that you've made your whole team and product more productive, just as they have done for you is something on an entirely different level.
This Tuesday, after more than 10 years at Adobe, I walked in, turned in my badge, laptop, VPN token, and travel card. I said goodbye without any tears. I left feeling very proud of the software worked on, but the last project I'm the most proud of. I didn't attend the last All Hands meeting for our team because they were talking about "Future Stuff", and frankly I wasn't sure if I could deal with them announcing my 10 year anniversary without crying, so I didn't want to risk it. Our team was awarded 2 peer awards. I know I was a part of that.
However beautiful the strategy, you should occasionally look at the results. -Sir Winston Churchill
I know that somewhere out there is my next team. They want someone dedicated who they can count on, who is as excited about making great software as they are. They are putting people before process right now. They are more concerned with reaching team goals than numerical efficiency. They want to be successful more than they want to be perfect at following a process. In fact, in order to be successful, they may even make their own process if that is what it takes. Success in software isn't a process. Successful testing isn't a process either.
I don't know of thinking of it as a process helps. I'd like to think of it instead as an action, a goal, and important thing "to do". What is the "to do" of testing? Well, mainly to gather relevant information, with the goal of improving the overall quality. I am not one of those testers who believes the limit SHOULD be that testers provide information. Sometimes that is the limit of our power, but I believe testers can and often do coach and pair with developers to contribute to quality of the product up front. Even just by asking questions early on, even just by listening and adding in tests, even just by quickly providing feedback that developers can count on every build. If they know each day they are going to hear sanity pass/fail and benchmarks faster/slower that is feedback they can rely on. It is a reason to be careful, it is a reason to finish up so that the tests won't fail. It isn't about being tricky or "getting those developers", anymore than it is about them sneaking a bug past us. Giving reliable quality feedback to the team often can build trust. In the same manner, when developers who have a tricky fix to check in are willing to walk me through it, give me their opinion on what to look out for, and show me that they CARE about the testing, they build trust with me. They have a bank account of credibility with me as a tester. If they did this on every little tool tip spelling change I'd be annoyed and wonder if they were paranoid or what, but if on a complex or late change they take time to do this I think it is very cool. In the same way, in a situation where we need to do less testing than is ideal, I'll write a brief test strategy email and send it just to the few devs involved. Then I'll meet with them in person if I can, or just on IM or phone to see if they have suggestions. Ok, not JUST to see if they have suggestions, but also to see if it triggers any warnings in their mind when they think about that fix. If I did this for every single fix it would be a total waste of time, but I think it builds trust if they know I'm really serious about getting good coverage in high risk situations.
I bring this up because this pairing is inefficient. If process and savings are your top goal at every point in the project, our team should have been penalized for this communication. However, if you are rewarding the success of the product, the quality of the product, and the value the team delivers by meeting the goals as a team, this process makes perfect sense. That is why I want to talk more about how to do testing as a verb in different environments, in different business structures, and despite whatever process is in place to deliver value. I think process can be harmful and come between the team and the goal rather than guide the team to reaching the goal. When that happens, usually the team will hide the real process the use, and pretend to use the prescribed process, when actually they aren't following it. One example of this faux process I see is people who claim they have "no manual tests". What they mean is that they don't document or admit to doing any tests, but people DO test the software. They just don't talk about it, or it is when they are "doing other things", like verifying a bug fix, fixing automation, or isolating a potential issue.
My last project was the closest I ever have been to having a process support the real work we were doing. We used scrum. That is why I like agile. However, I am not an agile zealot. I want to work on a team where we spend our time and energy meeting the goals the software has, and working well as a team, not forcing an unnatural process. It is joyful and rewarding to see the software getting better as a result of testing and bug fixes during a sprint. That sort of reward makes it not feel like work. Being productive is an unbelievable feeling. Knowing that you've made your whole team and product more productive, just as they have done for you is something on an entirely different level.
This Tuesday, after more than 10 years at Adobe, I walked in, turned in my badge, laptop, VPN token, and travel card. I said goodbye without any tears. I left feeling very proud of the software worked on, but the last project I'm the most proud of. I didn't attend the last All Hands meeting for our team because they were talking about "Future Stuff", and frankly I wasn't sure if I could deal with them announcing my 10 year anniversary without crying, so I didn't want to risk it. Our team was awarded 2 peer awards. I know I was a part of that.
However beautiful the strategy, you should occasionally look at the results. -Sir Winston Churchill
I know that somewhere out there is my next team. They want someone dedicated who they can count on, who is as excited about making great software as they are. They are putting people before process right now. They are more concerned with reaching team goals than numerical efficiency. They want to be successful more than they want to be perfect at following a process. In fact, in order to be successful, they may even make their own process if that is what it takes. Success in software isn't a process. Successful testing isn't a process either.
"I think process can be harmful and come between the team and the goal rather than guide the team to reaching the goal. When that happens, usually the team will hide the real process the use, and pretend to use the prescribed process, when actually they aren't following it"
I fully agree. In fact, adherence to the process can sometimes become the goal. No-one would admit that, but the system of project governance that's set up, and the prevailing management style, mean that the emphasis is firmly on the process, with the application being almost a by-product.
When that happens you get an informal project, rather like the black economy, where the real work is taking place, with people doing unofficial tasks that aren't mandated by "The Process" but which help to get the work done.
It's horribly wasteful, because many valuable team members are spending their time on running the process rather than doing the job.
As you say processes should be a guide. Ultimately success depends on the people, their experience, skills and enthusiasm. Well crafted processes are of huge value, but they're an aid to good people. They're no substitute for them. It's a fairly obvious truth, but it's one that many big organisations are reluctant to live by.
Good luck for the future, though I suspect your success won't be dependent on luck!
Reply to this
I DM'd you on Twitter and thought I'd come back and comment. Right on about building trust with developers. There are developers that I especially like working with - the ones who "get it" and let me know areas of concern in their own code and are always open to my questions. And that's the type of developer who seems to like working with me too. We see each other as being on the same side rather than adversarial but each bringing different skills and perspectives. The product we end up with is pretty rock solid compared with projects where QA gets involved later in the process. Very encouraging to hear you articulate it!
Reply to this
Process means "a series of actions". So "Software Testing Process" means "series of Software Testing actions". Does that not sufficient to say testing is still an action in development approaches?
Reply to this
That is my point exactly. The series of actions matters much less than WHAT the actions are and who is performing them.
The series isn't that big of a deal. The talent and skill of the person doing them is a better predictor of success.
Reply to this