How to create a really terrible internal Test Tool-6 Ways to cause it, 6 ways to help avoid it.
Steps to Create a Bad Internal Test Tool
1. Have people who are not testers define the requirements.
2. Have someone who doesn't understand the context of the problem, hopefully someone who doesn't speak the same native language, make a very liberal interpretation of those requirements and "let the technology guide them" to the right solution.
3. Do not listen to the main users of the software, assure them it is a staged development model that will improve as time goes on.
4. Keep implementing features on top of the features that are already impossible to use.
5. If people tell you it is difficult to use, show them how wrong they are and write down the exact 87 steps it takes to do one simple thing. Explain to them that they must not be using it correctly.
6. Have the engineer who implemented the code write all of the documentation and training, and then if someone points out that it has no context or conceptual data and doesn't make sense, argue with them and tell them, "Just follow the steps."
Steps to Help Avoid a Bad Internal Test Tool
1. Have very early pilot users who are testers, test leads, test managers, and even higher up stakeholders, but not just at one level.
2. Create a very early prototype that a person could use on a small scale.
3. Take usability bugs SERIOUSLY. This will block adoption of your tool. It isn't a minor UI issue if people can't or won't use your tool.
4. Have your power users create and review your documentation if they are willing to. Even ask them to create short training workshops and do demos for you. They will be better at explaining to people how to use it than you will because they ARE using it, not writing it.
5. If more than 20% of the people you've asked to look at it tell you that it is too difficult to use, do NOT start writing more features until you fix the useless ones you've created already.
6. Don't start adding features just because one person asks for them. It should be based on how many people need it, not on the title and visibility of the person who asked for it.
While I'm on a rant, some tester had the audacity to say, "You can never have too much automation." What? My head nearly exploded. You can never have too much well designed, effective, cost saving, reasonably maintainable automation which start to finish saves you time over running the tests manually. Sadly, there isn't enough automation currently meeting those criteria, and far too few people actually measuring. Automation doesn't equal GOOD. It just is a buzz word. Garbage in, garbage out.
Thanks for letting me vent, oh internet.
1. Have people who are not testers define the requirements.
2. Have someone who doesn't understand the context of the problem, hopefully someone who doesn't speak the same native language, make a very liberal interpretation of those requirements and "let the technology guide them" to the right solution.
3. Do not listen to the main users of the software, assure them it is a staged development model that will improve as time goes on.
4. Keep implementing features on top of the features that are already impossible to use.
5. If people tell you it is difficult to use, show them how wrong they are and write down the exact 87 steps it takes to do one simple thing. Explain to them that they must not be using it correctly.
6. Have the engineer who implemented the code write all of the documentation and training, and then if someone points out that it has no context or conceptual data and doesn't make sense, argue with them and tell them, "Just follow the steps."
Steps to Help Avoid a Bad Internal Test Tool
1. Have very early pilot users who are testers, test leads, test managers, and even higher up stakeholders, but not just at one level.
2. Create a very early prototype that a person could use on a small scale.
3. Take usability bugs SERIOUSLY. This will block adoption of your tool. It isn't a minor UI issue if people can't or won't use your tool.
4. Have your power users create and review your documentation if they are willing to. Even ask them to create short training workshops and do demos for you. They will be better at explaining to people how to use it than you will because they ARE using it, not writing it.
5. If more than 20% of the people you've asked to look at it tell you that it is too difficult to use, do NOT start writing more features until you fix the useless ones you've created already.
6. Don't start adding features just because one person asks for them. It should be based on how many people need it, not on the title and visibility of the person who asked for it.
While I'm on a rant, some tester had the audacity to say, "You can never have too much automation." What? My head nearly exploded. You can never have too much well designed, effective, cost saving, reasonably maintainable automation which start to finish saves you time over running the tests manually. Sadly, there isn't enough automation currently meeting those criteria, and far too few people actually measuring. Automation doesn't equal GOOD. It just is a buzz word. Garbage in, garbage out.
Thanks for letting me vent, oh internet.
Problems & Solutions:
1. Have the users define the requirements; agile, xtremeprogramming and integrate their desires in the specs.
2. I've got my hammer and will solve all problems with the technology I want to work with. And why not try a neat new technology I want to get my head around for the solution?
3. The users don't know what they want, the users are dumb and ignorant, why listen to them? they're a nuisance anyway and won't know how to properly use the software.
4. We'll want to implement some new, fancy design patterns and complex features that we don't even understand, the users, they're lost anyway.
5. The users must follow the instructions one step after another and if they don't get it we'll send in an army of consultants to advise them for 18 months.
6. That's right, just follow the steps and be thankful if it's written in standard English and not in binary format. That would really get them thinking.
Your steps to create an excellent internal test tool are great Testy!
1 Very early pilot users: testers, test leads, test managers, higher up stakeholders get their hands on the product and give immediate feedback to the dev team.
2 Create a very early prototype that we can use on a small scale.
3 Take all bugs seriously, if the user doesn't want to use the tool, that's failure. Eliminate all bugs. A bug is a feature the user isn't satisfied with and have him sign off
on it before moving on to the development of the next feature. You see, the last project I worked on, I didn't have written specs and the users changed their minds about
the feature set every day. They even wanted me to use open source tools, that took me a day to convince them of the contrary. Written sign offs in all phases of dev/test.
4 Documentation, better executable form of documentation: Unit Tests. The users will need trainers and consultants that we will provide at a cost. Agree that the testers
should write the alphabetical documentation or the developers or power users. The only problem with comments and documentation is that they are static and not dynamic. The
features and code base and with that the tests change all the time. Just this morning I had to rip out a series of unit tests that broke when I made some minor changes
to my code base. That's the problem with documentation, it's obsolete the moment it's written, likewise with code that isn't under test: it's immediately legacy code.
5 Careful with statistics: 20% of what pool? Wouldn't fifty percent be more relevant? But right, good software has to be intuitive and easy to use. the beauty is in simplicity.
6 That's right, the feature request has to come in written form from the target user community. Can't please everybody, goal is to please the largest majority and the stakeholders.
Garbage in, garbage out. Fully agree and the effectiveness of well trained manual testers can not be matched by automation at this point. But we'll get there.
Reply to this