No More Ugg Boots!
Do you remember when every celebrity under the sun was paying hundreds of dollars on Ugg boots? They were such a huge trend, and so very very ugly.
I am tired of using automation tools that could be Armani Suits but instead are Ugg boots or a $1200 handbag, or even $3000 jelly flip-flops. Can I just get a sturdy backpack? Please? Didn't so much need the high pricepoint in learning just to have my tool go out of fashion in less than a year.
This is what happens when test managers don't understand what they are asking for. A test automation tool IS a software project. Ignoring the users of it is really not a good idea. Designing it for 2% of those using it, also not a good idea.
I have trend whiplash so badly that the last automation tool I saw (it was a mandate-I had to go), the entire time we were looking at the code, my only thought was, "I've been poor. I'm good at it. I'm a great saver. If I have to choose between doing this all day and my job there is no choice. I could wait tables. Seriously, I'm kind of cute, I'd do ok." It is not a good sign if Automation has been rammed down your throat so much that you have a gag reflex whenever Automation is so much as mentioned. Umm pushy much? I only blog about this because I think this is industry wide and a huge problem. I'm extremely passionate about testing. Does my burn out on Automation mean I'm a "burned out tester" now? How could I not be burned out when my work on automated testing in the last 3 years has been much like digging a hole just to fill it back up again? I will say, however, there are 2 automated tools which I DO support, will use, and almost (squirm) like? Yes. I like them.
Companies who are reckless with their talented testers, their brain engaged testers, who can't appreciate and value people beyond their coding skills deserve to be brain drained by other companies. We've seen the crappy results from other companies in the industry and I hope we don't need to repeat their short sighted and failed experiements in order to fully understand that having all testing automated is a horrid goal. To have all software tested by "developers in test" means that anyone creative who is scientific on the side no longer has value.
I've got my hiking boots on. Keep those furry monstrocities off of my feet. I've got somewhere to be!
I am tired of using automation tools that could be Armani Suits but instead are Ugg boots or a $1200 handbag, or even $3000 jelly flip-flops. Can I just get a sturdy backpack? Please? Didn't so much need the high pricepoint in learning just to have my tool go out of fashion in less than a year.
This is what happens when test managers don't understand what they are asking for. A test automation tool IS a software project. Ignoring the users of it is really not a good idea. Designing it for 2% of those using it, also not a good idea.
I have trend whiplash so badly that the last automation tool I saw (it was a mandate-I had to go), the entire time we were looking at the code, my only thought was, "I've been poor. I'm good at it. I'm a great saver. If I have to choose between doing this all day and my job there is no choice. I could wait tables. Seriously, I'm kind of cute, I'd do ok." It is not a good sign if Automation has been rammed down your throat so much that you have a gag reflex whenever Automation is so much as mentioned. Umm pushy much? I only blog about this because I think this is industry wide and a huge problem. I'm extremely passionate about testing. Does my burn out on Automation mean I'm a "burned out tester" now? How could I not be burned out when my work on automated testing in the last 3 years has been much like digging a hole just to fill it back up again? I will say, however, there are 2 automated tools which I DO support, will use, and almost (squirm) like? Yes. I like them.
Companies who are reckless with their talented testers, their brain engaged testers, who can't appreciate and value people beyond their coding skills deserve to be brain drained by other companies. We've seen the crappy results from other companies in the industry and I hope we don't need to repeat their short sighted and failed experiements in order to fully understand that having all testing automated is a horrid goal. To have all software tested by "developers in test" means that anyone creative who is scientific on the side no longer has value.
I've got my hiking boots on. Keep those furry monstrocities off of my feet. I've got somewhere to be!
Test automation tools seem to be built to make test automation look easy for novice testers. This makes it difficult to create automation that goes beyond the most simplistic scripts for things that don't change.
The tools are NOT sold to those of us that treat automation as software development. They are sold to executives looking for ways to improve quality and cuts costs. And then once the tool is purchased, very few want to admit that they made a mistake and find ways to declare some kind of success -- even if it is costing more or doing less.
Ben
Reply to this
I've seen both extremes. Sometimes the tools are written so that only testers with full on programming skills can use them. Most of the good testers I know don't WANT to write code all day and don't enjoy it.
It must be very difficult to select an automated tool as an executive or even as a test manager, especially when you can't have your testers create and maintain tests with them.
I think a good process for selecting would require a group that represents the various skill levels and have them give the automated a tool a good beta run. The issue with this is most automated tools are "in development" themselves, or need to be customized, so you are now testing another product instead of the product you should be testing. Add to that how unfeasable it is to test what happens when you upgrade versions of the tool and/or the product under test, and it really is a guessing game as far as the time it will take to maintain the tests and troubleshoot false negatives that the automation will generate. I have no ideas regarding how to select an automation tool to work around this difficult problem.
Reply to this
I'm curious as to what automation tools you actually do like. I've used quite a few with differing amounts of success and hair pulling. One very popular tool actually misspelled end if as endif in its very lengthy and useless documentation. In my experience, the worst automation tools were always chosen by an executive and the best were chosen by "real" testers. My current favorites are free,
Reply to this
My favorite automation tools are two in house solutions. I don't have much choice over which tools I use because our requirements at my company are pretty steep. We need whatever the tool is to be cross-platform, and there are so many automation mandates you can't really select something else. One that I like is a system that manages any automation that you run. So basically, it will kick off any scripts, reimage any machines. The best thing about it is that it has self-healing/self-recovery and reporting. So, if it fails it logs the failure well, it will start over 3 times to try to get through the tests, and then if it still fails it will give you good details. It also has great reporting of the test results and it isn't limiting about what kind of automation you an run. Any script, any language, any operating system. The good thing about this tool is that it doesn't guess what you are doing, it just lets you run whatever you want and gives you information back, almost for free. In addition there is another tool that works with the previously mentioned solution that was designed to work through the UI, but it doesn't use screenshots. Instead, it uses automatically captured UI and then you write pseudo code (sort off) and it works entirely through the UI and menu system. It has most basic programming components (so, for example, you as the tester have to design all of the validation), but it isn't in any specific language. It is powerful because it uses the UI like a user would, so finds different bugs. It is annoying because sometimes it breaks because the UI changes and that is very irritating to maintain. I think that is a very common issue with UI tools. The other issue with this tool is that it doesn't work well across applications, which is a problem because I test 13 applications and the points of integration are my main focus. If I can't go between them, I can't get much done. Between the two, if it's faster to script it you can. If you need to go through the UI you can do that without capturing endless screenshots and comparing pixels. Not that I'm against comparing pixels, just that it has limited use and it's important to be able to set a tolerance of acceptable variance if you do. What are your favorite free tools? So you like scripting? For the record, I love testing. I absolutely hate writing code and barely tolerate scripting. I have interest in learning how things work. I like writing pseudo code just fine. I hate sitting down and writing the actual code. It seems like a terrible chore to me and whenever I have to do it for months I seriously contemplate quitting my job. Not just because I'm not great at it, but also because it is a waste of the talent that I do have. I'm a really good tester, and you could say I'm a "poor to moderate" coder, at best. I'm a happy and enthusiastic tester, and a miserable person when forced to write code all day. I don't wish to waste other people's time and money, and I want to be useful. If I'm sitting around struggling with writing code and someone else could be doing it four times as fast, I don't feel good about myself or my job.
Reply to this
Now, reading all that I took a drink from the mini fire hose.
James Bach speaks about "Tool jockeys".
I'll play the devils advocate and partially an opinion I have: "the objective is to automate all low level processes freeing up time for the resources to focus on more challenging problems." Obviously old jobs will disappear but new higher value added jobs will be created.
I'm passioned with coding and enjoy testing but am aware that there are certainly people and processes that are 10 times faster than me at this point in time. But with passion and dedication that level of performance will be reached. Fully agree with you that having the shiniest, most expensive tool is not the goal in itself. The goal is to ship code and find all bugs at the end of the day. So I shut up and code, test, work. Do it right the first time, double check everything and write tests. TDD the last Word.
Reply to this