Testy Redhead
http://blog.testyredhead.com
Testy Testy

Working with People-Disabilities

I'm thrilled to report that I've just graduated my 6 month training program! Business travel is something I love. Last Thursday, sitting in a hotel in San Francisco, the best meals you can get, and being in so much pain I was unable to leave the room or even eat dinner got me to the point where I have to admit it. I can't travel for business until my health has improved and my pain is under better control.

It's been really bad lately, and that got me to thinking, why would anyone hire a person with disabilities? Here are my reasons why you should, and how to work with such a person, although each person is different. First off, I am a person with a disability, not a disabled person. I have nerve damage in my right leg which causes significant pain. It is not something that is currently curable, so for right now, keeping the pain under control to live the most excellent life possible is the strategy. I believe it will be cured in our lifetime, but until then, it is simply something to live with.

-Hire a person WITH disability, not a disabled person. Know the difference. Does everything "happen to them"? Is it an all day pity party every day? Are they actually doing their part to get excellent care? Do they have a victim attitude? Does the health challenge they have define everything about who they are? If so, it's useless to try. Don't hire. The disability isn't physical.

-I really want to be working. My health problems are severe enough that it is quite possible I could fight to not work and get benefits. I don't want to. I want to be here. Not just to pay the bills. That's HUGE. Having a job helps give me a reason to keep a regular schedule. To get up every day. To keep my personality. So many people with the problems I have just get lost. They lose who they are. They take more and more narcotics and before you know it, they aren't themselves anymore. I don't want to be that person. I'll fight as hard as I can to avoid it.

-I'm more loyal than most employees because I need my health insurance. I can't function without it. Please understand this and never needlessly make me worry about the stability of my employment. If there is a stability problem, please give me as much notice as possible so that I can get this covered. A gap in health care really could knock me out of the workforce long term. It really could change the course of my life in a negative way. I'm a planner, so I have savings to make sure I'm reasonably able to take care of myself in this regard, but I can't be productive if threatened and it just isn't cool to do.

-Give me feedback about my performance just like you would anyone else. I don't want to know if I'm doing good ...<< MORE >>

New Features

I just got my first look at a must anticipated upcoming feature and I'm blown away in a good way and now I am so excited!

I wish that 3 other features I looked at and was disappointed in would be magically removed and every resource used given to this one feature that outshines the rest of them. Why waste engineering resources by spreading them out so much? When a test team is working really hard and their area of focus is just lame, there is nothing they can do that will make people want to use it. However, over here there is a feature with all the potential in the world if the quality is high enough not getting test resources. Software project planning is pretty complex, and I know there are lots of negotiations that take place.

I'd love to have a release where someone is tough and says, "We are cutting half of the planned features and taking twice as long for this project." Not financially viable initially, but what if the half of the features you are doing are groundbreaking and have world class quality?

We have this one feature that has disappointed me every time I've tested in for over 5 years now. In my entire testing career, I had one opportunity to ask a question of our CEO in a small setting. I asked him simply, "Have you ever used feature ______?" We don't have that same CEO anymore. His answer was clear, honest, and diplomatic. He had used it, recognized problems with it, and expressed hope for the future with it. Have you ever spoken with executives? They say more in a few sentences than I can convey with 2 pages of text. That kind of consolodation doesn't happen overnight. They are true artists at communication efficiency.

I'm not sure who in the company is so smart and charismatic that they are able to keep resourced with something so terrible to use that I was willing to use my one question to the CEO to find out if he'd ever used it, but whomever they are, I promise I want to stay off of their bad side, so I'll never specify which feature it was or bring it up to a CEO again. It's caused major resource drain and stability problems, while remaining unpopular with the vast majority of our users. As a serious advocate of usability, this particular feature is a villian in my world.

Anyhow, this new feature I can't talk about. Simple, elegant, fast, easy, AMAZING! What you can do with this is going to make my top 10 list.

I freaked out about chatting in 1993 and people thought I was crazy using the modem and talking to people with my computer. In 2000 I thought Livejournal was very cool and I started blogging. Digital photography was the coolest thing I'd ever seen in the early 1990's and people thought I was misdirected for taking so many photos and never printing them. When PDF format came out I ...<< MORE >>

Pure Loathing of Software

Today was a brilliant day! I had a wonderful day of work in every way. Dinner was fantastic. Even the cats behaved well.

So why did I just go on a 5 minute swearing tangent including writing hate mail in my mind?

Because it just took me 5 minutes to get a hyperlink into my Word 2007 document. Brilliantly, even the help menu was hidden to protect me from peril!! Hooray! Nothing on the context menu indicating that hyperlinks exist even though I have text selected and a hyperlink on my clipboard. I also need a psychic friend to paste special. Also, hyperlinks are now considered links and not "text formatting". Ok, that makes sense, but since when did Word try to make sense? They have gone so far protecting me from menu items that I can't do anything. It's sort of like the fine security we got with Vista. A great idea, but I'm so safe I can't do anything in this straightjacket. Little harm of me hurting myself (or getting any work done). Someone, please take the protective lid off of my spork?

I casually bring up my lack of joy with the new Word. The TWO users who used to work at Microsoft tell me they LOVE this interface. *blinks* *stares blankly more* I have no response for that. They have some very powerful kool-aid over there. Either that, or it's their policy of hiring only very very smart people. I will admit that both of these people are much more intelligent and patient than I am. They are quite amazing that way. That does however, make them sort of aliens in the American culture. I call this calm rationalization a problem of sick pants. I can't spell sycophant without a dictionary, so I call it sickpants. Too many smart people loving eachother. No one willing to remember that users may HATE your clever idea. This is why you need some testers willing to point out the naked regent about to stroll down the street even if it is career limiting. I want to shake the hand of the one MS tester (most likely a contractor who can't code), who brought up that in the future some crazy redhead who's been using word since the DOS days is going to go on a 5 minute tangent and hate this so much she will blog about it and scare other people. Maybe enough people love it that it was worth making me hostile. Somehow, I'm guessing on that particular product they have more upgrade users than newbies, but that's just a guess.

Well,  I suppose at least people notice it? And have a strong reaction? I just want to see the percentages. I'm thinking there are more users with the hatred that I have than the joy and love that they are showing, but perhaps I'm a small (and admittedly ranting) minority.

Here's some background. I stopped watching Lost after the first episode. I do not want to be toyed with. I am not entertained. ...<< MORE >>

Coming soon to Pacific NW Software Quality Conference-Testy Redhead!

My abstract was selected for pcsqc 2008 which happens in Portland from October 13th-October 15th! The theme is collaboration.

I am just over the moon with excitement to be speaking on my favorite topic at a testing convention for the first time. I have lots of work to do before the conference, but I hope that some of you will come out to the conference and attend. Come talk testing with me.

It could have been the Grand Canyon, the training I've been going to, or all of the wonderful people I work with who have been supporting me all year in trying new things, but I'm so delighted to have this opportunity. I feel like I did the day I got my first job in tech support. I have the feeling that the hardest part is getting that once chance, and now I've been given one chance. That's all I really need is the opportunity, and where it goes from there depends on my hard work, dedication, passion, and talent. The part that I can't control (being given a chance), due to luck, due to persistance, due to kindness from other people, has fallen in to place. I'm overwhelmed with gratitude. It may seem like it isn't a big deal to those of you who speak at conferences all of the time, have multiple doctorates, or are published authors. To me it is huge!

One of my goals is to add my voice, my perspective, and my passion for software to the chorus that is constantly changing in the industry. Just as we need people who are brilliant with tools, who are fantastic with numbers, and who are great with theory, we also need people who have intimate knowledge of users, who care about the soul of software, and who never stop fighting for an improved experience when you sit down at your computer to get a task accomplished. I may never be the person who has the most impressive metrics, or who wrote the best testing tool, but I can be the person who contributes some new ideas to protect the less tangible aspects of software that make it memorable and amazing to use, and that is something worth working to contribute.

I've also learned something else comforting. Not being a manager doesn't mean you aren't a leader. Also, being a good follower doesn't mean you aren't a leader. If I believe in the person, I love to follow other people. There isn't any shame in being a good team member. I don't have to lead everything I participate in in order to be happy. In fact, all of the leaders I like and respect are good at following too. I want to remember that as I try new things. Listening and taking feedback is how you learn. Following the right person and ideas is a sign of good leadership and followership, not a sign of weakness or lack of initiative.

...<< MORE >>

Was it fun?

My company has something about the culture for years that has been really special. Real substance behind the philanthropy efforts, a very human feel, and a sense that fun belongs everywhere, not just in a play hard, then work hard (mostly work hard), but never combine the two sort of way.

I had a testing assignment this week and last night was FUN! I worked really hard. I found a great problem, and I had so much fun communicating it. It made my whole night and I was late to dinner and I didn't even care. That's the sort of thing I like to work on.

I want to be the person who makes sure as we move to every kind of possible metric to measure a person and a task that fun doesn't go away. Just as the soul of software can be at risk when you make changes, so can the soul of your entire workforce, and with it, what made it special and successful in the first place. If you hire just one type of person, you lose every voice of dissent, and with it, often the voice of reason. It is generally bad to mention anything negative to those above you in the food chain, but sometimes they need to hear it. Otherwise they have a year like Britney Spears and everything's gone down the tubes and someone else has to go in and do a very expensive cleanup. When there is a cleanup like that, it's never announced "someone Royally made some poor choices and apologized and now we are going to fix it". Instead we are just "going in a new direction" and "improving our process". I have a way with words so long as I believe them. My friend will sometimes ask me to rephrase what she says before she has fights with her boyfriend so that they go over more favorably. However, as a matter of integrity, I think if you make a huge error, you admit it, apologize, and explain how you are going to move forward.

I recently found out that pretty much you have to be into code or have a CS degree and significant coding chops to be considered for a QE Manager job. I think that is industry standard. Why does no one ask you to give them a demo of the software they are supposed to be managing, yet they must know the code? Do they even know software has a soul? Do they know that their people do, or are they just good at making charts to show how the metrics are doing?

So instead of conversation 1, you have conversation 2.

Conversation 1
Person 1: "We should make this feature. It's really cool!"
Person 2: "Yeah, but it isn't useful and really sucks when you take out all of the stuff you can't fit into THIS version."

Conversation 2
Person 1: "We should make this feature. It's really cool!"
Person 2: "Yes. I've investigated your coding logic and rate it as "pass". More input ...<< MORE >>

Large Dirt Hole a.k.a. Grand Canyon

As promised, I'm reporting out on the Large Dirt Hole from the perspective of the upper South Rim.

I did an exploratory pass of the entire South Rim by doing a quick sanity check on the edge and going in deeper and random points up to the fence at viewpoints as well as looking into binoculars for a closer view. We did climb the tower at Desert View on the far east end for an overview. At first the functionality was overwhelming!

The depth could not be explained in photos or numbers or written comparisons, so I understand why people told me I had to just experience it for myself.

Luckily I had someone there with me who was unwilling to accept the current boundaries. Safety was really not a concern. After watching him go on one life risking stroll after another, I followed him out one area. Do you see in the photo below the portion in the foreground that juts out into the canyon?

So, on the paper? I've decided to go as far as I can. This means it will follow MY style. It will be personal. It will have jokes. It will have more heart and fewer numbers than most other technical papers. In fact, it won't be a technical paper. It will be an article of ideas. If that is too far over the edge, I'm lucky enough to have backup. ...<< MORE >>

Brilliance Formed?

Look for a few moments at "Pillars of Creation", one of NASA's most famous images of the Eagle Nebula. No one is going to bother to read my paper if I call it the "big dirt hole". Perhaps some marketing is needed to get people to come and see it. If it is good, word of mouth will help me. If it is bad, I can get feedback and try again. Why would this be terrifying? Only because when I finish it, I face the chance of rejection. Also because I overthink everything I do. How many people out there just do it? Just declare "Grand Canyon" when they wonder somewhere deep down, "Is it possibly big dirt hole?" ...<< MORE >>

Cold Hard Logic

I had the pleasure of attending a User Group last night of a product near to my heart. I worked on the first version of it and always was a huge supporter.

During the event the speaker and teacher kept referring to "cold hard logic" to explain that while he could understand WHY things worked they way they did, or could explain it away, that he still couldn't do what he wanted. It made sense logically, but it wasn't intuitive.

This is why listening to your testers is important. This is why testing for the user experience is important. No one cares if your decision makes sense. They care if they can do what they want to easily. They care if they are disappointed and have to use workarounds, or if you provide what they need and more.

Obviously, you can't make all of the users happy all of the time, but if your decisions have to be explained away by "cold hard logic", obviously you've made some regrettable choices along the way. Yes, there are reasons, but the facts must be faced that cold hard logic isn't what is important in life.

Love of a thing or love of software for most of us is how it connects us to people. This is what Web 2.0 is about. This is what viral video is about. Cold hard logic just won't cut it anymore.

The next time I hear a complex explanation that makes sense about why a feature is usable, I'm going to think of it as an excuse. Yes, you make sense, but it still doesn't create a passable user experience, let alone a good one. Not to say that we should be uncooperative or unconstructive in our language or dealings with others, especially well intended developers. I'm just saying, it is our job as testers to represent the customer, even when it doesn't make logical sense. ...<< MORE >>

Protecting Awesome User Experiences

I had the honor to visit an office I'd never been to the last few days. I met some people who not
only want to work with my team, but are willing to train and mentor us in something.
They also understand that web creations do not maintain themselves and still remain relevant. The important data is the data added by humans. There is information everywhere, but what do we care about? We care about things that make us feel and think.

I gave a short presentation in a meeting about collaborative testing. I showed one way we represent an overview of a path through multiple applications in complex scenarios. We showed areas where there were some problems, areas where there were serious problems, but we could work around it, and areas where we were totally blocked from completing a task during the collaborative testing. I then told them that in representing it this way, we just weren't good enough. We aren't representing the experience. In order to do that, we must also show what things were delightful. What should we "not mess with"? Just as software can need improvement, I believe it also needs wildlife preserves. If it is something so delightful that it is the only reason many users enjoy it, please, just leave it alone unless that changes. It is fine to add to that. Maybe new options, but do NOT change the default when it is brilliant. The team I met with contributed many ideas about this and they took my overview to play with it. They had never even seen the overview before and thought it was great even without the missing part, but they liked the idea of seeing both.

I'm not sure why we have been ignoring the opportunity to provide this data. It is a huge part of overall quality.  Testers, managers and developers assume that if it wasn't fantastic, bugs would be written against it. I've tested some very stable code for a feature that sucked so much to use that I met a loyal customer who was using a script to work around using the feature entirely. Software users ultimately will provide some of this data, as will the market and research. However, as a tester, why not share this critical information? I think it is because it is subjective, and so often rejected as information of value. Collaboratively, however, if we agree on what the top 5 delightful things about the software are when we use it, our odds of predicting user reaction improve dramatically.

I know I harp on this collaborative testing, as it is my passion, but we are getting better at finding the dull subjective data. Does it work as designed? Does it do more than it was designed to do? These questions can be answered with "Yes" or "No". These are things that can be determined with automation.

How do I feel about this software? Is it just fine if this one feature is mediocre? What is the worst thing about using it? What things ...<< MORE >>

First Technical Paper

I'm in the process of putting together my first technical paper. In general it is about how you can use collaborative testing to test the user experience. I'm not sure what is going to happen with the paper, but I hope that once it is approved, I can share it here.

If you have any tips for me on writing a compelling abstract, an original and focused technical paper, and/or presenting based on a technical paper, this is my first time. I could use some guidance from the old pros.

In other news, I participated in my first every Pinewood Car Derby. My auto, based on a lipgloss theme, did well enough to make the finals, but did not win a medal. However, I stayed on the track and looked good doing it. For a first attempt, I'm quite proud. I know where to improve next time.

...<< MORE >>

Overcoming Fear

I've been working a great deal on professional development. I've never grown so much in so short a time. I'm looking for tips on how to raise my confidence and how to believe in myself more.

This week I submitted 2 ideas to an internal innovation website contest. I didn't win, but for a first try, both of my ideas were slightly above average in terms of votes for all ideas submitted company wide. That may seem like a loss, but if you consider how smart the people I work with are, I don't think so. In the past I would never have submitted them at all, just making excuses that my ideas wouldn't be taken seriously. You can't make the top 3 ideas if you never try, and you have to start somewhere. Mediocrity is a fine place to start because I'm not staying there. This is just my first try. Besides, average here is way above average in the world at large. It takes experience, influence, and popularity to get traction on an idea.

I'm finding out that the main thing holding me back from my dream job isn't skill or experience. It's fear. I'm afraid that I don't have experience. I'm afraid to fail.

I signed a contract last week for my special training that I'd be doing 3 things by March. One of those is applying for a job that I really want that is offered internally at my company. No matter what the results are, I need to toss my hat in the ring. When I signed that contract I tried to think of every reason in the world I shouldn't go for it. I explained to the mentoring group why it wouldn't work and why I was conflicted about doing it. I was shaking while signing that I would do it. The time for self-defeating internal talk is over. If I'm held back, it shouldn't be by me. No only means no for now. What's the worse that can happen? I hope the worse that can happen is they don't interview me and at least tell me what to work on to be qualified for that position if they don't think I am. The best that can happen is they see my passion, potential, and how fast I'm learning and give me a chance to grow in to that role. The medium thing that could happen (neither great nor terrible) is that they interview me so I can get practice interviewing, and they give me feedback. For me it isn't so much that I have to have the "job right now", it's that I want to get there and know I'm moving towards it. Besides, it's reporting to the one person I most want to work for, and opportunities don't come up that often. I feel like a crazy person to say the truth, which is, "I really want to be on your team. You want me because you don't have anyone who will work harder to make you not just look good, but BE good than I ...<< MORE >>

Fire Fighting

Things that seem strange and negative to people are considered pretty normal to me at this stage in my testing career.

Things like reorgs. We had a really good one a few weeks back. Didn't lose any people, just some people reassigned. I have a new boss. It's my new boss #8. So far, I really like him. It's kind of a setback to have to earn the trust of yet another new boss, but I consider it normal and look forward to demonstrating my talent and loyalty as quickly as I possibly can. Each new boss is another opportunity to learn something.

Things like testing that wasn't planned or agreed on coming up and having to make a decision. This "fire fighting" isn't strategic at all. However, I think it is important to efficiently do what needs to be done. I PLAN to be flexible. Some people consider it a weakness. I don't agree. If I can do something to help us ship something on time, I'm easy in. I've seen different strategies when this sort of surprise comes up. Some leads will take a hard stance and refuse any work they didn't agree to in an effort to "protect their testers". I don't agree with that strategy. Creating a false reality for your team isn't true protection. For real security, you have to be mission critical with needed skills and do something that has recognized high value. On the other hand, if you are nothing but a dung heap that people pile their unneeded work on, you can't do anything of high value. All you are is reactive.

I know that it is human to like to be rewarded by money. Money is a show of how valued your work is. While money always matters when it comes to work, mainly, I want something else.  If money were no object, I still would work. Most likely I'd work 4 days a week and not 5, but I wouldn't quit working if I had endless money. I want to feel like I matter to people. I want to feel trusted, appreciated, and special. I want someone to believe in me and give me opportunity. I would take a cut in pay to have the chance to do more of the work I love doing.

I've had some people tell me that my style is "Ready, Fire, Aim". Those people don't understand what my style really is. It is "Ready, Fire, Aim, Fire, Re-aim, Fire, Aim, Fire, Aim, Fire, Get machine gun after research, fire and continue until the objective happens." They don't take into account that I have more tenacity than they anticipate. My lack of patience for inaction is not accompanied by a lack of follow through as it usually is with people who have my personality type. I'll do what it takes to not disappoint, even if it is harmful to me. For that reason, having someone who cares about ME as a human is vital for me to do well. ...<< MORE >>

User Focus

How important is user focus? Which users and who determines that? Does a user become unimportant because they are considered a "sure thing", much like after the honeymoon? New users become the focus, so we can take our core user base for granted? Or do we only care about the loudest customers? Do we only like the nicest customers, or those who come to us willingly in a format that works better for us, like forums? Is the only user information that matters what comes through a scientific process, such as usability studies, or are survey results sufficient to stand alone?

Are money, comparative sales statistics, and support cost comparisons a true measure of focus on the user?

How do we sift through that user data?

It matters to me very much how users feel. That is not scientific. Do you love this software?

At my company, I used to feel that user advocacy was a valued part of testing. Now I feel the trends are shifting. If the user you are advocating for isn't the main focus of the target market segment they would like to capture, they are of little importance. Even in Girl Scouts we learned "Make new friends, but keep the old. One is silver and the other's gold." It isn't stubborn or holding on to the past to stand up for the CORE users who built your company and paid for everything you are developing now, nor is it betrayal to keep up with industry trends and note what people need to do right now and in the future. I think it is critical that all users are considered.

Have you ever gone to a college class where they are teaching your software and talked to the people there? I have. I helped at a class held in New York for a product I worked on on a customer visit. I also was a student. How hard is it to learn to use this stuff? Are they having fun? After that class we went to a shop that had been using it forever. There were some things still keeping them from having fun even though they knew so much about the product. More than we did in some areas, especially on how it interacted with other software.

This battle being waged for the developers has gone too far. Most users of software are not all scripting and coding (except, of course, for Visual Studio and other products which coding is their only purpose). How as testers can we stop corporations from throwing their loyal users out the window to make more room for untapped markets?

I'm still looking for the answer. I'm hoping that some percentages would help. What percent of our users are doing ________ vs. what percent are doing _________? How much do we invest in maintaining the users we already have vs. getting new ones?

Some growth is good growth, even if done fast. Do you know what you call fast growth that eats away at your health and eventually kills you? You call that ...<< MORE >>

Causing a 4 Car Pileup

Forcing an Error Pileup-A Fun Tester Trick
Long time, no blog. The holidays were brilliant, but now I'm back in action, being as testy as ever, so I want to share with you one of my favorite ways to test for errors.

Automation is a great way to "force errors", however, almost always it is one at a time. Causing error message stacking is a great way to test for race conditions, find crashes, and get into those states where you can't access a modal dialog, so you are stuck until you force quit.

Here is my big secret: I have no patience. I will not wait for an error to appear before moving on, nor will your users. They are clicky clicky mouse movers in many causes. They will refresh like mad, move on while one process is running.

So, my testing tip of the day is, don't wait for the program to finish. If you know of error conditions, in addition to trying to cause the same one to fire two at a time, try mixing them up today. The first time you find that state where you have an error dialog without focus and you have to force quit, come comment in my blog and it will make my day.

Automation and Those Darn Robots
Hope that the '08 is off to a good start. My resolution this year is to work "with those crazy robots".

Industry wide, it is time for the investment in automation to stand proud and be trusted. For those humans still testing to compliment the robots and let them be judged on their own merits. Those robots with value will prove what they are worth. Those with lower value must be allowed to come to light to improve. Improvement and not just change. I love that idea. I mean, do I really ever want to manually smoketest an installer in every language on 4 platforms again? Of course not. What a waste of testing time, experience and money. If any monkey could do it, why isn't it automated? If we paid to automate it and are still testing it manually, we are stealing our own money by not trusting our automation.  I spent part of today trying to convince someone on my team of this. They looked a bit nauseated and doubtful. I hope they will be afraid and do it anyways. I really really hope so. It's past time.

I had a discussion today with an ex-tester who is now a tech writter. I was explaining to him that what it really takes to get a testing job right now is a whole lot of coding skills, well, at least the ability to talk like you have lots of coding skills. Notice, I didn't say you need talent, experience making anything useful, creativity, or good testing methodology. You don't even have to be great at working with a team. You don't need to have any talent for finding bugs. You just have to fit the checklist of one of the Software Developer in ...<< MORE >>

Change Management

I think the main reason I ended up being a tester is because I'm more curious than average. You could say that I'm nosy, and it wouldn't be too far from the truth, but beyond nosy, I'm interested.

Last week in a meeting I heard someone refer to change management, but not in the way I understand it. I understand it as a way to track versions or a method to control code changes in a system. As I do sometimes, I wrote myself a note to find out what in the heck this change management stuff is about in the context it was mentioned.

I've attended a day long seminar before on "Introducing Change" given by a woman who was a software developer and had a doctorate in Psychology. I was approaching the information presented as a way to get my testing ideas to see the light of day.

Reading more about it, I now realize that I have been naturally doing some of this on my teams over the years. If I really believe in the ideas, regardless of who's ideas they are, I get behind them and really move them forward. I do have some of my own ideas, but over 7 years I've done much more work on bringing momentum to good ideas that other people have.

I have a double edged sword of strength/weakness in this area. The strength is, if I really believe a change is going to work, and is for the better, I'm all in. I'm the first one in line to be an early adopter. I'm not change resistant and I will use my enthusiasm and charm to get other people either excited about it, or to feel less threatened by it. This is easy for me. I am one of those people who likes change. My current job is never the same from day to day. I work on many things. This is fun, but I think it would give some people whiplash.

The downside is, if I think that the change will not work or I don't believe in it or trust you, I'm very effective at creating an angry mob. Not just a verbal group of people disagreeing, but the underground resistance as well. I don't mean to do this, and I think I'm just sharing my opinion. I give little thought to how my venting might influence other people.  I am not part of the underground resistance. I can't fake it. If I don't think an idea is good, silence or "agreeing to comply even though I don't agree" is the best I can do. I can't pretend to support the idea and lie about it. Many people will appear to do as the Romans do, but actually do what they believe is best as soon as the Romans are out of the room. Others will leave Rome never to return.

So, there are many many suggestions on how to manage change. What can I do? I can help with getting people to understand why ...<< MORE >>

Testing Idea

This morning in the shower I was thinking about collaborative testing (Yes, this is a normal thing for me). I was thinking about some of the things humans are good at doing together than machines are not. One of those things is prioritizing and eliminating unimportant tests. Those decisions take subjective information.

It is very common in testing to target tests to areas of change either code changes, or feature/product changes (what is new or altered). We use this practice on a daily basis. It is more difficult to use across multiple products and/or silos.

My idea today was what if I could simplify? Could I make a visual respresentation of the areas of change over multiple products? What information would have to go in to make it useful over time? In my context, it would be impossible at this time to compile all of that data on a build by build basis, but what if I could compile it at milestones, or even just for the release in general? Would it be useful if not precise? I think it would be a good guide as long as we knew it was just a general pointer (that way), and not accurate turn by turn directions.

I even started thinking about how it would be possible to automate for updated build to build information, but of course, I reminded myself that automation isn't my talent or interest, so while it may be possible, I'm not the right girl for that job. However, I do love maps. I'm one of those females who can not only read and fold a map, but who can navigate the roads of New Zealand, Italy, or any country we might be travelling. I have a compass on my keyring (shout out to Rick Steves). Each time I travel with other people, I make sure I have the map and say, "I'm the navigator!" Seriously. My family even refers to me as "The navigator" sometimes. When testing with other people, why are we not using more overall maps? What is this idea? Map based testing? Ha!

Have you used such a method before or heard of a team using this? I talked to one person about it who wasn't excited, but I think it is likely I wasn't expressing the idea in any detail and the discussion turned to feasability of information collection across multiple teams (human issues).

I'm thinking of the idea more from a scientific perspective. It's more of a "what if I had ___ thing? Would it help me?" Can I get ________ thing is a different discussion all together.
...<< MORE >>

The Ever Popular Pairwise Testing

Each time I review the "testing basics" to make sure I haven't forgotten anything (I do this each year or so, I recommend a refresher for anyone), the topic of Pairwise testing comes up again.

In trying to get into more depth on the topic, I came across an article from 2004 by James Bach. What I like about this article is the research he did.

Best practices only save time and money and improve efficiency if applied correctly and with context.


The typical pairwise testing story goes like this:


1) Pairwise testing protects against pairwise bugs


2) while dramatically reducing the number tests to perform,


3) which is especially cool because pairwise bugs represent the majority of combinatoric


bugs,


4) and such bugs are a lot more likely to happen than ones that only happen with more


variables.


5) Plus, you no longer need to create these tests by hand.


Critical thinking and empirical analysis requires us to change the story:


1) Pairwise testing might find some pairwise bugs


2) while dramatically reducing the number tests to perform, compared to testing all


combinations, but not necessarily compared to testing just the combinations that


matter.


3) which is especially cool because pairwise bugs might represent the majority of


combinatoric bugs, or might not, depending on the actual dependencies among


variables in the product.


4) and some such bugs are more likely to happen than ones that only happen with more


variables, or less likely to happen, because user inputs are not randomly distributed.


5) Plus, you no longer need to create these tests by hand, except for the work of analyzing


the product, selecting variables and values, actually configuring and performing the


test, and analyzing the results.

...<< MORE >>

Interviewing-What makes a good QE Lead?

Today I came across some questions I'd written down nearly 2 years ago. I was looking to hire someone to replace me as QE Lead . Here are the questions I came up with.


After reading the questions, I realized I only wanted to know 2 things.


1. Is this person a good, creative tester with interest and experience in this area?


2. Can this person work well enough with different types of people to be a lead?


I guess those are the top things I consider "must have" items for a QE Lead. A pretty short list. The round of interviews resulted in a good hire. The person has worked out really well in that role. They had much more than my bare bones hope that they could "do the job". I'm glad it turned out, but I've been thinking about how to incorporate questions to find out if I can get what is "ideal" for the job, not just functional.


The most hilarious thing about the person we hired is that on paper, they had the resume that annoyed me the most of any resume I'd ever seen. It did stop slightly short of saying they invented  the entire Internet, but it came across as over the top. I was shocked to meet a humble and intelligent person who was anything but a bragger in person. It just goes to show you that you can't tell everything from the written word. Once I got to know the person better and they'd been on the job while, we discussed what my impression of the resume was and why I interpretted it that way. We both thought it was pretty funny and laughed about it.


So, I ask you, what do you think are important qualities for a QE Lead? What questions would you ask to find out if a candidate had those qualities?


Questions I Had Prepared



  1. Have you ever used (product under test) before? Describe what it does.

  2. Describe your experience with (Technology area) so far. What do you enjoy working on the most that is (technology) related? Why do you want to work on (Specific Technology)?

  3. What is the most difficult bug you've ever isolated?

  4. What is your greatest strength/weakness where testing is concerned?

  5. Do you believe that natural talent plays a role in the performance of testers? What role does it play and how would you maximize it?

  6. How would you go about isolating the following bug? Note: This is an actual fixed bug from the database. Xml file crashes application if imported with "(Edited for privacy)" option is selected. (note: Removing certain tags causes the problem.) How would you verify the fix?

  7. Describe a situation where you worked with a difficult developer.

  8. What kind of a boss works best for you and why?

  9. Describe a situation where the analysis that you performed was incorrect.

  10. If someone on your team was a solid contributor, but not developing the technical depth ...<< MORE >>

Amazing Blog!

This blog entry on software quality is one of the most amazing I've read recently.

Whenever I feel alone, or like I simply can not win the losing battle in defense of creative human testing, something like this comes along to provide me with hope. Yes! Smart people who see the value of testing beyond what you can programmatically verify.
...<< MORE >>

Performance in General

I got to thinking about performance in general. Who's responsibility is it?

1. User Research-Should get some data on what performance users expect in which context.
2. Program Managers-Should define performance with comparisons per system (Faster than, as fast as, no more than % slower than _____________ on the same machine.)
3. Developers should write the code with this in mind and explain to testers when the code is optimized and ready to be tested. This is a performance handoff.
4. Testers then test against these requirements and pre-defined goals. They try different situations, stress tests, repeat scenarios, memory intensive tasks, different OS settings and hardware, long runs, benchmark tests. These tests are some of the best automation candidates.
5. Testers look at perceived performance (Progress bars, number of steps to do something, screen feedback, cursors, ect) from a customer perspective.

So, in general, software performance has split responsibility.

Work performance does not have shared responsibility as much. If there is something getting in the way of you giving your best work performance and you say nothing and do nothing about it, that makes you responsible. It isn't the fault of your co-workers, your boss, the lack of requirements, the lack of a spec, the failed delivery of testable code. It is your fault. As a tester, it is your job to make sure that you can do the best possible testing. Obstacles will come up. Find a way around them. If you can't, directly address them.

I'm very lucky to have had some feedback last week from someone I really respect. There are people I like, and people I respect, and sometimes both, as in this case. As far as work is concerned, I'd rather work with someone I respect but don't like than someone I like and don't respect.  In my personal life, I need someone I like who likes me more than I need to respect everything about them. At work, there was a situation I was just tolerating. Basically, I was waiting for it to improve or go away. I've now had a revelation thanks to a gentle reminder. I'm no victim. The only person who can make sure that my performance is something I am proud of is me. The only criteria for me to be proud of my work is to know that I did the best I possibly could given the circumstances, and that "my best" continues to improve in a tangible way as I learn. It's been a rough few months for me on the job. This is the first time ever where I know I made some mistakes and could have done better. I know I should have been brave enough to admit when some things were a total waste of my time. Knowing when to call it quits on working on something is a tough lesson to learn. When I do things that my heart just isn't in, I can't be more than mediocre, and I'm a very open person. If I disagree, it is written all over my face. ...<< MORE >>

Hand Crafted

This holiday season has been unusual for me. Having just purchased our first home, we get the chance to host Christmas dinner! While I'm excited about it, it is expensive to make dinner for 15 people while putting up out of town guests, and we didn't have any decorations for the outside of our home, so that added expense as well. Then our cat got very ill and almost died, requiring many emergency vet visits and both cars unexpectedly needed repair. As a result, I didn't have as much budget as I would like for gifts this year, so had to think of a strategy.

I heard about hand made gifts, and love to give unique presents. I'd assumed that I could not save any money by getting hand crafted items. As a hobby, I design and make knitwear, so I know how expensive and time consuming it is to make things, especially here in the US where the supply costs are so high. I stumbled across Etsy.com and found some absolutely amazing gifts. Not only were they unique, but they were thoughtful, personal, and some created by true artists.Yes, some items were really costly, but I ended up spending an average of $23 each on many amazing gifts I gave the first one to a friend last night and she was blown away! I know she will remember and treasure it so much more than if I'd given her a $25 gift card to someplace.

So, what does this have to do with testing? Well, hand crafted testing may not cost as much as you think. There are situations where you just need the cheapest thing, or where you need some testing that isn't practical to make by hand. For example, an iPod. However, it is more meaningful and useful if it has a hand crafted cover and a song list you created for that person. How the person getting it will feel is different, because it really is customized for them from someone who cares.

How the people using your software feel about using it is vital. That is where your reputation as a software company comes from. It can not be trusted to robots. It should not be trusted to someone who doesn't take personal pride in the quality of it. I will always prefer an naturally talented tester who sincerely cares over a burned out person with the best education and "skills list" than you can imagine. It doesn't matter if you can write circles of code around another guy, have a doctorate degree, are a published author, a well known expert, or have dozens of patents if you have no personal investment in the results. You can't fake caring, and it is required to do an excellent job. You can do a mediocre job without it, but I've never seen someone do a great job without passion.

My point is, there is still a place for artistic hand crafted testing. Yes, you can ...<< MORE >>

If you read just one thing on test automation before trying it,..(or trying it again after poor results)

Test Automation Snake Oil by James Bach in 1999. Of particular interest to me are the "Reckless Assumptions" and the Sensible Approach to Test Automation section. In fact, if you could only read 1/2 a page, the Sensible Approach section alone would save time and money.

I've read this article a few times. Sadly I then got feedback on my Automation Plan telling me to "put in (this) tool". I tried to explain why it doesn't help us or suit our needs, as well as why it wasn't realistic for our entire test team to become full Java developers in one month, no matter how intense the training is. Positive thinking is good, but in testing, realistic thinking is more important. Well, I try to balance it out with this link. This is the link I wish was in my test automation plan instead of the link to the automation tool which made me think seriously about becoming a waitress.

For those of you not familiar with test automation I'd like to also provide a few fundamental things that most people don't seem to understand who have never tried it before.

"Scripting" means that you are using a scripting language such as Javascript, VBscript, AppleScript, Perl, ect. Generally it does not need to be compiled and runs within a scripting environment of some sort. While it does work outside of one application, the scripting DOM is vastly different between applications. You can learn the core concepts the basic syntax and those transfer, but just because you know how to script in one context doesn't mean you are a scripting expert in another context.

"Automation Tool" is a very generic and broad term. Does the tool schedule and run automation? Does it allow you to test through the GUI? Does it allow you to write unit tests? Does it report back results? If so, how? Can you set it to a tolerance to avoid unneeded false negatives on cases? Does it just track your automation? How easy is it to use? My point is, don't say "Automation" and think that everything that falls into the category is the same.

"Scripting" is not equal to "Test Automation". If you are using scripting to run a step by step test case, even that isn't an automated test case unless you've created proper validation and reporting. Sure, you're scripting it to DO something, but if you didn't validate properly you didn't test anything. I've seen test teams with "80% Automation Coverage" but NO Test Automation. Don't just DO something, TEST something. They are different entirely. If you didn't validate, all you know is that the machine didn't explode when you executed that script. Any number of other things could go wrong, including it not working at all, and you still wouldn't know.

"Automated" does not mean that human tests no longer need to be run.

I took a risk management class last week and we followed the "cause chain" up to ...<< MORE >>

Globalization Eye Opener

I took a special class today at work that was specifically about testing for internationalization and "localizabilty", which means you are "faux localizing" your resources and testing before your translation is completed.

I've now been testing for almost 8 years! I tried to count, and I can safely say that this is at least my tenth product cycle. I've tested in most every language (Most Latin Based languages, the Americas, Japanese, Chinese, Korean, ect). I've never tested in Middle Eastern languages or Cyrillic (Russian, Eastern Europe). 

Shockingly, having never tested on many features where sorting was of great significance, there is an entire area of testing that is brand new to me. Searching, GUI changes, Truncation, consistency, encodings,..these are all old news. Here is the biggest thing I learned all year: Not everyone sorts alphabetically in ways that make any sense to me. I've never even considered how in the heck to sort Asian characters.

It just makes my week to learn something so key, that I've been missing from my testing knowledge all of this time without ever knowing it! It wasn't just a lightbulb moment for me, it was one of minor shame that I could miss getting such an important and obvious concept. I knew this was important for search, for metadata, and have written great bugs about those items. Right now I can't wait to get in there and use my new information to break some sorting. I'm going to be looking like mad for ways that sorting is possible.

It has been so long that I forgot, many years ago I created a hilariously geeky set of test files which tested input/output of every possible UTF-16 character possible both in the file name and in XML content and allowable tag names each individually named so that it was very quick to identify problems. I think input/output is a wonderful area to automate, and I had to share that I have had good results when automating mundane tasks such as verifying correct encoding and other functionality tests. Automation is not so good at finding bugs, but it is very good at doing some functional testing, so long as the expected result is pretty well defined.

In other thoughts, MS Arial Unicode is the biggest test font I've found for doing unicode testing yet, but is there anything as large and complete in number of characters for the Mac platform specifically? Help me oh fellow lovers of Steve.
...<< MORE >>

How to create a really terrible internal Test Tool-6 Ways to cause it, 6 ways to help avoid it.

Steps to Create a Bad Internal Test Tool
1. Have people who are not testers define the requirements.

2. Have someone who doesn't understand the context of the problem, hopefully someone who doesn't speak the same native language, make a very liberal interpretation of those requirements and "let the technology guide them" to the right solution.

3. Do not listen to the main users of the software, assure them it is a staged development model that will improve as time goes on.

4. Keep implementing features on top of the features that are already impossible to use.

5. If people tell you it is difficult to use, show them how wrong they are and write down the exact 87 steps it takes to do one simple thing. Explain to them that they must not be using it correctly.

6. Have the engineer who implemented the code write all of the documentation and training, and then if someone points out that it has no context or conceptual data and doesn't make sense, argue with them and tell them, "Just follow the steps."


Steps to Help Avoid a Bad Internal Test Tool
1. Have very early pilot users who are testers, test leads, test managers, and even higher up stakeholders, but not just at one level.

2. Create a very early prototype that a person could use on a small scale.

3. Take usability bugs SERIOUSLY. This will block adoption of your tool. It isn't a minor UI issue if people can't or won't use your tool.

4. Have your power users create and review your documentation if they are willing to. Even ask them to create short training workshops and do demos for you. They will be better at explaining to people how to use it than you will because they ARE using it, not writing it.

5. If more than 20% of the people you've asked to look at it tell you that it is too difficult to use, do NOT start writing more features until you fix the useless ones you've created already.

6. Don't start adding features just because one person asks for them. It should be based on how many people need it, not on the title and visibility of the person who asked for it.

While I'm on a rant, some tester had the audacity to say, "You can never have too much automation." What? My head nearly exploded. You can never have too much well designed, effective, cost saving, reasonably maintainable automation which start to finish saves you time over running the tests manually. Sadly, there isn't enough automation currently meeting those criteria, and far too few people actually measuring. Automation doesn't equal GOOD. It just is a buzz word. Garbage in, garbage out.

Thanks for letting me vent, oh internet.
...<< MORE >>

New Strengths or Flawed Testing?

In taking a training course I got another copy of Now Discover your Strengths 1.0. This is the third book I now own. I decided to retake the test. Only 2 of my stregths remained the same over the course of this year.


Here are the new strengthfinder results:
Ideation
People strong in the Ideation theme are fascinated by ideas. They are able to find connections between seemingly disparate phenomena. ActivatorPeople strong in the Activator theme can make things happen by turning thoughts into action. They are often impatient.

Woo
People strong in the Woo theme love the challenge of meeting new people and winning them over. They derive satisfaction from breaking the ice and making a connection with another person.

Communication
People strong in the Communication theme generally find it easy to put their thoughts into words. They are good conversationalists and presenters.

Arranger
People strong in the Arranger theme can organize, but they also have a flexibility that complements this ability. They like to figure out how all of the pieces and resources can be arranged for maximum productivity.


As previously blogged, last year they were:
Connectedness
People strong in the Connectedness theme have faith in the links between all things. They believe there are few coincidences and that almost every event has a reason.


Ideation
People strong in the Ideation theme are fascinated by ideas. They are able to find connections between seemingly disparate phenomena.


Input
People strong in the Input theme have a craving to know more. Often they like to collect and archive all kinds of information.


Activator
People strong in the Activator theme can make things happen by turning thoughts into action. They are often impatient.


Empathy
People strong in the Empathy theme can sense the feelings of other people by imagining themselves in others' lives or others' situations.


So, to summarize, I don't think I've lost my empathy. I think that what has happened instead is that those softer traits have taken a back seat during a time of massive change. I need the Communication and Woo skills more right now. I don't think woo is a real strength above other things. As far as the "Arranger" part goes, that is most of what my work has been this year. I've gotten better at that through practice, but I don't think it is a core part of my personality.


As a result of taking this test again, I think I have three strengths worth looking at, not five. It is silly to think that each person should have five or focus on five. In fact, Activator and Ideation above all others are strengths that are especially dominant in me. The others will change each year I'm pretty certain.


Activator (I fit the description perfectly)
Ideation
A mixture of Empathy and Communication (I use both together, but not each alone. I call this having social skills in general).

...<< MORE >>

Slipping Quality Due to Growth

How do we judge what quality customers will accept? How low is too low? When do consumers start to notice?

I saw this article about Toyota quality today. It startles me that this could be nearly ANY of the software companies who are rapidly growing right now. I hope it is never my company.

I don't blame the testing department of Toyota for this issue. It's the competative spirit gone wrong. It's a lack of emphasis being put on quality. Those little sacrifices and compromises add up. Now they have to pay in good will and customer loyalty. The decisions made back most likely in 2004 come to haunt them. I wonder how long it will take them to turn it around (if they can).

It is much easier to keep a reputation for having the highest quality than it is to gain or regain it. I wonder if it is cost-effective? I've never seen data on this.
...<< MORE >>

Testing in the Absence of Proof

I was inspired to make a blog after reading this. The context under which the word Religion is used is really interesting to me. Partly because it indicates that testing ideas and methodologies which can't be quantified aren't scientific and are just matters of belief.

As a touchy feely type person, and also a person from an artistic background who has since gained an interest in science, I feel sorry for people who have no belief beyond what they can verify. I think that the things that matter most in life can't be verified or quantified. I think that innovation can't be proven. You can't prove a new idea is good before you try it. This way of thinking makes some people feel really uncomfortable, but I'm not sure why. The most basic of scientific principles starts with a hypothesis. That is scientific curiosity. Even if an idea proves to be false after testing the theory, I have great respect for those who "wonder if" and will go find out. Those are the people making a real contribution to the testing profession. Those who seeks to stabalize, quantify, assure rather than test, and compartmentalize all testing really depress me. Trying to quantify testing is not a new idea. I find it rather boring. It is something I do because I have to, and as quickly as possible. I don't think it really adds much to the end quality of software.

I'm not suggesting that we shouldn't verify our assumptions, test our ideas, improve and adjust based on the findings, only that lack of evidence and proof is no reason to prevent us from trying something new. If we are to make true advances in human testing, small or large, we must have the courage to apply new ideas. Then as we are trying the new ideas, still have the humility to logically figure out what is working and what is not and make changes. Would the same idea work in another context?
...<< MORE >>

Automation and John Henry Plus The Perils of Training

I haven't been blogging because I'm learning to become a coder part time. I've now made a deal with my boss so that I don't go crazy (well anymore than I already am).

It looks a little something like this:
Half.day perWeek testyRedhead="coder". If testyRedhead != coder then resume testing and being <hostile

My reluctance, mistrust, and feeling that I can test better than automation can reminds me of the story of John Henry's Hammer. In this story, John Henry tries to beat a steam hammer but kills himself in the process. Ultimately, he fails to beat the industrial revolution. Is this heroism, or is he just stubborn and dead? My issues are no different. Being change resistant is unwise in the face of a steam hammer of changing industry trends.

I've been attending training. In fact, one of the best classes I've ever taken was last week. I feel 20% more comfortable with object oriented programming than before. I did quite well in the class, especially considering my lack of experience. However, it pointed out several things to me and clarified them by shedding a blinding light on several facts.

1. It isn't that I can't do it, it's just that I hate it and am not interested.

2. I like working on UIs, skinning, and overall project design and even pseudo code, but I hate writing code.

3. I'd rather get a root canal than work on any back end database code.

4. I could force myself to do this, but if it becomes a major part of my job I won't even make it one year more.

5. Each thing I learn points out the vast area that I have no knowledge of, and rather than a curious interest, it fills me with an unexpected level of dread and unhappiness.

6. My favorite part of the class was finding the bugs I found in the released software.

7. I wasn't put on the earth to be unhappy. Yes, there are things that must be done that we don't like in every job. If those things are growing and the parts you love are shrinking, much like a failing relationship, you must try to turn it around, and repair it. If you can't, you have to get out before you are no longer yourself, and become something miserable that is barely surviving life. I am hoping the "contain to 1/10th of job" compromise will take hold and will be something that works out for both parties. You can't have everything your way all of the time.

In other news, I'm hoping I don't get the "opportunity" (a.k.a. mandatory order from boss) to attend another course (a.k.a. self-impressed speech) by famed professor/author and Florida Software Testing Posse Member James A. Whittaker. The last "opportunity for learning" I had on the topic of Breaking Computer Software was the one and only training I've ever walked out of. It is the first time I've felt both insulted and horrified. I can't figure out if it was the fact that the ...<< MORE >>

Motivation

I have yet to find the man, however exalted his station, who did not do better work and put forth greater effort under a spirit of approval than under a spirit of criticism.
Charles Schwab

Each time we ship a product I have this phase afterwords which is sort of like the depression that sets in after the holidays when all you have is winter and the part you looked forward to is over. Now it's just paying the bills and taking down the decorations. The waiting and excitement are over. I'm no longer in a frantic state of final testing. Now begins the slow task of planning, requirements, and the ever dreaded "side-projects". I'm not much of a "side-project" person. I don't like writing code. I like learning, but really, I like testing. I enjoy being constantly busy and changing what I'm doing all of the time. I don't get whiplash from doing 5 different things in 2 weeks. I want someone to give me an impossible task to accomplish.

So, when things slow down, I start to get critical. Rather than doing everything I can to make it, I can see every flaw. Every way that the process can be better. That we can be better. Not just the place I work, but myself, and everyone around me. I can't stop trying to improve things. I am used to taking out all of my energy on the software I'm testing. Now what?

It's tempting to just turn that energy on something else.

I think I've lost the ability to sit back, relax, and enjoy the results of a goal met. I sit around anxiously thinking, "What now?"

To make matters stranger, I've reached two monumental goals in my personal life. I bought and moved into a beautiful house. I've always wanted a house, and this one is really wonderful. It is the culmination of years of saving and much effort and risk taking. To top it off, I've completed a very difficult health related task, and for the first time ever reached a physical "goal weight". That goal was over 5 years in the making and is always a "work in progress" even now. I felt on top of the world for a few days. Now I realize how much I have to maintain, how much I have to lose. I feel the responsibility of it and not the truth. The truth is, I am quite lucky and I can handle this. This happened in part because of my work, and in part because of luck. It didn't happen by itself.

The other truth is that I love the company I work for, the software I work on, and above all else, the team of people I work with. Being too critical is not a good quality, even in a tester. It is certainly not a good quality in a test lead. ...<< MORE >>

iLove iRobot

What fun it is to type "household appliance" as a keyword for my testing rant today. My boyfriend brought home a Roomba iRobot for us to use in our new house. Not only do I enjoy shouting, "Release the Robot!" but I also love watching it work. It also doubles as a cat toy/cat torture device, depending on which cat. The younger cat looks interested, while the old one looks horrified.

How it works
It stumbles along trying to find the boundary of a room. When it runs into a wall (imaginary as created by the false wall signal or a real wall or object), it bounces a bit as to not damage either participant in the collision, and then goes in another direction. If it encounters significant dirt, it spins in one spot for a moment before moving on.

Error Handling
We did encounter one error where it got stuck under our dining room buffet hutch. The two previous times it was stuck it was able to unstick itself easily by trying different angles. However, this time it really was stuck. It alerted me to the situation by making a sound so I could manually unstick it.

Usability
On the down side, it takes much longer than a human to vacuum a room of equal size. On the up side, the floor is really clean when it is finished, it is fun to watch, and I can do something else while it cleans the floor. Plus, it is fun to use and I think of it as my "little buddy". It is easy to remove the bin to clean the dirt out. It is easy to tell the battery state from the lights on the top of the vacuum. It even finds it's own charger when you are finished if you ask it to.

Learning Curve
Any person of reasonable intelligence willing to read the manual who has electricity and some D batteries for the fake wall can use this thing to clean floors. Now, in all fairness, repairing it would be difficult. You have to be reasonable about what limits this system can handle. I'm not going to expect it to vacuum up huge items or anything wet.


Why is my test automation not like this? How can I make it a trusty sidekick, which while it isn't perfect, is honestly useful and enjoyable for me to use? I think that the key is, I didn't create the iRobot. I didn't program it. I just use it. I love using it. It works very well for the purpose intended. I think if I tried to get it to do dishes I'd be less satisfied. The key for me is getting a tool that is already for the correct purpose and using it well. I think my attitude towards automation will improve vastly if I can do that. Not being good at writing code isn't a crime. It doesn't mean that I'm not a great tester.
...<< MORE >>

Bug Harvest Idea

PLEASE COMMENT-This is the first time I've taken this idea public and I'm really looking for honest, constructive feedback. Even if think it isn't at all useful, please explain why. It will help me.

Intent

A bug harvest is an organized and focused blackbox group testing exercise with a twist. It is called a "harvest" because just as those who pick strawberries may not be the fastest at picking apples, you want to select testers who already work on similar areas or technologies for effective and quick group testing in this context. The word harvest also implies that the fruit must be ripe. It is called a harvest because of the notion that bugs are introduced into the software with code changes, making certain times very "ripe" for the picking of bugs. Most experienced testers know exactly when that time is, and are used to using code changes to prioritize their testing. In some companies there is a handoff process when certain features are ready for test. There may be multiple times in the project where fast coverage by multiple people is very beneficial, especially if the development is staged. It is not meant to replace any other test methods, such as an open bug bash or bug hunt, but instead is an additional method that for use when bugs must be found under tight time constraints. It is intended to cover new code changes as quickly and efficiently as possible to compress the time it takes to get feedback and bugs to developers. It can be used in basically any software development model so long as your testing department has more than one tester, but I have only personally used it in a waterfall development cycle and one staged development model.

The other time I use this method is when fire drill requests come in. Wait, a new operating system and we have 3 days to test on it before they lock down the code?  Oops, we forgot to tell your team about our new dot release, and it could impact compatability with your software. In this case, why not throw every tester in the world at the problem? Well, because it leads to duplicate bugs, takes everyone off of what they are working on and stalls your project, and because you can do just as well, especially in early stages with a small focused swarm of testers attacking the area as with a large misdirected frantic swarm. I love Bug Harvest as a method for dealing with firedrills and have gone to it more than 5 times in my last product cycle with great results.

Who, When, Where
One of the keys to holding a successful bug harvest is to invite the right testers. This is not a bug hunt or an open call for all people to beat on a new feature. This will most likely overwhelm the developer with bugs, some "not bugs" and create a large pile of defects to be sifted through which may or may not be about the feature you wanted information on. Only testers with ...<< MORE >>

Test Automation Planning and Dogma

I'm working on my automation plan today and found this article. I can't believe it is written in 1999 and the same mistakes are being made at most companies still. My blogging time is really short right now, but I'm hoping in a few weeks I'll be able to blog more.

I was told by someone that my dogmatic talking about human testing is dangerous and that most people don't agree with me. It feels sort of like the mafia to hear that sort of thing, although, I know the intention was friendly, just to let me know that I'm most likely making career limiting moves. My desire for balance in testing is unpopular? With whom? Certainly not with most people who test or use software that results from it.

I must however give thanks for the reminder that I'm fighting a losing battle, but I've never been a political or popularity motivated person. I'm the dangerous kind. The kind who does what they believe is right despite criticism. While I view this as being ethical, I think others view it as stubborn, naive, and delusional. I also realize that taking on all of the things you perceive as injustice is one really good way to ruin your life if you don't put limits on it. Activism is tiring, so you have to be quite selective.

I may not have enough power or influence to do anything, but I have something still more important to me than having a big impact. I respect my own actions, even if other people don't. That is worth more to me than my job. I wish that more people had the courage to speak dogmatically about testing if they felt that way. Holding back ideas because they are unpopular just makes me want to scream that the Emperor is naked. Hey you, yes you, Test Automation. Put some pants on!
...<< MORE >>

How to Test Better Than Microsoft

Summary: Make usability and the customer's human experience using the software more important than creation and use of any particular method or tool in testing. You can't beat them in tools, number of people, amount of money, automation, or willingness to be competitive (well, I would argue that dog fighting rings could beat them in being competitive, but that would just be snarky). They have one huge weak spot, and that is in the area of creativity and seeing important things that can't be measured.

Judging from my views on Automation, I'm guessing most of you can deduce that I'm not a Microsoft Employee. Many people feel that Microsoft is a huge leader in software, and for that reason, they must have the best quality principles and methodologies in use, and every other company should follow them. In my blog today, I'd like to share why I completely disagree with that notion.

Having talked with many people, both Microsofties and Microserfs former and current, there are many positive things about testing at Microsoft. The availability of tools, the ease of finding them, the constant process improvements being driven by the competitive culture, and even the perception that aggression is "effective" in some groups have been sighted as strengths. I've also heard that the culture encourages people to take on side-projects which impact as many people as possible in order to get raises and bonuses. I am not suggesting that good testing isn't happening there, I'm only suggesting that it is possible to do better. I think that Google IS doing better. I think that our company IS doing better. If companies take the lead from Microsoft while they are growing, they will not be doing better for long. I haven't seen anyone have luck at imitating them and still produce better software than they do.

1. Currently having the most money is not a good measure of testing success.

2. Saving the most money is not a good measure of testing success.

3. Many of their users perceive their quality as very low and hate using their products.

4. Their products have very little "soul" and feel bland, generic, and slightly boring and inhuman in general. From this I assume that usability bugs are not taken very seriously, or, even worse, aren't being found and reported because it isn't easily tested using automation. How can something both be boring and "needlessly complicated?" Use Outlook and find out.

5. Who can automate the most and who can pass code coverage are also not a good measures of testing success. Covering the code is NOT equal to testing the product.

6. Who can cover their butts the most while testing is also not a good measure of testing success.

7. Having highest percentage of testers who can code does not mean you have the best test team.

8. The lengths someone will go to, cutting out their personal life balance is not a good measure of employee loyalty and quality.

How can I comment on this having never tested at Microsoft? First, a confession. I was married for 5 years (as ...<< MORE >>

No More Ugg Boots!

Do you remember when every celebrity under the sun was paying hundreds of dollars on Ugg boots? They were such a huge trend, and so very very ugly.

I am tired of using automation tools that could be Armani Suits but instead  are Ugg boots or a $1200 handbag, or even $3000 jelly flip-flops. Can I just get a sturdy backpack? Please? Didn't so much need the high pricepoint in learning just to have my tool go out of fashion in less than a year.

This is what happens when test managers don't understand what they are asking for. A test automation tool IS a software project. Ignoring the users of it is really not a good idea. Designing it for 2% of those using it, also not a good idea.

I have trend whiplash so badly that the last automation tool I saw (it was a mandate-I had to go), the entire time we were looking at the code, my only thought was, "I've been poor. I'm good at it. I'm a great saver. If I have to choose between doing this all day and my job there is no choice. I could wait tables. Seriously, I'm kind of cute, I'd do ok." It is not a good sign if Automation has been rammed down your throat so much that you have a gag reflex whenever Automation is so much as mentioned. Umm pushy much? I only blog about this because I think this is industry wide and a huge problem. I'm extremely passionate about testing. Does my burn out on Automation mean I'm a "burned out tester" now? How could I not be burned out when my work on automated testing in the last 3 years has been much like digging a hole just to fill it back up again? I will say, however, there are 2 automated tools which I DO support, will use, and almost (squirm) like? Yes. I like them.

Companies who are reckless with their talented testers, their brain engaged testers, who can't appreciate and value people beyond their coding skills deserve to be brain drained by other companies. We've seen the crappy results from other companies in the industry and I hope we don't need to repeat their short sighted and failed experiements in order to fully understand that having all testing automated is a horrid goal. To have all software tested by "developers in test" means that anyone creative who is scientific on the side no longer has value.

I've got my hiking boots on. Keep those furry monstrocities off of my feet. I've got somewhere to be!

...<< MORE >>

Find your Strengths-Here are mine

According to the book Now, Find your Strengths. here I think it is pretty accurate. I love the boss who had everyone on our team read the book and take the test. The idea that you should focus on areas you have natural talent is a very simple one, but effective.

Here are my "themes" as according to the test. What does a person like this love? A person like this LOVES to get ideas on collaborative testing, share ideas, and put them into practice NOW NOW NOW. Ha! I also love working on product interaction and feature integration because the way that code connects is interesting to me.


C O N N E C T E D N E S S
People strong in the Connectedness theme have faith in the links between all things. They believe there are few coincidences and that almost every event has a reason.


I D E A T I O N
People strong in the Ideation theme are fascinated by ideas. They are able to find connections between seemingly disparate phenomena.


I N P U T
People strong in the Input theme have a craving to know more. Often they like to collect and archive all kinds of information.


A C T I V A T O R
People strong in the Activator theme can make things happen by turning thoughts into action. They are often impatient.


E M P A T H Y
People strong in the Empathy theme can sense the feelings of other people by imagining themselves in others' lives or others' situations.
...<< MORE >>

I've been allowed!

This just in: My manager has agreed to allow me to follow my dreams of cross-team QE demos and bug finding competitions. At my company, agreeing to let me persue this does not mean time and budget or resources to persue it, just that I have a chance to stalk people for resources, convince them to help me do it, and to use my charm and charisma (plus begging and making cookies), to try to get it to happen. On my side? Excessive enthusiasm. Against me? I'm already pretty overloaded, but hey, it's worth some risk if you really really WANT to make a difference. It's worth me giving mediocrity to less important things to really give this area I care about my very best. Nothing gives me thrills quite like testers from multiple teams in a bug finding frenzy for prizes and small scale fame. I'm doing my best to make testing more fun and efficient at my company in the way I believe is best. What else can I work on that is as important? Making more useless automation to maintain? Covering my butt more completely with further layers of documentation?

I just heard back from the legal department at my job! I am now free to talk at will about my testing idea with the general public!

This is great news.

Plus the lawyer told me that he hopes I write a paper. I plan to not let him down, although it will be much more blog than paper.

I'm swamped with bugs this week and testing shiny future operating systems, so most likely I'll get time to start this weekend or next week. Expect few blogs here until "The big one". I want your feedback, so please be ruthless. Tell me my ideas are NOT new, NOT origional, and pick them to shreds. Umm. Then help me put them back together as the stronger leaner few survivors?

Coming Soon: My Bug Harvest Idea in detail.

*wanders off singing to herself much like Bruce Almighty-"I've got the powah!"*
...<< MORE >>