Define cheap. You gave me $0.02 & told me to buy lunch with it.
Today I tested a feature which supposedly allows a user to upload a training video. I reported a bug that I was unable to load any of our small files, even the smallest video we had. I got an appropriate error which informed me of the upload failure due to exceeding the file size. The maximum file size?
2,000,000 bytes
Well, that doesn't look so bad, does it? If you gave me that many dollars, I'd be pretty rich! So, that leaves me trying to explain a few things.
1. People don't speak in bytes. I mean, not since the early 1990's. Not anyone I like to talk to, unless they are talking about converting FROM bytes to something more useful in the UI or using byte comparison to detect changes for automated checks, but I digress here. The point is, bytes do amazing things, but video takes more than that. If we are attaching a .txt file, I'm happy with this limit.
2. Before they followed gmail in offering ever growing free space, I believe Hotmail allowed 20GB space for users for free! Don't quote me on that one, because it's been so long since I had to much about with deleting my photos and excel spreadsheets that I've forgotten. The client I'm testing for is paying money. YouTube lets any video jockey who wants to upload a cute kitten riding a roomba have bandwidth at 5GB a pop and that's while compressing the daylights out of it for you so it will fit in the 5GB beach bucket. We aren't talking cinema quality here. Yet we can't allow these paid users upload more than .001 GB of video? Feel free to point me to .001GB worth of video to show me what kind of video training I can upload. I'm sure it will be very informative. Rated G or PG only please. I keep content to PG13 or below. It's me, so I have to allow for an occasional "d" or "s" word, but I try to avoid dropping any "f" bombs on my kindly readership.
3. Why? Why bother to support a format without supporting the ability to upload files IN that format? It's like the gift of the Magi, but less sweet. However, I had fun giggling about this defect. The funniest thing about this defect? It isn't considered a bug.
What? How can this not be a bug? We're talking about 0.001GB allocation. Several years ago, before they moved to mostly unlimited, free Hotmail allowed users 20GB. YouTube is giving us 5GB per video. Has been for ages. Cloud space is going for $.03 per hour (after a setup free, some contracts that said something about rumplestilskin, and a few cloudy things). By calling this NOT A BUG do you realize what you've done? You've made me do it. Yes, I researched the average hourly wage of EACH user by area and job title in the US who we intend to use this system, and conservatively, based on public data, that is an average of $26.43 PER user.
So, for each user who spends time trying to upload something and fails, we lose not just the support cost, but .44 seconds per MINUTE over just allowing them to do what the system intends to do, which is upload the blessed training video.
Now, for the love of Kaner, do not make the tester go crazy again, or Bachs help me I WILL calculate the likelihood that the user will go view a cat riding on a roomba rather than watch the software training that could help them do their job.
So, why is it important to USE software rather than just use automated checks without any complimentary professional human testing? Because this meets the requirements for each portion of the system. This unusable combination of this, which when considered by any person makes no sense, is exactly what would be delivered to users.
Let's test together. By that I mean, let's test our products working with other products. Let's test our features working with other features. Let's test our data in other browsers. Let's test all of the integration with all of the interactions all happening at once. Because that is what life is like. Life is not one little isolated morsel by itself. It is the full beautiful messy symphony of noise. We may like different types of music, but I'm trying to find out, can you dance to it? Is there any rhythm here? When I talk about testing the workflow, about testing for the user experience, it is this exact thing I'm talking about. Not about converting Bytes to GB, but about having some point to your testing. It isn't about being right. It's about being useful. And sometimes, it's about shaking your groove thing to a really great application.
2,000,000 bytes
Well, that doesn't look so bad, does it? If you gave me that many dollars, I'd be pretty rich! So, that leaves me trying to explain a few things.
1. People don't speak in bytes. I mean, not since the early 1990's. Not anyone I like to talk to, unless they are talking about converting FROM bytes to something more useful in the UI or using byte comparison to detect changes for automated checks, but I digress here. The point is, bytes do amazing things, but video takes more than that. If we are attaching a .txt file, I'm happy with this limit.
2. Before they followed gmail in offering ever growing free space, I believe Hotmail allowed 20GB space for users for free! Don't quote me on that one, because it's been so long since I had to much about with deleting my photos and excel spreadsheets that I've forgotten. The client I'm testing for is paying money. YouTube lets any video jockey who wants to upload a cute kitten riding a roomba have bandwidth at 5GB a pop and that's while compressing the daylights out of it for you so it will fit in the 5GB beach bucket. We aren't talking cinema quality here. Yet we can't allow these paid users upload more than .001 GB of video? Feel free to point me to .001GB worth of video to show me what kind of video training I can upload. I'm sure it will be very informative. Rated G or PG only please. I keep content to PG13 or below. It's me, so I have to allow for an occasional "d" or "s" word, but I try to avoid dropping any "f" bombs on my kindly readership.
3. Why? Why bother to support a format without supporting the ability to upload files IN that format? It's like the gift of the Magi, but less sweet. However, I had fun giggling about this defect. The funniest thing about this defect? It isn't considered a bug.
What? How can this not be a bug? We're talking about 0.001GB allocation. Several years ago, before they moved to mostly unlimited, free Hotmail allowed users 20GB. YouTube is giving us 5GB per video. Has been for ages. Cloud space is going for $.03 per hour (after a setup free, some contracts that said something about rumplestilskin, and a few cloudy things). By calling this NOT A BUG do you realize what you've done? You've made me do it. Yes, I researched the average hourly wage of EACH user by area and job title in the US who we intend to use this system, and conservatively, based on public data, that is an average of $26.43 PER user.
So, for each user who spends time trying to upload something and fails, we lose not just the support cost, but .44 seconds per MINUTE over just allowing them to do what the system intends to do, which is upload the blessed training video.
Now, for the love of Kaner, do not make the tester go crazy again, or Bachs help me I WILL calculate the likelihood that the user will go view a cat riding on a roomba rather than watch the software training that could help them do their job.
So, why is it important to USE software rather than just use automated checks without any complimentary professional human testing? Because this meets the requirements for each portion of the system. This unusable combination of this, which when considered by any person makes no sense, is exactly what would be delivered to users.
Let's test together. By that I mean, let's test our products working with other products. Let's test our features working with other features. Let's test our data in other browsers. Let's test all of the integration with all of the interactions all happening at once. Because that is what life is like. Life is not one little isolated morsel by itself. It is the full beautiful messy symphony of noise. We may like different types of music, but I'm trying to find out, can you dance to it? Is there any rhythm here? When I talk about testing the workflow, about testing for the user experience, it is this exact thing I'm talking about. Not about converting Bytes to GB, but about having some point to your testing. It isn't about being right. It's about being useful. And sometimes, it's about shaking your groove thing to a really great application.
Posted by Testy Redhead at 18 Jan 2011 
Categories: Me, ethics, Humor, Bugs, Working with Others, Subjective Data, tools, reporting, usability, Human Testing, user, stats, Users, methodology, Development, Human Bias, Troubleshooting, Test, Learning, Survival, Error Handling, language, user experience, workflow testing, Process, users, strategy, User Rant, Training, math, User Experience, test documentation, blackbox, Metrics, test planning, Software Testing, TEAMS, training, Measuring, Leadership, test cases
Categories: Me, ethics, Humor, Bugs, Working with Others, Subjective Data, tools, reporting, usability, Human Testing, user, stats, Users, methodology, Development, Human Bias, Troubleshooting, Test, Learning, Survival, Error Handling, language, user experience, workflow testing, Process, users, strategy, User Rant, Training, math, User Experience, test documentation, blackbox, Metrics, test planning, Software Testing, TEAMS, training, Measuring, Leadership, test cases
There are security issues surrounding large HTTP uploads, and limits around that size are the defaults for many app servers, so you might be seeing a deployment artifact as opposed to a limit in the software. It's not about the cost of permanent storage.
Anyway videos under 2MB do exist. When people send videos via mobile phone MMS, they are no bigger.
And google turns up many examples: http://www.google.com/search?q=training+video+2mib
This could be considered a requirements issue as much or more than implementation quality: a requirement that just says "a video" isn't meaningful without context.
Reply to this
You are right. It isn't JUST about cost of storage. I suggested we have a log-in for one person who uploads the training and that account be given the ability to load whatever is needed.
Instead, the limit has been lifted for a short time. I should mention that all users must log-in and the actions are tracked.
Honestly, I'm interested in supporting the REAL video webinars made for this system which includes demos. If I have to give up VGA and go MONO, why bother with a video? I'll upload an ANSI art and we'll call it good.
Reply to this
Oops, URL was intended to read http://www.google.com.au/search?q=training+video+2mb
Reply to this
Thanks for making this informative and fun! You have a gift!
Reply to this
Nice.
I love watching the dinosaurs of quantitative analysis who've dominated software theory and design over the past 50 years wonder why its getting so cold and what those soft white things falling from they sky could be. It's sounding like you're running into common reaction #3 to being confronted with a challenge from system analysis, denial. "It's not a bug because our customers don't do it," is funny when you are the ones preventing your customers from doing it in the first place.
That being said I expect you would have run into reaction #1 if they had acknowledged it as a defect, doing more of the same. "Quants" tend to think a flaw in their models and designs is indicative of a flaw in their specific theory rather than a deeper flaw in their entire approach. The solution to an encountered problem is then to develop a better algorithm (usually defined as more complex) rather that to look at the problem from different angles.
Reply to this
Hi Lanette...
Calm down. Deep breath. Be seated. "Clear blue ocean, clear blue ocean, clear blue ocean." Ah. Better now? Good.
There's something you don't understand. I'm not chiding when I say that; I'm making an observation. I don't understand it either. Why would they want to limit video to 2MB? I'm confused.
When I'm confused, I remember the story of Jon Bach, who was training to be a tester with James. One day Jon got depressed and confused and frustrated and went into James and said, "I dunno... I don't think this testing thing is going to work out. James said, "Huh. What do you mean?" Jon said, "I'm testing this thing, and I'm just so confused." James paused for a moment, peered at Jon and said "When you're confused, that's a sign that there's something confusing going on."
So again, I'm confused too. Why does someone consider this a non-bug? When I have a question like that, I try not to get angry, because that clouds my view of what else might be wrong. So I go to the Rule of (at Least) Three and ask about the confusion. I don't give up until I have at least three possible answers.
- Maybe they didn't understand my report. Maybe they think that I'm complaining about the error message, and they don't realize the limitation (for other reasons below).
- Maybe they're mistaking 2 million bytes for 2,000,000 GB or 2,000,000 MB. That sort of thing can happen more easily than you think. For example, you say above "We're talking about 0.001GB allocation" where in fact it's 0.002 GB.
- Maybe there is a requirement for video uploading and they're complying maliciously. That is, as long as the feature exists, they get paid.
- Maybe it's true: all they expect is tiny video snippets from smart phones.
- Maybe they've decided that this IS a bug but there are more important bugs to fix in this build cycle.
- Maybe this is a trial balloon version of the feature, and they want to see customers complain about it so they can justify more hardware.
- Maybe someone set the report as "not a bug" when they meant to set it as "showstopper" in the bug tracking system.
- Maybe, probably, there's something I haven't thought of.
So in this situation, I would try to get the root of the confusion and find out why this isn't a bug. But I'd also use this Pro Tip from Gil Broza: "Why?" puts people on the defensive, so ask "How does this fulfill this implicit requirement...", or "What if the user tried to..."
You're a tester. I don't think you're the product owner here. To me, as someone who has been both, the role of the tester is just as Jerry said at STPCon: we make people aware of things they didn't realize. Maybe they're not aware of the implications of their decisions, and you could help with that. But there's something else, something that an expert tester needs to become aware of: What do THEY consider to be a great application? It's that--and not what YOU consider to be a great application--that you're trying to help them achieve. Right?
---Micha
Reply to this