You pick the right tool

 

As you can see, it is sometimes crucial to select a right tool. Even to stick to it, regardless on how awkward and painful it might feel. I just had to open with this scene, it is anyhow from one of my all time favorite movies. The book it is based on, is definitely one of my top 5:s.

We’ve all been there. It’s late night, we’re out in the park, having fun and all we have left is bottle of wine. And of course the corkscrew is nowhere to be found. So you start using your imagination. You might have a normal screw and a screwdriver, maybe a pair of tongs, too, you might be carrying a multi-tool, for what I know. Or not. The cork stays in the bottle.

You go through your pockets again, ask your friends, someone says that all he’g got is a pen, other one offers you a rock.

You see where I’m going here? A pen is mightier than a rock? Well, at least now it is.

Now you could open the bottle by smashing the head to a rock. Or smashing the rock to the head of the bottle. That might work, too. In the other hand, there’s always risks of getting bottle broken so, that the glass gets inside the bottle. Besides, that will create a mess in the grass and I a devoted dog owner and animal lover hate when people break bottles or other glass and leave the stuff there. Besides, people can get hurt too, for real.

So that leaves you the pen, right. You could write a letter – I know, it’s old school, but it’s nice sometimes, for real – to someone with a corkscrew to drop by. What’s there to loose? You got the whole weekend ahead of you (Did I mention it is a Friday night?) Or you could rob a corkscrew store with it, not the world’s best idea, though, giving it to be late night and the stores are all closed.

You could also take the pen and push the cork inside the bottle. That actually does work. You might get to spill some, but then again, at least there isn’t going to be shattered glass anywhere for the paws of the two to four legged friends. Notice I left a place for three legged dogs or cats, even squirrels.

So, you take the pen and grab the bottle. Your hands are getting a bit sweaty so you what your sleeve around the bottle. It keeps it in its place. Not completely, but firm enough. You take a breath.

You push the pen against the cork and push. Nothing happens, except the veins in your temples seems to be exploding, your face turns to red and the pen hurts your palm. A lot. You let go for awhile, and try again. No change to the situation. Cork is till on and you seem to be screwed.

A friend of yours, the one that has taken one more than you, mumbles something and offers you the stone from his hand. You look at him and smile and are about to shake your head, but change your mind and take the stone.

It is smooth on the other side, the side agains your palms has some edges and it feels, if not cold, then at least cool. You take the stone, push the bottle against the ground and hold it between your feet and hold the pen on your other hand against the cork. Then you slowly but firmly hammer the pen inside the bottle with the stone and finally, you’re done. Everything’s fine again, you drink the wine, get in to the night and wake up next morning with a hangover and some blurry memories. You might even end up having fun, who knows.

What I mean here is that you should choose your tools, for real. First of all you need to know that you have a need for a tool, then you need to check what requirements you have for it, then you need to find it. Thanks to the interwebs, it is fairly easy nowadays. After finding the tools, take several, and use them for awhile in the situation needed. So to say, evaluate them.

If you run into problems with the tool (you should, for real, even a sledgehammer needs some maintenance), try to find out if anybody has had the same issues. Most likely you’re not alone.

Check the maintenance costs. If you use more time maintaining the tool than the flaky tests, you probably have the wrong tool. Regardless on how good it looks, sounds or feels.

And once and for all; don’t get stuck with the first evaluation, don’t get stuck with your evaluation choice, either. In case the tool loses its focus and usability, make sure you can move away from it.

I myself are at the moment in that kind of situation: using a multitool with a gentle learning curve, but the maintenance and the license is starting to feel bad. It was a tool I was familiar with, a tool I’ve used for years in the previous companies, and it used to be an open source tool. They ended the open source path last year (if I remember correctly) and otherwise turned the usability a bit more worse, too.

So I’m considering moving the tests to another multitool, an open source based tool with steeper learning curve, but a considerably larger user group. Actually, I’ve done my consideration, all I need to do now is to transfer and modify the current tests from the first tool to the second one.

Some days of testing

Well, of course everything always goes as planned. Or then again, of course not. And when they don’t go the way you thought, planned or wanted, you might get a temptation to just let it go. But letting go would let you only quit, not finish. Determination or stubbornness – call it what you will – you need it. Especially when handling things that are voluntary and not always easy to squeeze in to your day.

That being said, I’ve been struggling. Anyhow, here’s a small report on things I’ve managed to accomplish so far.

Day 7: Find an accessibility bug.

I suppose it was this one that broke my flow. First of all, I came in a day late, dollar short. Second thing was that I’ve never done any accessibility testing. That, me being in the case, leads some  issues.

I have always been ‘try first, stumble and read the manual to get to know what did I do wrong’ -type of guy. This time was no exception. I ended up reading few short sentences on the  W3 accessibility testing wiki and rushed my way to turn on accessibility features on my notebook. After fooling around for awhile I got nowhere, if not to the land of frustration. The laptop is old and slow and it had difficulties to read the web pages in Finnish so that I would actually understand. That was due to the lack of memory (most likely) and bad/lacking support of the chipset. I’m running CloudReady/Chromebook OS on Asus EeePC with Intel Atom chipset. It fails to work on most of the OS I’ve tried.

So I gave up.

You see, testing is my profession and therefore it actually is my passion. But not on my spare time. I have divided my days between work and non-work, and it suits me fine. It just don’t help doing test related tasks (regardless how fun they are :D) on my spare time. It’s not that I wouldn’t like, but as I have my priorities at work, I also have them when I’m at home. And in that kind of issues, it actually is really easy to quit. Which I did.

But now I went and read a bit more on accessibility testing and noticed that there is few semi-automated tools listed on the page. I suppose I’ll give them a try at work on some of our interfaces. Semi automation is still partly automation (which I do like a lot) and I might also find something worthwhile. You never know where you end up when you start testing without a decent specification. And even with a spec you might find yourself in between rock and a weird place.

 

My plan was to write about the other tests I did (but did not write about), but this post starts to get long enough, I’ll have to fill the other blanks later on.

Crazy Testing

 

Last week I joined 30 Days Of Testing -challenge that was organized by Ministry of Testing. And I have to admit it has been fun. Besides that I have found myself doing things I should’ve been doing all the time, I’ve also done something new, too.

First of all the ‘Listen to a testing podcast‘ was very revealing as well as fruitful. I’ve been now listening to the Test Talks while riding bike to and from work and found out new ideas, new ways to think about testing. It is really refreshing to notice that, even though I’ve been working as a single tester (and I’ve done that on purpose, also)  quite a lot for the past years, there’s a community out there. People thinking, if not like me, but at least similar kind of issues and situations that I am facing in my profession. So to say, it has been a blast to notice that 😀

Yesterday’s challenge was to do a ‘Crazy Test’. So, what I did was the following:

  1. Connect iPhone to the car with Apple CarPlay
  2. Use Siri to start music (‘Play music from artist Metallica‘)
  3. Wait until Siri gives you a ping that it has received the command (but before it confirms what it is about to do)
  4. Put on reverse gear

Now, my expected result would be that I could reverse in the car by watching the rear camera and listening to music (at this example, Metallica). Of course it didn’t do anything like that. I received the ping from Siri to indicate that it had received my message, but when the rear camera was switched on, the command was weirdly gone. I could reproduce the behavior even when making a phone call request.

Now one might thing it is a safety feature. But I can’t think it that way, for me it feels like a bug. Software is not working the way I expect it to do. If I turn on the phone and call manually and start to reverse, the call is connected. As well when it comes to music. I can pretty well listen to music (even Metallica) and reverse at the same time.

Now, is this a race condition, or something else? Where is the problem? In Siri/Apple CarPlay? In the API between car and carplay? Has this scenario been tested? Does not seem likely. Or if it has been tested, perhaps the results have been ignored in order to get the software out in time.  Or the responsibility of the behavior has not been clear; should it be car that takes care of this, or the phone?

You might also say (and I wouldn’t argue against that) that this wasn’t particularly crazy test. More or less exploratory boundary/edge case. I was not driving my car sideways on the ice, or 110 Km/h on the highway. I don’t actually care. It was crazy enough for me. Plus that I noticed something new, learned something new, which I suppose should be the main point in life and work altogether. i mean, you can never be crazy enough, not unless you learn more and get crazier.

And I’d like to automate that test. For real 😀

 

 

Testing in popular culture

This morning, while browsing for a podcast for this 30 Days Of testing -challenge, I found myself thinking. Yes I know, it’s a harsh condition and I try to avoid it regularly, but one can’t help oneself. Not when you’re born this way, you know, with brains and all.

Anyhow, I started thinking, who was the test engineer in Starfleet Academy that tested Kobayashi Maru? Clearly, for it being a computer scenario, it should’ve been tested. How otherwise could’ve James T. Kirk beaten it, even by cheating? And furthermore, in the ship, there’s only operators available, mainly. Someone must have been done a hell of a coding in order to get the USS Enterprise to get around the orbit in the first place. And, as we all know, when there’s a developer there should be a tester available.

Which brings me to this: Is there testing involved in popular culture? In the ‘Saving Matt Damon’ – movie, The Martian (which by the way is a great book, not as good movie, Ridley Scott blew it), NASA does skip the tests, based on risk assessment, apparently,  in order to send the supplies to the Mars a bit more earlier, even the length of the tests is slightly discussed. That’s most likely due to that the author of the book is, if I recall correctly, a SW engineer.

But is there more QA/Test references in popular culture? If not, why not? We’re working on a field of SW (Well, HW needs to be tested, too, but that’s another story) and the field gets bigger and bigger all the time. It is clearly so, nowadays, that hackers and developers can actually tell people what their profession is and people in general have some sort of clue what they are doing for living.

Me, in the other hand, if I tell my profession to my relatives, am faced with a puzzled smile and slightly confused glare. Which I can buy, nobody seems to know what this testing is and what it is all about. Getting more testing stories in popular culture would actually help a bit.

It would be interesting to know, if I’m wrong here. What I know, is the narrow field of popular culture I’ve been following. I might be completely wrong, which is always all so human.

By the way, did you know that the Chernobyl disaster  was caused by running tests? Some experiments, as it seems. Sounds slightly like exploratory testing to me. It’s always a refreshing thought when someone bashes around the nuclear plant systems.

Just one more: Who was the guy, who tested the Death Star particle exhaust vents security? And who approved the solution?

Note: And the answer comes from the  deeps of Twitter:

Thanks, @Marcel_Gehlen

PS. I did find the podcast, too.

Delivering a Hippo

When delivering software through the delivery chain, I’d like to refer to the software as a Hippopotamus. Previously I have heard Michael Bolton (Not the singer, or the guy from Office Space) refer to software (and the delivery process) and especially requirement handling by telling a story about blind men and elephant. I do have my reasons to switch the elephant to a hippo, which I then explain later on.
If you like elephants more than hippos, too bad.

The Story

There is a group of blind men. They get a task to describe what a hippopotamus looks like. They need to give the description to a man that will build a crate to transport the hippo to the wild.
The group is first gathered around the head. They describe the beast as a big headed creature that has a huge mouth and sharp, long teeth, relatively small eyes and small, but alert ears. The creature seems to also have a slightly bad temper. They measure the size of the head and give the description to the transport team.
After the description, the transport team builds a crate and the delivery starts.
They get the whole head to the crate, it fits perfectly. But then there comes something else.
It seems they’ve forgotten something.
At this point it’s about the time for me to reveal a bit on how do I see this. First of all the group of men, the blind ones, I saw them first as bunch of software developers delivering the latest features on maintenance project. But to be fair, the group should also contain some test engineers, requirement responsible (product owner or whatever the title is), project manager and so forth. For as far as I know, regardless on the role on the project, in this kind of scenario we people tend to get blind. And it is not related on this scenario. When we as humans concentrate on the problem at hand we have great capabilities on concentrating on just that and to forget everything else. Which is somewhat crucial. It is crucial for the delivery to be delivered as it should’ve been. It should also be crucial to focus on other things than just the delivery itself.
What the group of blind men has forgotten is that the hippo has it’s body. The body is bigger and wider than the head and therefore it has no place in the crate. However, if they want to deliver the head, they should also deliver the body. So they turn back to the hippo and feel the sheer size of it’s body.
They describe the quite huge creature to the  transport team. The transport team, for they are clever, will now build a crate to contain both the head and the body. Not just the body.
So everything should now be fine, or what do you think?
The team has now delivered the complete hippo to the transport team and managed to get it inside the crate. Sure, and everybody is happy. The main functionality is delivered with the maintenance package without huge amount of flaws in it. As it usually is.
But did they forget something? No, everything is neatly packaged and ready to be delivered.
Well, the hippo has the rear end. And as you know, hippos tend to mark their territory with their rear end. They are clever in that way that they spread their manure (aka hippo -shit) around by wagging their tail vigorously. It’s an effective way to spread the message. And there we have the slip through. To be honest It wouldn’t matter if the group of men were blind or not; slip through will eventually happen every now and then to the teams that are or are not visually impaired. Really, we’ve all been there.
What we can learn from here is few things. First of all, the roles of the team should be spread a bit. There should be at least one person taking care of the maintenance and regression. Whether that is a test engineer or a developer does not actually make any greater difference. It usually is enough that you verify that he regression should not happen and that the old stuff is delivered also. The other thing to consider would be to take one of the transport team to check the delivery.
And for the love of God (or whatever deity your world goes around – money, consuming, stock exchange, sex, rock n’ roll etc), don’t get me wrong here. I do not mean that we need more people to the team. I do think that more blind men see just as much as less of them. What I mean with the role is that we, humans, are capable of switching to different roles. Even during the workday. Regardless what I think about multi-tasking (I am strongly against, really, it does not work for me), I do think that same person that handles the delivery of the main functionality can also handle te delivery and verification of the maintenance delivery. And the verification of regression tests. Does not even have to be a developer to do that. Or QA, depending on from which side you look at it. There’s plenty of things we QA -engineers can learn from the developers and developers can also learn a thing or two from testers. Some of the test engineers are even capable of writing code themselves and most of the developers are capable of testing their (or someone else’s code) during the day. And after this is done, they can continue concentrating to deliver the main functionality. It really is as simple as that. Nothing fancy here.
So, now we’ve covered the main functionality and the regression. The head and the body are happily delivered. What to do with the tail?
First of all. We will always have slip throughs. Really, there is no way we can get rid of them. There is no way we can test and code in a way that will lead to a delivery and simultaneously not have anything weird hidden in the delivery. But of course we should do anything we can to prevent them happen.
What could be helping here was that there is a role in the team for experimental testing. Do something unconventional. Someone from the team should get to bash the hell out from the application, tease the boundaries, break it, create chaos. Do things users won’t do (that list is actually shorter than you think ;)). That will, eventually reveal some of the bugs that the more conventional testing won’t do. If possible, automate that and do it all the time. Just keep the chaos running.

Learning curves

It seems I have a road ahead. Not sure what is going to happen, but at least I know that I’ll enjoy the ride. The ride will be long and it will be hard at times, but all I have to do is to remember that I have all the time in the world for this and the target is not to create a perfect application, but to learn how to automate tests and develop with Python and Robot Framework. Both skills I see as needed in the future.

First of all, after everything was installed yesterday, I sat down and went through some tutorials. Both git & Robot Framework are getting more familiar now. Far from being perfect, but at least I’ve been using both. Git is, by the way, a lot more intuitive with the commands than CVS. And it really neat to use a service like GitHub to store my stuff in. At the moment the solution I started to lab with is stored as private in GitHub. In the future I plan to create a public variant on that in order to get it shared properly.

I also managed to start the Algorithms -project and created the first excercise. It is a simple insertion sort I managed to get together by help from stackoverflow (a really good site to search for information;)) and a slight use of my brain. I actually needed the stackoverflow -solution more or less as interpreter to the algorithm -books pseudo -code.

And last, but not least, I actually have now done my first time ever ATDD -approach on developing. The test case was created and it failed in the beginning. After some coding I managed to get it also go through and got a PASS as result. And the test case is of course using selenium webdriver -library to verify that the web -page is up and running.

As a bonus I actually got an idea about a solution on how to parse some input strings properly with groovy -script in SoapUI. A thing I will need to do at work next week anyhow.

So to say, it has been a proper weekend.

Hunt ‘em down

Mankind has been practicing hunting, let’s say, for a few years now. we’ve been capable to get our daily rations of protein, carbs and fat by hunting, finding, gathering, looting whatever has been available. And I ‘d like to think we’ve been rather successful on that, too.

Just think about it ; we’re still here, few other species have gone extinct and most of the current ecology –system is more or less constantly threatened by our existence. So I’d like to make a conclusion that we’re are pretty well adjusted for hunting down whatever it is to be down hunted.

So how come it is so hard to do with requirements?

I mean come on, people. There will always be the written down requirements, those are the easy part. Regarding that they are available for all software project participants. Which they might be, and then again, they might not. So let’s not concentrate on them at the moment. Let’s make an ass out of u and me and assume that the written down requirements are available, and what is most important, their whereabouts are known in the whole project for all the participants.

So QA starts their work basing their testing on the requirements and information available. No big deal. Bugs will be found, they will be fixed, life goes on.

But what about the stuff everybody just magically know should work? The packaging standards of the applications, the delivery setups, the way the applications should react to other applications in the system? What if those are de-facto standards, written down ages ago, known to most of the people involved, but just in this particular project there’s a bunch of newcomers.

During the initiate phase such things should be brought up and delivered to them, ain’t it so? sure it is. Well, that is a nice picture, but honestly people, it does not always work as that.

The truth (You can’t handle the truth!) is that we do  live in a perfect world. Yes, you read it right. It is perfect and understandable unambiguous and messy and in the end, it is beautiful to see this multitude of failures, flaws and not working things among the everyday life’s working stuff and understand that it all is part of the process.

Now where I’m going with this? Here; all you need to do is to take your stand and adept to the situation.

As a QA –engineer, your responsibility is to plan and execute test cases with certain expected results. You cannot say that you can’t do them, that you don’t have the requirements. Not if you get to know that you should’ve known. And for the record it is more than OK not to know.Like I pointed out yesterday, it does not matter if you did mistakes, or if someone else did them by not providing the requirements to your lap. What matters that you take a leap towards the newly found information and hunt it down.

Now you might tell me that it is project responsibility to deliver all needed information considered and bring it to you. Hopefully on a silver plate. I’d like that, too. And while they’re at it they could also bring the coffee and take care of my laundry.

Now here’s the crucial part, at least for me; The world as we live is perfect. It is perfect for human beings who are capable to hunt down all the targets they aim to do. And that applies to requirements in software projects. You might not succeed all the time, none of the hunters do, either, but who cares. The point is that you can hunt and that you can catch your prey. Let the prey here be information. Actually, if you’re reading this, your brains have been capturing at least some information on this page. Same applies to requirements. Regardless if they are written down or not.

The next question is, of course, what to do with them? Well, depending of the structure, you’ll need to get to the best parts of them. You might need to skin, peel or mush them. And then split them to chewable pieces. Some of them are nutritionally really good, some of them are completely unusable. Pick what you can use and leave the rest. The ones you picked up, you’ll feed up to your Test Case –creating ogre, called Untar the Defector. And you’re done. For now. Tomorrow will come and bring the new challenges, and your ogre is forever hungry.

Soundtrack for the post:Queen – Live At The Rainbow