test automation

Day 4 (of 30 Days of Automation in Testing)

/ jankka / Leave a comment

Day 4: What kind of testing can automation support you with?

Unit testing comes to my mind as the first one.It helps me as a tester in a way that I let developers do the deeds and learn to trust they do things right. Even reviewing the unit test code is actually a neat learning experience, once you get to understand the unit test logic.

Functional testing during or right after the build, can also be done by same unit testing approaches, which is also a neat way to verify the functionality. The benefits on that one is that the code is readable by the developer who wrote it and (in case it’s not made with Perl) even with other members of the team or company. To have the tests usable and readable in the same IDE developers use, leads also to easier debugging. What usually lacks is decent reporting, but then again, all we need to know that it fails when it should and passes when everything really is OK.

Mocking the test environment and harnessing the SUT. They walk hand in hand. There’s plenty of nice mocking tools, wiremock for example. One might also re-invent the wheel here. I mean the mock can be just a file you fetch from another system or write yourself. Does not sound too much automating, though 😀

Load testing is also one of the standard ones to be used with automation. Given the powers of the servers, new container technologies and distributed load testing services, you can build your own solution or use a ready made service, like blazemeter or octoperf.

Repetitive functionality tests, regression tests and release tests in any step of the development process are good to automate, given that you are going to use the automation more than few times.

Then the whole world of Continuous Delivery and Continuous Integration. It will not happen properly if you do not have the right tests put in right places. Well it might be OK to let the users test the product, but that would mean that you should have an automated monitoring system that gathers the data you will need. That’s also one way of automation that will help.

And of course all of the test data generating tools, self-made or otherwise, bash- or python -scripts for setting up the environments together with containers and their managing tools (docker, docker-compose and kubernetes to start with).

I most likely am missing something here, still. I mean where do you draw the line when it comes to automation in testing? Is it only the testing tools (like jMeter, Robot Framework etc.) or the tools that let you yourself automate mundane tasks to ease your testing process.

 

 

Advertisement

30 Days of Automation in Testing

/ jankka / Leave a comment

Once again it is summer, once again Ministry of Testing comes up with interesting 30 days challenge.
This year I spent my vacation already in May, so that excuse for not doing this won’t do; I’ll have to tackle this one.
The challenge is about test automation, sponsored by SmartBear (I do not like their products, but what can you do except not to use them 😉 ).

Day 1: Look up some definitions for ‘Automation’, compare them against definitions for ‘Test Automation’

Whereas automation definition in Wikipedia (the source of the sources) is actually handling a wide range of different mechanical and programmatical approaches, Test Automation is considered only to be software related and listed as part of software development. I do consider that one could test things mechanically and control the checkers, tools and results by machine, at least when it comes to simple checks. Wherever you’ll need human interpretation, it is actually easier to get a fellow human to make a judgement. Machines are good to make repetitive tasks easy, but they are also well capable to repeat an erroneus step several times automatically, unless someone controls the situation.
But anyhow, the biggest difference between automation and test automation according to wikipedia is the narrowing of test automation on something that’s done with software.

Day 2: Begin reading an automation related book and share something you’ve learnt by day 30.

Agile Automation and Unified Functional Testing by Rajeev Gupta

I’m also trying to make it through this course:
REST API Automation testing from scratch-(REST Assured java)

Day 3: Explore the automation thread on The Club and contribute to the conversation

Added my contribution to https://club.ministryoftesting.com/t/when-to-start-automation/11743/10

Delivering a Hippo

/ jankka / Leave a comment
When delivering software through the delivery chain, I’d like to refer to the software as a Hippopotamus. Previously I have heard Michael Bolton (Not the singer, or the guy from Office Space) refer to software (and the delivery process) and especially requirement handling by telling a story about blind men and elephant. I do have my reasons to switch the elephant to a hippo, which I then explain later on.
If you like elephants more than hippos, too bad.

The Story

There is a group of blind men. They get a task to describe what a hippopotamus looks like. They need to give the description to a man that will build a crate to transport the hippo to the wild.
The group is first gathered around the head. They describe the beast as a big headed creature that has a huge mouth and sharp, long teeth, relatively small eyes and small, but alert ears. The creature seems to also have a slightly bad temper. They measure the size of the head and give the description to the transport team.
After the description, the transport team builds a crate and the delivery starts.
They get the whole head to the crate, it fits perfectly. But then there comes something else.
It seems they’ve forgotten something.
At this point it’s about the time for me to reveal a bit on how do I see this. First of all the group of men, the blind ones, I saw them first as bunch of software developers delivering the latest features on maintenance project. But to be fair, the group should also contain some test engineers, requirement responsible (product owner or whatever the title is), project manager and so forth. For as far as I know, regardless on the role on the project, in this kind of scenario we people tend to get blind. And it is not related on this scenario. When we as humans concentrate on the problem at hand we have great capabilities on concentrating on just that and to forget everything else. Which is somewhat crucial. It is crucial for the delivery to be delivered as it should’ve been. It should also be crucial to focus on other things than just the delivery itself.
What the group of blind men has forgotten is that the hippo has it’s body. The body is bigger and wider than the head and therefore it has no place in the crate. However, if they want to deliver the head, they should also deliver the body. So they turn back to the hippo and feel the sheer size of it’s body.
They describe the quite huge creature to the  transport team. The transport team, for they are clever, will now build a crate to contain both the head and the body. Not just the body.
So everything should now be fine, or what do you think?
The team has now delivered the complete hippo to the transport team and managed to get it inside the crate. Sure, and everybody is happy. The main functionality is delivered with the maintenance package without huge amount of flaws in it. As it usually is.
But did they forget something? No, everything is neatly packaged and ready to be delivered.
Well, the hippo has the rear end. And as you know, hippos tend to mark their territory with their rear end. They are clever in that way that they spread their manure (aka hippo -shit) around by wagging their tail vigorously. It’s an effective way to spread the message. And there we have the slip through. To be honest It wouldn’t matter if the group of men were blind or not; slip through will eventually happen every now and then to the teams that are or are not visually impaired. Really, we’ve all been there.
What we can learn from here is few things. First of all, the roles of the team should be spread a bit. There should be at least one person taking care of the maintenance and regression. Whether that is a test engineer or a developer does not actually make any greater difference. It usually is enough that you verify that he regression should not happen and that the old stuff is delivered also. The other thing to consider would be to take one of the transport team to check the delivery.
And for the love of God (or whatever deity your world goes around – money, consuming, stock exchange, sex, rock n’ roll etc), don’t get me wrong here. I do not mean that we need more people to the team. I do think that more blind men see just as much as less of them. What I mean with the role is that we, humans, are capable of switching to different roles. Even during the workday. Regardless what I think about multi-tasking (I am strongly against, really, it does not work for me), I do think that same person that handles the delivery of the main functionality can also handle te delivery and verification of the maintenance delivery. And the verification of regression tests. Does not even have to be a developer to do that. Or QA, depending on from which side you look at it. There’s plenty of things we QA -engineers can learn from the developers and developers can also learn a thing or two from testers. Some of the test engineers are even capable of writing code themselves and most of the developers are capable of testing their (or someone else’s code) during the day. And after this is done, they can continue concentrating to deliver the main functionality. It really is as simple as that. Nothing fancy here.
So, now we’ve covered the main functionality and the regression. The head and the body are happily delivered. What to do with the tail?
First of all. We will always have slip throughs. Really, there is no way we can get rid of them. There is no way we can test and code in a way that will lead to a delivery and simultaneously not have anything weird hidden in the delivery. But of course we should do anything we can to prevent them happen.
What could be helping here was that there is a role in the team for experimental testing. Do something unconventional. Someone from the team should get to bash the hell out from the application, tease the boundaries, break it, create chaos. Do things users won’t do (that list is actually shorter than you think ;)). That will, eventually reveal some of the bugs that the more conventional testing won’t do. If possible, automate that and do it all the time. Just keep the chaos running.

Re-evaluating and bouncing

/ jankka / Leave a comment

I managed to actually re-evaluate my plans and combine them to my skill-set. There is a slight cap there. I can now see, that the skills of my programming are not near the needs of the project. Now that means that I’ll have to build the resources.

This means, of course, that the Algorithm -training is going to be continued, but that it is not enough. In fact I decided to get back to basics and go through two topics.

  1. Programming in Python 3
  2. Twisted tutorial (and everything related)

So getting those 2 things done will, eventually, get me closer to my ultimate goal: to become an efficient technical tester (A Test Toolsmith, some might say).

Now earlier, in this kind of projects, I’ve let myself sidetrack myself. Pretty easily also. So in this case, I’ll have to learn myself some discipline and perseverance. What I mean is that no matter if I get the ideas on how to develop the project I have in mind (being the messaging answering machine or my Crossfit whiteboard), I should only write the ideas down and get back to them after I’ve finished the ‘course’.

Now the learning should not take too long. Honestly put, I have had programming education in Java and Python. Java -education took place long time ago, just before I was hired at my first Test Engineer -job. The Python training, in the other hand was only few years ago. So in that way re-learning the stuff (and adding more, the course was just a basic one) should not take too much time. And this time I just have to make it all the way. The benefit on having this done at home, as my private project, is that I can also implement my skills right away after I’ve done the learning. Lately the usage of my learned skills has been more or less nothing. I know it is matter of determination at work, too, but previous tasks at work has kept me so busy otherwise that the usage of any coding capabilities has been more or less closer to the zero. Now that also seems to be changing, hopefully for good.

Besides Python, I do have a sneaky feeling that I should learn a bit more groovy. Which actually means that I should refresh also the java -skills. Not the worst idea either. Maybe after the Python is done I can concentrate more on groovy. Besides, we do use groovy -scripting when using SoapUI at work, so that one comes also naturally by the work, too. The other day I managed to even fetch a file from FTP to my desktop. Woohoo 😀

Last, but not least is that I do still have my project plans. To create something useful by using Python, Robot Framework and ATDD -methods to test and create everything. I just need to build my wheels first. Slow and steady should eventually win the race.

Building up the competence

/ jankka / Leave a comment

After being back from vacation has been neat. At least what came to the office atmosphere. The first week I worked almost completely from home, week after that there was almost nobody at the office, so I could really work easily by myself. Now I notice that the sounds of the keyboards rattling and mice buttons clicking keeps bugging me off. Not sure why, though. Usually I get the feeling when I’m tired. And now it’s monday and I won’t admit being tired after nice and refreshing weekend. Luckily there’s Spotify and earphones. 😀

During the holiday I was more or less online all the time, which can be seen as a good or bad. But as my friend has told me, everything has its good and bad drawbacks. I myself found the holiday really good and refreshing. Both on personal and professional level.

As you might know, I’m working as Test Engineer in QA/Test consultant company. The thing is that I got a phone call and an email during my first holiday week telling me I got an interview with a potential (and really interesting) client next monday, if it just suited my plans. Well, it did. I went to the interview and ended up having a test exam delivered to me the next day. Not completely unexpected, for they were looking for test developers.

I’ve been in  a similar situation before, last year at Spotify and that did not go too well. Or at least I found something I was not capable of doing. Regardless on the training I’ve taken and the education I have, my development capabilities were far from perfect. I did my best ( at the time) and came up with solution made with bash -scripts.

This time, though, I decided to really give it a shot with a decent language. I did have my Python -training   last year, for test automation purposes the language feels still to be really good one. Sadly I did not have any possibility to start using it at my previous work. Well, I might have could, but I never did even try. Besides that I did stick up with my old habits (and bash -scripts), which really rarely leads to anything new and exiting. 

So I went and jump into Python. Ended up doing some Python -related stuff few hours per day for around 10 days. With some breaks, for it still was my holiday 😉 It basically meant I did work around 3 days with Python and ended up with one solution. Then I got back at work and noticed I could make it a little bit better before delivering to the client. Which I did.

Now that the version is delivered I can see it’s not perfect. They rarely are. So at least I’ve got something to work with still.

This all has been really interesting. Getting inside the development, even for test purposes, has been an eye opener. More or less it seems that this has been the one tool missing from my case. So in that way I really do understand using my own time on learning and getting inside Python. For now on it will definitely be easier for me to adapt to that kind of situation. Besides that, now I can keep up learning myself more and more. I suppose that is the best way to do it. To build up and maintain the competence while I still can. My target is now to keep on developing something with Python daily. The same thing I’ve had my writing; I’ll need to create a new habit out of it. By that way I’ll keep up my touch to the language and hopefully will get my competence built up more and more every day. Well at least weekdays. Weekends are still worth to be used with family 😀

Now I’m not saying people should use their own time doing job -related stuff, and I myself don’t see I was doing it like that. I just used the work as an excuse for getting an uplift for my own competence. And (you might argue with this) my competence is not made only for work purposes, it is something I can use in my everyday life, somehow, too. Even if I won’t sit down and code I can learn myself how to get to know new stuff and that won’t hurt in the real life situations either.

Now, few tips:

  1. Eclipse & pyDev is a brilliant couple. I did myself use ADT Bundle for Eclipse for we need Android development environment for test automation purposes.
  2.  Stackoverflow has been a brilliant source of information when in need. And I’ve been using it a lot 😀
  3. In case you need to add plot graph on your web site (I need to) it seems matplotlib could do the trick.  Here’s how to get it installed in OS X Mountain Lion
$ ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
$ brew doctor
$ brew install python
$ export PATH=/usr/local/bin:/usr/local/share/python:$PATH
$ easy_install pip
$ brew install gfortran
$ pip install numpy
$ pip install scipy
$ brew install pkg-config
$ brew install freetype
$ brew install libpng
$ pip install matplotlib
$ python
>>> import numpy
>>> import scipy
>>> import matplotlib