Month: September 2017

Logstash filter for Robot Framework

/ jankka / Leave a comment

We are currently working on CI/CD -setup at work. As part of that, the tests need to be able to be implemented as a part of the pipeline.
Generally, the pipeline consists of steps/stages done with jenkins pipeline. The benefit on this is that the whole process and definition of the stages (Deploy, test etc) will be done by the developer team and stored in the teams own repository and is therefore controlled by the team also. Which is definitely a great step towards for the teams having more freedom and more responsibility when it comes to deliver the applications/solutions to the production. Needless to say it will also affect to the visibility of the quality and to the need of tests.

Plus that it will definitely keep the test team on their toes. Keeping ahead becomes a really neat challenge 😀

Now that does add more requirements also on the testing tools. First of all, the tools we use should be able to be used from containers. Which means that everything is dockerized. Well, the test code itself is in the repository, but the engines running the tests are in the containers.
We use, whenever we can, a general docker images from dockerhub.
Sometimes it won’t work like that. So we end up re-inventing the wheel.

That was the case with logstash. We will need to be able to filter the Robot Framework’s output.xml and send it to elasticsearch. There was two possibilities to do that; logstash filtering or xml parsing. The xml-parsing remains to be done still (I am going to do it), but I did manage to create the logstash -filter. It is not completely flawless, not even the most elegant, but at the moment it seems to be working as it should. To be honest, I was aiming to have a one more blunt instrument for our test needs.

The filter:

robot-results.conf

input {
 file {
 path => [ "/output.xml"]
 }
}

filter {
 xml
 {
 source => "message"
 store_xml => true
 target => "doc"
 xpath =>
 [

"msg", "doc.msg",
 "arguments", "doc.args",
 "kw", "doc.keyword",
 "status", "doc.status",
 "status/@status", "doc.test.status",
 "robot", "doc.robot",
 "errors", "doc.errors",
 "statistics", "doc.statistics",
 "suite", "doc.suite",
 "tag", "doc.tag",
 "total", "doc.total",
 "/kw", "leftovers",
 "/arguments", "leftovers"

]



}
}



output {
 elasticsearch {
 hosts => ["elastic"]
 index => "logstash-%{+YYYY.MM.dd}"
 }
}

Dockerfile:

FROM logstash

ADD robot-results.conf /etc/logstash/conf.d/robot/results.conf
CMD logstash -f /etc/logstash/conf.d/robot/

Running the container:

docker run --add-host=elastic:127.0.0.1 janmat/logstash-robot

 

Install Secure Lab Tools in Fedora 26

/ jankka / 1 Comment

I am working my way to dive in to the world of security testing. We have been going through the tasks on 30 days of security testing from Ministry Of Testing Dojo. The themed tasks is actually a really good way to keep up in learning new topics and deepening your knowledge on different issues at hand. Ministry Of Testing has a nice series of themed months on the catalogue and I warmly recommend to check them out.

We have been doing the themed months a bit differently. First of all, we accept the fact that there is weekends and people do not have to live, breathe and urinate testing. Even though it does help from time to time. So, our approach has been mainly to do 30 days of testing during the weekdays. Which means that instead of 4 weeks, we’ll accomplish it in approximately 6 weeks.

Anyhow, one of the things beside the security testing challenge has been us having a course on Ethical Hacking. The course is available in Udemy and it is reasonably priced, so I recommend that, at least if you’re not familiar with penetration testing and hacking techniques in general.

So, we go through tools and techniques and use Kali Linux for that. Which seems to be powerful to use. As I am running Fedora 26 on my workstation, I am running the penetration test stuff on Fedora Boxes (more stable than VirtualBox), but I noticed that it would actually be nice to have the tools on my actual workstation, too.

So I went and googled a bit and as I knew, someone had already solved my issue.  As I am using Finnish language on the laptop, my installation command was like this:

# sudo dnf groupinstall Turvallisuuslaboratorio

For most of the people who do not have the capability to understand Finnish, it would make sense to use something more, how to put it, understandable language, like English.

So, in that case I suppose the command should work like this:

# sudo dnf groupinstall security-lab

By the way, while writing this, I did write the Kali Linux on a USB disk. It actually feels better to have it there than fooling around with virtual machines (in this case). Even though I’ll have to reboot the computer if I want to run it.