A Test Automation Blog

I think every test automation engineer will come at a point where you need to look at improving/optimizing your test automation script so it will take less time to run. But doing so is always tricky. Your tests should run as reliable as possible, at some point speed also becomes a factor in the equation. So what can you do, what works and which trade-offs should you avoid? Core reason for your TA suite The most important thing to keep in mind is that you should never give in on the core reason why you implemented test automation. If you only have a set of tests that run locally and the results are for you to interpret anyway, speed might be more important and a test is allowed to fail once in a while. But when your test suite is connected to a CI cycle and the results are automatically interpreted, it is very important to have super reliable tests instead. Obviously you want to have fast and reliable tests all the time. But time can only be spend once and is limited in most pr

Are you on a company network that is using a proxy to connect to the internet? And do you want (or are you planning) to connect your tests to an external selenium-grid. Than stay tuned, before you get lost in a bunch of answers on the internet that don't resolve the issue you're facing. Source:  https://nl.wikipedia.org/wiki/Proxyserver I'm on a new project and looking into Robotframework. But then I bumped into an issue with the POC I had to make which might result in me convincing the company to switch to Ruby. Anyway long story short. They use a proxy and although the internet has all the answers, you do need to know what specific question to ask to get the correct result. And since proxies are not used that much anymore, it's hard to find the skilled people to help you out and push you in the right direction for a solution. The setup To access the internet, I need a configuration on my computer that will tell Windows to connect to a proxy server. I

It looks like Google is still using this page to update us, see these  released new statistics . It has been a while so let's see what has changed over time. At least Android 9 (Pie) has quiet some market share by now. https://developer.android.com/about/dashboards/ Statistics, facts and trends Versions over time Facts about the Android versions compared to previous month: Not much changed, besides that, now more than ever, there is no clear dominant version There are still seven Android versions with more then 7% market share Android 9 (Pie) now has over 10% market share and will continue to grow Android 8.1 is still going up together with 9.0 Android 8.0 is decreasing  Android 7.1, 7.0, 6.0, and 5.1 are showing a steady decrease Android 5.0 and 4.4 are at the end of the cycle Codenames over time Facts about the Android codenames compared to previous month: Android Pie present and already on 6th place Android Oreo is at the top a

In case you're wondering why I haven't created a new blogpost about the Android statistics in a couple of months, it's because there are no new numbers shared yet. https://developer.android.com/about/dashboards/ What does this mean? Are they not updating this page anymore? Is there another source/page that could be used? Are they postponing new updates because Android Pie (9.0) does not have market share yet? All valid questions, but to be honest, I don't know the answers. They might be busy with a new page to share this information (they once gave me a hint they might be busy with that). But so far they have not shared this with me. So I'll poke around a bit more to see if they'll tell me or update the dashboard. If Android decided to stop sharing this kind of information, I'll switch to an alternative source of information. The one I have in mind is statcounter http://gs.statcounter.com/android-version-market-share/mobile-tablet/worldwide But if

As we all know, non-touch devices like a desktops have slightly different input from touch devices like mobile. This also means developers do have the option to build a website that does expect touch input instead of a mouse click. If your test suite does do a `element.click` when touch is expected, the click does not fail (at least not currently), so no error is thrown/shown but your script will fail. In one of my previous blog posts I mentioned a solution with execute_script and touchend . I did find out ( with the help of the watir community on Slack ), this can be resolved easier with Selenium and touch action (only supported on Chrome at the moment of writing). The specs In general spec documents are a bit difficult to interpret because of the lack of a clear example, but it gets easier to understand the structure over time. If you look at the specs of selenium-webdriver  you get an overview of all possibilities. There are a couple of points you could look if you think abo

Another month past, Google  released new statistics , so let's find out what changed compared to previous month. Is Android 9 (Pie) finally showing up in the statistics? How many Android versions do you need to support with your app? Let's take a look at the graphs and find out. https://developer.android.com/about/dashboards/ Statistics, facts and trends Versions over time Facts about the Android versions compared to previous month: The landscape is still a jungle of versions with no outstanding dominant version and next month the gap between all will be even closer There are seven Android versions with more then 7% market share Android 9 (Pie) is not showing up yet (maybe next month) Android 8.1 and 8.0 are the only ones with an up trend Android 7.1, 7.0, 6.0, and 5.1 are showing a steady decrease Android 5.0 and 4.4 are at the end of the cycle  Codenames over time Facts about the Android codenames compared to previous month: Androi

A picture sometimes says more than words. This is also true for test automation. Therefore it could be that you want your framework to make a screenshot when a scenario is failing to make debugging more easy. But how can you make sure your framework does this for you? Since we're using the `lapis_lazuli` gem, this is actually already been taken care off for most scenario's, but if you're using the 'watir' gem, you need to modify your `hooks.rb` file like briefly described on this page . Let's first look at Watir and then move to LapisLazuli to see what advantages and disadvantages are between the two. Watir screenshots Watir by default does not make a screenshot of a failing test, so you need to write some code to make this work. Below a Watir example Keep in mind that this only makes a screenshot at the end of the scenario. Meaning that it can happen that the screenshot is being taken a fraction later and thus missing the real issue. So let's t

Google forgot to update the pie chart picture. Is that because they wait for Android Pie (Android 9.0) to show up in the statistics? Or is it already showing up? Google  released new statistics , so let's find out what changed compared to previous month.  https://developer.android.com/about/dashboards/ Statistics, facts and trends Versions over time Facts about the Android versions compared to previous month: The landscape is still a jungle of versions with no outstanding dominant version Android 9 (Pie) is not shown yet Android 8.1 and 8.0 are the only ones with an up trend We see a strong uptrend which might be related to the fact that more phones provide OS upgrades these days Android 7.1 has reached it's peak and is now heading downwards Android 7.0, 6.0, and 5.1 are showing a steady decrease Android 5.0 is at the end of it's cycle and is slowly becoming irrelevant  Android 4.4 is still showing a steady and is heading towards the 5% marke

Our test automation suite for a single page application (SPA) called service-hub , takes almost one hour to run all +- 200 scenario's in serie per browser. The amount of scenario's will keep on growing with the expansion of the product. So I needed to start using some clever tricks to speed up the process. One of those tricks is by testing in parallel sessions. This is a very good thing to figure out anyway, but is also taking more resources (and therefore cost more money). Another trick is by using the API for some of the preconditions that take a lot of time. But for that we need a token which is stored in the browsers Session Storage. Scenario example For the service-hub product which I'm testing, it takes precious time to add/remove a material from an activity if I need to do this all via the UI. There are many steps to take to make sure I'm on the correct page, tab, breadcrumb, sub-tab and finally the element and to cover all other scenario's that might