When is test automation helpful, when does it lead to unnecessary overhead?

These are precisely the questions we will answer in the following scenarios.

As the name suggests, test automation is not usually executed manually by testers. On the one hand, this has the advantage of saving resources, time and money, and on the other hand, errors caused by manual activities, such as typing errors, can be excluded from the outset. Compared to manual testing, this allows many tests to be performed in high quality and in the shortest possible time.

Scenario 1 – Test-Driven Development Approach:

A classic scenario in which test automation not only makes sense, but also imposes itself, is an implementation project with test-driven development. This development method dictates that the tests must already be determined (and written) before the actual implementation is started. Only then the development is started and after each iteration the predefined tests are executed. Many iterations can occur here, especially with complex problems. In order not to let the test efforts explode, an automation framework is advantageous.

Scenario 2 – Regression Test:

Regression testing refers to test cases that must be performed when a change has been made to the software. You make sure that the already existing functions have not been damaged. This is especially important when processes are linked and/or build on each other.

Automation makes sense here if the basic functionalities of the software are no longer changed, so that the regression test cases do not have to be rewritten each time. In particular, predefined functional tests can significantly reduce the workload of the specialist department.

If test automation is considered, the number of regression tests as well as their execution frequency should be considered. It is worth making a cost-benefit calculation here. Basically, the more often a test case needs to be executed, the more time is saved by test automation. However, as long as there is no basic outline of the software, automation makes less sense because the adaptation effort for the test cases is too high.

Scenario 3 – Establishment of technical infrastructure:

At the beginning of a project, technical resources may already be available, although implementation has not yet taken place. This may be the case if the requirements are not yet known to a sufficient degree. In this case, it is a good idea to use the free resources for the creation of a technical infrastructure. This also includes test automation. From our own experience, a simple test automation at unit test level can be written in as little as one month and significantly reduce the upcoming test efforts.

In addition to the availability aspect, it is also helpful if the system structure has a clearly defined input and output layer that can be controlled by an automation framework. can be controlled by an automation framework.

Scenario 4 – complex system:

If a complex system is present, test automation may be unsuitable.

An example of this would be a system that works with many data stores that change frequently. Thus, the input and output layers cannot be clearly defined and the framework would have to be constantly adapted. This could lead to a loss of time, since the framework itself must also be tested when changes are made.

Conclusion

We recommend analyzing and defining exactly which of the scenarios or which mix applies at the start of each project. If you are already familiar with such a scenario or mix, test automation will significantly reduce your workload in the future! However, it is not a panacea that solves all testing problems. Similarly, not every test automation tool fits every infrastructure and every project.

ADWEKO’s experts support you both in deciding whether automation makes sense and whether you should automate completely or partially, and in selecting a suitable test automation framework and setting up the test automation project. Feel free to contact us!