Soak Testing 101: The Basics

by Yulia Lomanova | March 9, 2017 8:44 am

Note: the article was updated in July 2020.

The last thing you expect from a recently deployed application is its instant crash or an abrupt slowdown in the operational flow. Luckily, there is a testing type, developed specifically for ensuring the reliability of a service in production. Meticulously performed soak testing comes in handy when you need to understand how capable your application is of running smoothly under a regular load for an indefinite period. Ready to expand your QA expertise? This quick guide outlines the algorithm and software diagnosis potential for soak testing.

What is Soak Testing?

The soak test is part of performance testing, intended to identify defects in the software that may show up after an extended time. To confirm the stability of the system, it simulates the typical load in an accelerated way, meaning events are introduced to the system more often than they would be in a real-life scenario. The preferable testing duration is determined by the required time for the application to run seamlessly and dependent upon the availability of the test lab. Given the extended run time, this type of testing also calls for some sort of automation.

When you test a fully deployed application, it should cover both the application code and its interaction with the side software (the 3rd party application frameworks, embedded web servers, service codes, etc.) on which your product is built. It is also important to test the first release of the software, as its reliability needs to be confirmed.

What Types of Problems It Allows to Identify

• Slowly developing memory leaks;
• Unforeseen limitations in the application resource usage;
• Layer & Database Connections Failure;
• High or increasing CPU usage;
• Increasing error rate or degradation in the application performance.

Soak Testing Process Algorithm (for apps running on a JVM)

The first step towards the goal of performing a successful soak test is to set up and measure resource usage. Make sure you can connect your JMX client to the service’s remote JVM. Before starting the app, check the available resources and how they have been used by the existing processes. Record them to get a baseline of the minimum resource usage needs.

Given that soak testing requires automation, the only thing remaining under your control during the process itself is to check several performance indicators, including the current resource usage, “free” RAM on the node, RAM usage per process, and Docker container resource usage.

The soak testing results analysis includes error rates, node / container-level and JVM-resource usage metrics.

Soak Testing Vs Other Performance Testing Types

Even though load, spike, stress, and soak tests have similar architecture, their aims differ markedly.

• As we have already mentioned, a soak test focuses on establishing the long-term reliability of the application under the normal operation.
• In contrast, load testing establishes how its throughput varies under increasing load up to a required maximum.
• Stress testing measures the system stability and error handling level under extreme load.
• Last but not least, spike testing exposes the system to a sudden increase or decrease in the number of users and transactions. It aims to find out whether the software can handle intense changes in load and sustain extremely fluctuating conditions.

Let Us Help You!

We hope that this article helped you to understand the basics of soak testing. To get more information about Software Testing & QA, check out our services[1] or chat with us right away.

Related Posts:

• HP LoadRunner vs HP Performance Center[2]
• E-Commerce: What to Test during Load Check?[3]
• The Basis of Performance Testing[4]

Source URL: https://blog.qatestlab.com/2017/03/09/soak-testing-basics/

---