We’ve already told a bit about the software testing hierarchy. There are four levels of software testing: unit, integration, system, and acceptance. Each of these levels examines software at a certain stage – when separate components are coded, when they are assembled into features, and when features get united into a system, etc.
System testing comes after a QA team has checked the smallest functional blocks and their interactions. This time, we decided to focus on this type of testing with characteristics and particularities that make it essential.
System testing is the inspection of a fully integrated system. The purpose of this checkup is to verify the system's conformance with both functional and non-functional requirements.
In the course of system testing, a QA team can notice the following defects:
Misuse of system resources
Unexpected combinations of user-level data
Incompatibility with the environment
Unexpected use cases
The inconvenience of use, etc.
System testing uses the black-box method. It means that a QA team doesn’t know how the backend works. They aren’t aware of the internal structure of software and check the features using frontend – the ‘facade’ of the program. This allows modeling user behavior closely since users interact with software in the same way.
The goal of system testing is to minimize the risks associated with the behavior of the system in a particular environment. For this, testers use the environment as close as possible to the one on where a product will be installed after the release.
The objectives are some smaller steps that allow achieving the goal. In system testing, there are several milestones that make the release of a flawlessly functioning system (read: software without critical bugs on production) possible. So the primary objectives are:
Reducing risks, for bug-free components don’t always perform well as a system.
Preventing as many defects and critical bugs as possible by careful examination.
Verifying the conformance of design, features, and performance with the specifications stated in the product requirements.
Validating the confidence in the system as a whole before moving to the final stage – acceptance testing that takes place right before users get access to a product.
System testing follows integration testing. The latter checks how several units, the smallest functioning parts of code, work together. So, integration testing helps the team to prepare for system testing, since it inspects blocks of units that are later united into a finalized build.
System testing goes further compared to integration testing. It inspects how all the units act after they become a wholesome system. As usual, the fact that units work as supposed after integration doesn’t mean they will function the same way in a system.
A test basis is the source of information that contains all the requirements for the software product under development. In other words, it is the documentation a QA team uses to draw conclusions on whether the product turned out to be the way it should be.
The basis for system testing should clearly define the criteria for features and performance. It should be well-structured so that finding relevant data for test case creation doesn’t cause any difficulties.
There are several documents that can be used as a testing basis, including:
Functional and non-functional software requirements specifications (SRS) that provide complete guidelines on how the system should work.
Risk analysis reports that outline the areas of risk so the team can prioritize the tasks and understand the outcomes of these risks.
Use cases that describe user flow.
Models of system behavior that verbally describe the processes and activities all the components are involved in.
User manuals that explain how to use the software.
There are two approaches to the system testing – one based on the requirements and the other one based on use cases. The former type uses project documentation as a set of main criteria for the inspection, while the latter aims to closely mimic user behavior.
Depending on the side that runs testing, there are alpha and beta testing. Both aim to find the bugs so that a dev team can fix them before the release.
Alpha testing is the checkup by a QA team in the testing environment and can require long execution cycles.
Beta testing is a task for other departments or people outside the company. It is focused on the user perspective and uses the majority of insights for improvements in the future.
When it comes to the types of system testing, you may need more effort to remember all. Actually, there are more than fifty types of system tests that cover both functional and non-functional aspects.
You can find detailed information on some of them in our other blog posts. For now, here’s a short list with a short explanation to get familiar with the basics.
Functional testing focuses on software features and checks all the functionalities to verify they match the requirements.
Performance testing examines software behavior in different environments and under different conditions. It features load, stress, spike, recovery, and other types of tests that estimate system response, speed, and stability.
Graphic user interface (GUI) testing checks the interface – all the visual elements customers use to interact with the system. At this stage, the team checks buttons, input fields, text rows, etc.
Usability testing estimates the overall convenience of using the program, taking into account both practical and emotional factors.
Compatibility testing is used to verify that a program, app, or other types of software we are dealing with is supported across different devices and browsers. It is extremely important for mobile devices that are available in such an abundance.
Localization testing is the adaptation of software for different markets. It involves more than translation, covering also metric system, date formats, text input and methods, etc.
Compliance testing is the checkup meant to verify that the software meets all the legal requirements. This type of testing is extremely important for healthcare and banking apps since they deal with sensitive data and imply strict governmental regulation.
Security testing is the way to make sure the app is safe to use and well-protected from hackers and scammers. It is essential for every application that requests personal data or offers verification via Google, Apple, Facebook, or other social accounts.
Regression testing checks an untouched part of the functionality after a dev team makes some code changes since they can be affected by accident.
These are only several basic types of system testing required by different kinds of software products and provided by most QA companies. The final list for your product will depend on its specifications and your expectations.
If explained in detail, step by step system testing process requires one more post. Here’s a quick outline – system testing in a nutshell – to help you get the algorithm of this process.
First, you need to create a test plan. After that, write the test and use cases. The next step is to create test data used for system testing. If you apply automation on the project, start with it. Then, run manual tests that cover the rest of the functionality.
The testing results are used to generate a report that goes to the development team. They are the ones who are going to fix bugs. Testers only run the inspection and suggest improvements. After the errors are fixed, run regression tests to make sure the untouched functionality hasn’t been affected. This last step can repeat several times until no critical bugs are left.
System testing is the third step in the hierarchy of software testing levels. It comes after integration testing and reviews a system as a whole.
During system testing, we work with a software build that is potentially ready for release. As a rule, there are still critical bugs that deteriorate user experience. The main task of a QA team at this stage is to find as many errors as possible so that the team can move on to acceptance testing – the next and final level in this hierarchy.