Did you know that in order to get the desired product at the release, its creators sometimes have to repeatedly change development approaches? This is confirmed by a study of Thomas Allen, which reflects the need to redesign the aerospace subsystem before it becomes operational.
The same happens with any software product, no matter how predictable the initial requirements for it are.
Thoughts on this phenomenon made us come up with this full-length article on acceptance testing – testing a product at the stage right before its market introduction.
Acceptance Testing Definition
Acceptance testing (also referred to as “user acceptance testing”) is a type of testing that, as we noted above, occurs at the stage of the very delivery of the finished product (or its finished part) to the customer base. The purpose of this procedure is to determine the readiness of the product for release and launch into public use.
Thus, compliance with all customer requirements is achieved by passing test scenarios and cases, which are composed based on the approved specifications of the developed software.
Acceptance testing results in two final stages:
- project (or its separate parts) polishing;
- project delivery.
Note that acceptance testing usually doesn’t cover all aspects of the created software – mainly, only the basic functionality is tested (because it is in relation to it that the specifications in the technical documentation are usually described).
Acceptance testing is conducted either by a customer himself or by a group of testers who are not part of the main development group. It is believed that using the human resources of a development company is not very correct, especially when it comes to a complex product or startup.
Acceptance vs System Testing
Often, those who are far from the terminology of testers confuse the concepts of acceptance and system testing. Let’s see the differences between them.
In fact, system testing is done before any acceptance procedures. At this point, testers check functional and non-functional requirements for the product, identifying bugs like:
- irrational use of system resources;
- incorrect data entered by the user;
- unintended user scripts;
- missing/incorrectly implemented functionality;
- lack of intuition, etc.
What Are the Benefits of Acceptance Testing?
The advantages of acceptance testing are quite obvious: early detection of system inconsistencies, as well as shortcomings that affect usability and intuitiveness.
Thus, acceptance testing makes it possible to deliver the product for release competently, efficiently, and without disrupting the deadlines in accordance with the customer’s expectations.
How to Do Acceptance Testing
Let’s review the usual plan for acceptance testing. In essence, this plan is a document where you outline an algorithm of actions for testers.
It usually includes:
- global goals of testing;
- application area of the software;
- terminology and abbreviations (these are required for proper plan interpretation);
- links (basically, a set of all documents involved in the acceptance plan).
The document begins with an overview. This is a small section describing what is contained in the main body of the plan. This is followed by a description of responsibilities – both on the part of the developers who prepare the product for acceptance testing and on the part of the product owner.
Now comes the main part. The objective criteria for product acceptance should be described here. They must first be officially approved by the customer and the development team and accompanied (if necessary) by physical artifacts, which must also be approved by the product owner.
At the same time, testers will have to test these artifacts as well. Typically, this procedure includes documentation analysis, software testing, compatibility with other software, etc.
This is followed by a schedule of acceptance testing tasks with an indication of the timeframe, resource and hardware requirements (processor power, firmware, etc.), software requirements (availability of drivers, test data generators, etc.), as well as requirements to the testers themselves (you may need the help of experts in the product’s niche).
The next stage involves the description of potential problems and their solutions.
Lastly, you will need to describe the algorithm for setting up the environment in which the acceptance testing will take place and provide specific tools, techniques, and methodologies that should be used in performing product acceptance activities.
When is Acceptance Testing Needed?
Acceptance testing is in place when:
- a product reaches the required level of quality and is supposedly ready for release;
- a product owner approves the acceptance testing plan.
The whole affair should be taking place until:
- a product owner approves the solution for release;
- all the flaws are identified and the product itself is sent for polishing.
Learn more about the development cycle ins and outs in our other feature.
Types of Acceptance Testing
There are the following common types of this procedure:
- alpha and beta testing. In alpha testing, users are usually simulated by the development team or in-house testers. When it comes to beta testing, however, real users gathered in focus groups are involved;
- contractual acceptance testing. This type of testing helps check whether the developed product meets the requirements approved by the product owner in order to understand how well the developers do their job;
- regulation acceptance testing. This one helps ensure that a product meets its industry specifics: legal regulations (such as GDPR), technology charters, standards, etc.
- operational acceptance testing. This type of testing demonstrates how safe and effective a product is designed to be suitable for real-world use;
- black box testing. Lastly, black box testing is focused on “blind” testing of the product’s functionality (that is, when the user has learned how it works and begins to interact with the UI intuitively, without anyone’s suggestions).
The Process of Acceptance Testing
The whole process consists of the following major stages.
Product requirements analysis
As we already mentioned, before moving on to product development, you need to take care of the documentation, which will indicate its specifications. You will also need business cases and system requirements specifications. Only then you can start developing test scenarios and start creating the plan that we wrote about above.
Creation of test scenarios and cases
At this stage, you will need to think about specific tasks for alpha/beta testers teams and prepare instructions for getting to know the software itself.
Collection of test data
You will also need to plan a process for collecting test results. Usually, bug tracking systems like Jira are used. In addition, at this stage, you have to make sure that the personal data entered by users will always remain confidential (we are talking about the GDPR policies that apply in the EU countries).
Execution of test scenarios
During this phase, your testers team will interact with beta testers focus groups. It happens as follows. Beta testers run their test scripts. Then, upon reaching the exit point, the resulting user experience is evaluated based on the predetermined criteria.
In the event that most of the value judgments are positive, the QA-testers and beta-testers teams move along the acceptance testing plan further until the moment the product fully meets the owner’s expectations and it can be launched into release.
Unless this happens and the product owner approves the existing version of the product, the developers have to redo it in accordance with the testers’ comments.
Despite the fact that by the time of acceptance testing the software product should already function normally, during the execution of the plan, QA testers and beta testers may encounter unforeseen problems. If this happens, you will need to pause testing and resume after troubleshooting.
Approval of compliance with specification or initiation of product polishing
As soon as you receive positive feedback from beta testers, you will need to prepare documentation that will serve as an official signal for the release of the developed software product.
Such documentation includes:
- an acceptance testing plan in discussion;
- test scenarios and cases;
- beta testing results;
- a separate document that defines all bugs, crashes, flaws, etc. identified during product development and testing.
All of these points are reviewed at a specific acceptance testing meeting where all involved parties should be present, including the product owner.
This meeting, in fact, confirms the performance of the product in its real environment of use, the high level of its UX, and also tacitly guarantees its competitiveness in the existing market conditions.
Success Factors
In fact, the actual value of a software solution is determined precisely by its target audience – their expectations, the degree to which their problems are solved, ease of use, availability, etc.
Indeed, without considering how end users will interact with your software or without realizing what they really want from the software, you run the risk of delivering completely unnecessary features or choosing a completely irrational way of solving issues of your target audience.
Vice versa, by timely going for acceptance testing, you can ensure that all your product efforts are made with the end user in mind. In addition, it will help you make the development process more rigorous since each change or improvement will be accompanied by a question: “Will the end user be able to use this feature and does he/she need it at all?”
Finally, a positive acceptance test result will confirm not only that there is a market for your product but also that consumers within that market will be able to use it effectively without any extra guidance (that is, on their own, using system hints at the most).
Conclusion
As you can see, acceptance testing provides guarantees that the finished product will meet the customer’s expectations so that you don’t blindly rely on the objective vision of developers.
That’s why it is very important to trust this event to a separate team of specialists in testing and QA. If you are interested in the services of just such specialists, contact us to get a high level of service at reasonable costs and transparent terms.
We also recommend learning at what stage of product development it is best to involve testers as a whole.
