All-around, timely testing has always been a major pillar of any successful software solution. Many market players, however, still miss out on segmented testing approaches, thinking that a single ordered package of testing services will do just fine. In particular, some businesses don’t differentiate performance testing from all other testing activities. Thus, by ignoring pre-launch performance testing, companies end up losing users and experiencing significant downtime and bug fixing pitfalls. For instance: Apple's 12-hour break in March 2015 cost the company $25 million; As a result of the 14-hour Facebook’s performance downtime in March 2019, the corporation lost about $90 million. Indeed, the cost of downtime and the financial impact of incidents can be too harmful even for large companies and major market players, to say nothing of medium- and small-sized businesses. According to the 2014 Gartner study, the average cost of business downtime is $5,600 per minute. At the same time, the Avaya report of the same year showed that financial losses during business interruptions due to technical problems ranged from $2,300 to $9,000 per minute, depending on company size and industry. In the late Gartner report (2016), the average downtime statistics rose from $5,600 per minute to $9,000 per minute. For small businesses, losses for 1-minute downtime can amount to $427 or more. For medium- and large-scale companies, the number can reach $9,000. What is Performance Testing in Essence? What Are its Goals? Performance testing is a set of testing procedures that emulate user traffic and requests made within a software solution. In other words, a website or application is tested in conditions as close as possible to the real market environment. Performance testing allows you to compare the expected results with the obtained indicators, determine the speed of operations in the application, as well as reliability, stability, and scalability of the system as a whole. As a result, you are able to: identify the improvements needed to introduce the product to the market. guarantee that the software works properly, the speed of loading pages or sections is minimal, and the increase in the number of active users doesn’t hinder software performance. reveal inconsistencies in different operating systems. The usual performance testing process is based on the following indicators: acceptable load rates; stress loading; response time; scaling potential or definition of the high productivity boundaries when it comes to a sharp increase in the number of users simultaneously interacting with the system;   a volume and types of existing performance errors.  These are optimized via such procedures as: load testing; stress testing; performance tuning; volume testing; bug fixing. Important to know Any loading delay should take no more than a few seconds. Slow download speeds spoil user experience and result in lowered conversion rates. Long wait times significantly botch user excitement about using your application as a whole. A 1-second page loading delay results in a 7% decrease in conversion rates, an 11% decrease in total page views, and a 16% decrease in customer satisfaction with the app or website. Source: Forbes For instance, a website generates $100,000 per day in revenue. With a measly 1-second delay in loading speed, its owner loses $2.5 million annually.   Source: truconversion.com Limited scalability also causes many issues when large numbers of users need to interact with the software. The app works fine with a small number of concurrent users, but it may degrade with a lot of logins. What Are the Benefits of Performance Testing? Software performance is a key factor in successful introduction of the solution to the market and positive user experience. The results obtained through testing allow us to identify vulnerabilities in the application and prevent them before the product is launched in the market. In the long run, you get to boost your online income by: letting you introduce an error-free, “sturdy” piece of software to the market; achieving fast overall performance and thus accelerating potential customers’ journey through the sales funnel; allowing for painless further scaling when the need comes;    helping to provide the best possible user experience as a whole; When is performance testing required? Performance testing is about going through all the major software functionality components for complete optimization. The sooner everything is checked, the earlier the problem will be discovered in the performance and the lower the cost of troubleshooting will be. The underlying procedures (mentioned above) usually start with predictions of potential traffic volume, conversion rates, and the number of possible product use cases. For this, it is important to evaluate the structure of the service, the planned number of registered users, simultaneous logins, and other indicators depending on the type of product. Next, you need to define performance goals and service load requirements. We usually compare performance testing to peeling an onion. You can achieve your goals by removing layer by layer of product performance vulnerabilities during multiple rounds of validation, troubleshooting, and retesting. It's important to test productivity in advance!   Multiple Test Cycles Types of Performance Testing Depending on the characteristics of the tested system, the UTOR team mainly distinguishes the following types of productivity testing: Volume (Flood) Testing, Load Testing, Stress Testing, Stability and Reliability Testing. Volume (Flood) Testing — this one indicates how many users or the amount of data an application can process with high productivity. Load Test — this is, basically, testing the response of the system to load spikes. This method simulates the number of virtual users who can simultaneously use an application or website. Stress Tests — this is when you evaluate the behavior of systems during peak activity. Such testing allows you to analyze the ability to regenerate, as well as to identify components that disappear at an extreme level. Stability and Reliability Testing – this one allows you to emulate the behavior of the system during atypical situations, shutdowns or restarts of various components of the product, or prolonged loads in the system. Types of performance testing classification may vary How Does Performance Testing Work? Despite the differences in the testing methodology, there is a general structure that guarantees stability under different circumstances, and also identifies weaknesses in the system. Before examining an application or a website, general requirements are identified, as well as criteria by which the productivity of the system is to be assessed. Examples of productivity criteria: the software handles over a thousand active users; the speed of generating a page with query results doesn’t exceed 3 seconds; when searching for profiles by specified parameters (with photos, for example), the server works properly with at least 150 simultaneous requests, etc. Testing cycle In a nutshell, the testing cycle looks as follows:  Step 1. Defining the testing environment Before getting to testing, it is important to determine the details of the hardware, software, and network configurations that will be used. Step 2. Defining productivity criteria The criteria for successful software testing are not the same for every other project. Before starting any work in this direction, we recommend that you analyze several solutions similar to yours. Step 3. Defining scenarios for planning & testing performance Describe how different types of users can use the solution you are developing. Key scenarios are essential for making tests as realistic as possible. After completing the test, analyze the data obtained and combine the results. Make changes to the system and repeat the tests again. Completing these steps will reduce the likelihood of application crashes. Performance testing metrics  It is important that performance tests generate large amounts of data. This will allow you to quickly and accurately analyze productivity issues, identify the causes of problems and quickly fix them. Test metrics should be clearly defined prior to testing. The following criteria are distinguished: bandwidth (the number of bits per second used by the network interface); the amount of virtual memory used; the speed of processing faulty pages by the processor; the average rate of hardware interrupts that the processor receives or processes per second; response time; speed of requests per second; maximum number of active users; the number of by-second requests to access the file on the web server; maximum waiting time for loading pages, etc. Performance testing tools and their features There are many tools for performance and GUI testing, functional testing, and others. When choosing a tool, focus on your product requirements and ease of use (or suitability for your skills). We recommend the following 3 prominent tools:   Apache JMeter, designed to test productivity and load, analyze and measure the performance of services. It is mainly used for web applications and web service applications. Locust, supports load tests and can be used to simulate a million or more active users. WebLOAD, includes a comprehensive development environment, load emulation console, and advanced analytics dashboard. It is a web and mobile tool for load testing and metrics analysis. Conclusion Performance testing is your solution for stable application operation, high income from its use or purchase, and a guarantee of a positive reputation on the web. Want to reduce the likelihood of application downtime and get a quality product to market faster? UTOR provides performance testing services to prevent any malfunctions of software products and ensure the stable operation of its components. Contact us to discuss your testing strategy.  

There is no doubt that quality assurance should be an integral part of software development process. However, which option is best: to hire an in-house QA specialist or outsource it to a team of experts?  In this post, we will discuss the pros and cons of each approach. Hopefully, it will help you to choose the best QA strategy for your particular case. Why you need QA outsourcing When you outsource the QA of your project, you hire a third-party company to discover the flaws of the product for you. The external team will conduct an audit of your systems, evaluate their state, and fix bugs if necessary. More and more software developers choose this approach today. Here is why.  The pros of outsourced software testing First of all, let us look at the undeniable advantages that QA outsourcing has to offer.  Cost and time-efficient Hiring an in-house team demands a lot of additional costs:  Your HR needs to find and compose the team. The time of the HR specialist also costs money; It will take the new employees some time to get familiar with the project and with each other. During this period, the efficiency of their work will be decreased; Depending on the type of QA, you have to invest in specialized software or testing facilities, for example, eye-movement tracking devices; The salaries of every employee should be planned in your budget. After the project is complete, it becomes not cost-effective to have your own QA team. As a result, you will have a team that exclusively works on your project and knows it well. But if you do not have much time that is needed to create a team or are not ready to purchase expensive equipment, working with a third-party company might be a better option. Outsourcing quality assurance to a specialized agency, you save money and time. An experienced team already has well-organized working processes and dispose of the necessary knowledge and equipment in order to deal with different QA-related tasks. If needed, they can augment the team attracting other staff members to the project and work faster without compromising on quality.  Unbiased look You can trust quality assurance to the same people who developed the product but it is not the best choice. The creators know how the product should work, and it can obscure their vision when testing it.  Also, adding new responsibilities to the to-do lists of developers, you distract them from their main job. Nobody can do everything at once remaining super productive. Unless you have a whole QA department in your company, it is better to outsource quality assurance. Attracting completely new people to the evaluation of the product, you can get an objective look at the code quality and functionality.    Experience  Professional intuition is very important in the work of a QA engineer. They have to investigate the flaws of the product, which truly reminds the work of a detective.  Even if you have seasoned software developers in your team, it does not mean they are good at discovering bugs or combining different QA methods to achieve the best results. They are more focused on writing good code in the first place.  Scalability If your project grows, and you need to attract more engineers to the project, it is easy to do when you have a partner. The QA company will take care of everything so that it would be easy for you to scale.  The cons of QA outsourcing When the team that works on your project is not in-house, it raises concerns. Let us look at the most common ones. Issues with security It seems risky to introduce third-party engineers to your project, especially if you are developing something highly innovative. However, if you choose a reliable partner, you can trust them with the details of your project without any doubt. You will work by a contract and sign an NDA, which eliminates any risks of the details of your project being disclosed.  Quality control Another thing that often stops project managers and business owners from QA outsourcing is that the work of third-party vendors is harder to monitor. The processes are not as transparent as in the case of in-house development. Here it is important to understand that different companies have various approaches to QA. You are entitled to choose the vendor that will keep you aware of the slightest details and will allow the in-house team to monitor their activities. Communication difficulties Working with an outsourcing team, especially, if this is a remote team might seem frightening. It is difficult to establish effective communication even inside the company, what to say about third-party engineers. However, professional QA firms have plenty of experience working with remote clients. They will offer you the tools for business communication and discuss the procedures that will make your cooperation as comfortable for you as possible.  What services do QA outsourcing teams provide? QA outsourcing is a flexible approach to software testing. You can work by different models: Hire a dedicated team Dedicated team model is the most common in QA outsourcing. Third-party engineers take ownership of the entire project from start to finish. They are fully involved in the testing product. The responsibility for the quality of the services is born by the third-party vendor. Therefore, the client can switch to high-level business tasks rather than monitor the entire process.  This model is preferable when you are looking for a long-term partnership and the technical requirements for your project are constantly changing. In the long run, it allows cutting the costs on testing. In this case, payment is made monthly. Invite an expert You need expertise in a certain type of testing such as automated testing, penetration testing, performance testing. A QA company can provide you with a subject matter expert for the task. You can outsource just part of the testing. Alternatively, you trust them to keep an eye on your project throughout the whole development process if it is small-scale.  This option suits you if there are no in-house testing specialists, or they cannot cope with the amount of work and need a hand. Contracting a member of an experienced team is less risky than working with a freelancer. How do you pay when you outsource QA? No matter if you work with a dedicated team or an individual contributor, you will probably choose one of the following pricing models. Fixed price. The particularity of this model is that the payment, the deadlines, and the expected results are fixed in the beginning and are not changeable on the go. It may seem attractive at first. If you have a small project and you are sure no modifications to the product will be needed, it is the right approach for you. However, if you are working with a larger project, which is constantly transforming, choosing Fixed Price means compromising on quality. Time and Materials. You are paying for the time your partner spends on QA. A contractor assesses how many man-hours it will take to complete the project and multiplies it by the rate of a specialist. The project is usually paid for in iterations when intermediate results are presented to the client. This model is very flexible and allows you to introduce changes as often as needed and be always aware of what is happening. A possible drawback can be that the estimation of how much the project will cost is only approximate at the beginning. Both of them have their pros and cons and should be chosen based on the size of the project, technical requirements, and the level of flexibility that you seek. QA outsourcing vs in-house QA Overall, quality assurance should be part of any development process. It allows you to check the functionality of the software systems, as well as make sure that the interface is accessible and intuitive to the users. In-house quality assurance demands a lot of organization work. Finding the right people for the QA and setting up QA processes are money and time-consuming. Buying the necessary equipment can be not cost-effective if you are only going to use it once. Moreover, in-house software engineers can fail to provide an unbiased look at the work of their colleagues or their own. This may affect the final result very negatively. On the other hand, outsourcing QA to experienced professionals provides you with much more freedom. They have the resources to conduct testing of any volume and complexity. In case something goes wrong, you will have a reliable partner to cover your back: be it the necessity to meet strict deadlines. A QA agency has already worked on a similar project to yours and can apply the best strategies to secure the quality of your product. In the long run, the costs of working with a third-party vendor are going to pay off.  Tips for successful QA outsourcing Every project is unique and there is no universal testing approach that fits all. However, if you decided to outsource testing, these 5 simple tips might help you to get the most out of your experience. 1. Choose the right partner There is a huge variety of testing companies out there. You can ask your business partners to recommend someone or check out independent aggregators. Some examples are Clutch and GoodFirms. They collect information about the best IT companies and independently assess the participants. On the website, you can also find the approximate rates and client feedback. You can find a good company regardless of your geographical location and ability to travel because many countries work remotely. You can save money working with software engineers from countries with low labor cost such as India, China, Poland, Ukraine, and the Philippines. 2. Work by a contract No matter if you work with a freelancer or a company, sign a contract that clearly states your obligations and rights. You provide access to the internal systems of your product and have to be protected in case something goes wrong. NDA can help you to feel more secure.  3. Stabilize communication Clearly stating when and how you are going to monitor the work of your third-party vendors will save both sides a lot of nerves. Transparent communication is possible even when you outsource quality assurance but it requires a bit more effort, especially in the beginning. A professional QA company will provide a manager from their side who will keep you updated about the progress on your project. 4. Control the quality Trust is important in the relationship with your partner. However, if you do not want to be disappointed, be involved in the project. This way you will be aware of what is going on and will be able to express your opinion and participate in the decision-making process more effectively. In order to be aware visit Agile meetings or request weekly reports about what is being done on your project. 5. Choose the right model As you know now, there are different possible models of cooperation with professional testers. Get in touch with a couple of firms, discuss how they work. Search for a flexible experienced team that is quality-oriented. Your goal is to find a partner who you will feel at ease with.  Conclusion Quality assurance needs to be done at every stage of the project if you want to meet the requirements of a highly competitive market. Unless you have a dedicated department already, in-house quality assurance is a messy and costly endeavor. That is why so many software development teams and businesses prefer to work with remote third-party teams. They will be in charge of all the processes so that you could enjoy the results.  

Many teams believe that the software should only be tested after it has been completed. This opinion comes from the Waterfall software testing methodology – an outdated process that brings about more problems than it solves.  In most cases, where the distinctive levels of software testing are overlooked, the software ends up having more bugs which are costlier to fix than if they were found earlier.  This means more work for developers and more costs for the company, as it has to cover extra hours spent on repairs. In the US alone, the cost of software bugs is projected to be about $2.8 trillion per year. But this isn't all of the problem. Imagine what might happen, in the event of software failures that affect not only the finances of consumers but also their lives. Here's one case example. Approximately 300,000 patients with heart disease received the wrong medication in 2016. The hospital's clinical computer had a SystemOne error that had been left unexposed for a long time. This led to a prolonged miss-assessment of the risk profiles of these patients. Undermanned patients developed stroke and other health complications while overtreated patients experienced serious side effects due to unnecessary dosing.  Extensive testing can potentially prevent such critical errors and reduce the cost of debugging. The cost of fixing bugs after the software launch could go up to $16,000, but it costs only $25 if found during the development stage. However, the big question is still, how can software testing be carried out correctly? Ideally, different levels of software testing should be performed during and after the development process. At UTOR, we make use of Agile testing. In fact, according to BetaNews, 97% of companies, including IBM, Microsoft, Cisco, AT&T, are using Agile technology for both development and testing. Contrary to Waterfall, the use of Agile ensures continuous testing during the building of the software. This is a software testing process that allows seamless interaction between the customer, developers, and the testers from when the first code of the software is written. That is, while there’s value in the items on the right, we value the items on the left more. AGILE WATERFALL Teamwork and Progressive Agreement  Over Protracted Documentation and Planning  User-focused Software  Over Process-focused Software  Customer Satisfaction and Strong Sense of Ownership  Over Little or no Customer Interaction Adjustments and Changes  Over Rigidity  Essentially, each stage of the software development life cycle undergoes a corresponding testing process.  If you want your software to operate without any hindrance, you must conduct tests at each of these stages.  There are 4 main levels of software testing that must be carried out before the software is launched. These tests can be carried out in-house by developers in an organization or outsourced to expert software testers like us. They include:  Unit or first level test Integration or second level test System or third level test Acceptance or fourth level test   The illustration shows the 4 levels of software testing, from testing individual units to testing integrated units to testing the entire system, and to the final level, where you check the acceptability of the entire system. Unit Testing  This is ideally our first level of testing software. How does it work? Here, specific lines of code, distinct functionalities, and desired procedures are isolated and tested. These lines of code, functionalities, and procedures are termed software units because they are combined to make up the software. They can also be referred to as components of the software.  For example, a line of code in a "calculator software" can have the function of implementing "addition," while another line of code implements "multiplication," etc. These addition and multiplication functions and other functions of the calculator software must be individually tested to ensure that the calculator operates flawlessly. You do not want to start a calculator that implements a division when users click on the add sign. Unit testing is a process that mostly involves testing the internal workings of the software. It's easy and quick to do because it deals with the software unit by unit, not as a whole. Who Performs a Unit Test? Unit testing should be done by the software tester(s). Integration Testing This testing level involves combining all the components that make up the software and testing everything as a whole instead of individually as done during unit testing. Also, from this level, tests can be split into functional and non-functional types. Why is integration testing needed?  First, the integrated codes could have been written by different developers, and need to be tested to ensure their correctness. By testing, we can identify and correct inter-working defects, simultaneous operation defects, parallel operation defects, etc.  Simply put, it helps us ascertain how well the units work together, and the condition of the interfaces between each of them. For example, we could verify if the following conditions from the checklist were met: If communication between the systems is carried out correctly If linked documents can run smoothly on all platforms Whether security specifications work during communication between systems Whether the software can withstand network breakdowns between the web servers and app servers The condition of an interface may be poor, such that it might take a long time for the software to switch between functions. A good interface switches between functions immediately the user initiates it.  Sometimes, a line of code may work properly when tested alone. However, when combined with other lines of code to achieve the desired function, an error can occur, indicating that the integration of one or more lines of code into the rest of the code was unsuccessful. Unit testing can prove that everything works as expected, while integration testing can prove otherwise. This is why we ensure that all recommended levels of software tests are carried out rigorously, to avoid potential pitfalls.      There are 4 main approaches we use to carry out integration testing for our clients. They include:  Top-Down Integration testing: The rule of thumb when applying the top-down approach is to test higher units before checking the lower levels. In other words, the performer has to test from top to bottom. Bottom-Up Integration testing: On the other hand, in bottom-up integration, the performer tests in the opposite direction ― from bottom to top. You ultimately try lower units and step up to the higher components.  Big-Bang Integration testing: Big bang integration testing refers to when all elements are tested concurrently in a single phase. The big-bang model is also applicable when validating small applications, where the top-down and bottom-up models can be bypassed. The modules are all programmed, combined, and tested together. Mixed (sandwiched) Integration testing: Available modules are being tested, whether they belong to high or low code lines. This means that no priority is given to either component by the performer. Unresponsive modules are simulated to determine how well the available code will work when the coding is completed and combined.  Who Performs an Integration Test? Integration testing should be handled by the tester(s). System Testing System testing has to do with verifying the required operations of the software and its compatibility with operating systems. In other words, we test both the technicalities and the business logic of the software; we run functional tests to check what the various functions of a system do, and non-functional tests to check how those functions work.  For instance, in functional testing, we check if a login feature responds when the user enters a password. But in non-functional testing, we check how long it takes the user to log in after password entry. Read more about functional and non-functional tests. To be done effectively, it has to be tested by external software testers that have absolutely nothing to do with the development of the software. System testing helps to determine how compatible the software is with the operating systems for which it has been designed.  Thus, all the software's completed versions are tried out on all the possible operating systems that users may have. There are cases where the Android version of an application may be functioning correctly, but the iOS version may have some issues.  System testing exposes such errors, and developers can detect and correct them before they're launched. System testing, in addition, ascertains how good the design of the software is. The internal code can work adequately, but the outward design might have some operational issues. This is why system testing is essential. Hence, all design abnormalities can be detected and corrected.  The general behavior of the software is also another aspect that system testing helps to authenticate. The software may be appropriately developed and yet malfunction, due to various system specifications.  System testing helps identify any abnormal behavior and helps organizations to outline the software's best system specifications. Failure to gather these requirements can lead to scope creep. This also means that the project can accrue more costs, longer duration, and a need for extra resources. The system too, will not live up to its promise of delivering business value for your organization The identified system specifications are then relayed to the end-users as requirements that their devices must meet or exceed. Today, every authentic software you buy comes with operating system minimum requirements.  Who Performs a System Test? System testing should be carried out by a QA team of software testers. Acceptance Testing  This level of software testing is similar to the system testing, but here, the test is carried out by some selected end-users. This is the only software testing stage that is carried out by users. This stage determines if the software is finally ready to be launched to the general public.  All the selected users give their various opinions about the operation of the software; they let the organization know whether the software meets their diverse requirements, and recommend areas that may need to be improved upon. Acceptance testing can also be referred to as User Acceptance Testing.  Though each organization may conduct acceptance testing using its own specifics — who and where to test — it is vital that we select users with different demographics, operating systems, and unique past experiences of using similar software. There are two types of Acceptance Testing:  Alpha Testing  Beta Testing  Alpha Testing:  Alpha testing is carried out by selected users but together with an internal team of developers who control the testing environment and try to mimic realistic conditions during testing. Here, the selected users are given passwords or access keys that will allow them to log in and out of the developer's platform without any hindrance to use and test the developed software/ application. During alpha testing, minor errors can still be found and fixed. Beta Testing:  Beta testing is carried out by the selected users on their own devices and operating systems. Here, the finished software is sent to the selected users to use it and test it over a while, before giving their various feedback. At the beta level, the software is expected to be void of defects, run perfectly, and meet the users’ needs. Who Performs an Acceptance Test? The end-user(s). How to Perform the Different Levels of Software Testing in the SDLC In conclusion, all the levels of software testing are essential and have to be completed before the application is launched. Meanwhile, in order to see the bigger picture, you may want to check out the types of software testing.  At UTOR, we ensure that the most experienced developers test your software. During acceptance testing, we carefully select users who will give genuine opinions about the design and operation of the software.  It is crucial to start testing your software as soon as the first code is written to eliminate any error or abnormality that may arise. Also, be aware that testing continues even after the software has been launched to implement future updates.  If you’re building software for your business and need to execute proper testing or you simply have some questions about software testing, contact us. We’ll walk you through the entire process, and provide detailed insights that you’ll need.            

    We’ve already told a bit about the software testing hierarchy. There are four levels of software testing: unit, integration, system, and acceptance. Each of these levels examines software at a certain stage – when separate components are coded, when they are assembled into features, and when features get united into a system, etc.  System testing comes after a QA team has checked the smallest functional blocks and their interactions. This time, we decided to focus on this type of testing with characteristics and particularities that make it essential.  What Is System Testing? System testing is the inspection of a fully integrated system. The purpose of this checkup is to verify the system's conformance with both functional and non-functional requirements.  In the course of system testing, a QA team can notice the following defects:  Misuse of system resources Unexpected combinations of user-level data Incompatibility with the environment Unexpected use cases Missing/incorrect functionality The inconvenience of use, etc.  System testing uses the black-box method. It means that a QA team doesn’t know how the backend works. They aren’t aware of the internal structure of software and check the features using frontend – the ‘facade’ of the program. This allows modeling user behavior closely since users interact with software in the same way.  Objectives of System Testing The goal of system testing is to minimize the risks associated with the behavior of the system in a particular environment. For this, testers use the environment as close as possible to the one on where a product will be installed after the release. The objectives are some smaller steps that allow achieving the goal. In system testing, there are several milestones that make the release of a flawlessly functioning system (read: software without critical bugs on production) possible. So the primary objectives are:  Reducing risks, for bug-free components don’t always perform well as a system.  Preventing as many defects and critical bugs as possible by careful examination.  Verifying the conformance of design, features, and performance with the specifications stated in the product requirements. Validating the confidence in the system as a whole before moving to the final stage – acceptance testing that takes place right before users get access to a product. The Differences Between System and Integration Testing System testing follows integration testing. The latter checks how several units, the smallest functioning parts of code, work together. So, integration testing helps the team to prepare for system testing, since it inspects blocks of units that are later united into a finalized build.  System testing goes further compared to integration testing. It inspects how all the units act after they become a wholesome system. As usual, the fact that units work as supposed after integration doesn’t mean they will function the same way in a system.  The Basis for System Testing A test basis is the source of information that contains all the requirements for the software product under development. In other words, it is the documentation a QA team uses to draw conclusions on whether the product turned out to be the way it should be.  The basis for system testing should clearly define the criteria for features and performance. It should be well-structured so that finding relevant data for test case creation doesn’t cause any difficulties.  There are several documents that can be used as a testing basis, including:  Functional and non-functional software requirements specifications (SRS) that provide complete guidelines on how the system should work. Risk analysis reports that outline the areas of risk so the team can prioritize the tasks and understand the outcomes of these risks. Use cases that describe user flow.  Models of system behavior that verbally describe the processes and activities all the components are involved in.  User manuals that explain how to use the software. System Testing Approaches There are two approaches to the system testing – one based on the requirements and the other one based on use cases. The former type uses project documentation as a set of main criteria for the inspection, while the latter aims to closely mimic user behavior.  Depending on the side that runs testing, there are alpha and beta testing. Both aim to find the bugs so that a dev team can fix them before the release.  Alpha testing is the checkup by a QA team in the testing environment and can require long execution cycles.  Beta testing is a task for other departments or people outside the company. It is focused on the user perspective and uses the majority of insights for improvements in the future.  Types of System Testing When it comes to the types of system testing, you may need more effort to remember all. Actually, there are more than fifty types of system tests that cover both functional and non-functional aspects.    You can find detailed information on some of them in our other blog posts. For now, here’s a short list with a short explanation to get familiar with the basics.  Functional testing focuses on software features and checks all the functionalities to verify they match the requirements.  Performance testing examines software behavior in different environments and under different conditions. It features load, stress, spike, recovery, and other types of tests that estimate system response, speed, and stability.  Graphic user interface (GUI) testing checks the interface – all the visual elements customers use to interact with the system. At this stage, the team checks buttons, input fields, text rows, etc.  Usability testing estimates the overall convenience of using the program, taking into account both practical and emotional factors. Compatibility testing is used to verify that a program, app, or other types of software we are dealing with is supported across different devices and browsers. It is extremely important for mobile devices that are available in such an abundance.  Localization testing is the adaptation of software for different markets. It involves more than translation, covering also metric system, date formats, text input and methods, etc.  Compliance testing is the checkup meant to verify that the software meets all the legal requirements. This type of testing is extremely important for healthcare and banking apps since they deal with sensitive data and imply strict governmental regulation.  Security testing is the way to make sure the app is safe to use and well-protected from hackers and scammers. It is essential for every application that requests personal data or offers verification via Google, Apple, Facebook, or other social accounts.  Regression testing checks an untouched part of the functionality after a dev team makes some code changes since they can be affected by accident.  These are only several basic types of system testing required by different kinds of software products and provided by most QA companies. The final list for your product will depend on its specifications and your expectations.  How to Run System Testing?  If explained in detail, step by step system testing process requires one more post. Here’s a quick outline – system testing in a nutshell – to help you get the algorithm of this process.  First, you need to create a test plan. After that, write the test and use cases. The next step is to create test data used for system testing. If you apply automation on the project, start with it. Then, run manual tests that cover the rest of the functionality.  The testing results are used to generate a report that goes to the development team. They are the ones who are going to fix bugs. Testers only run the inspection and suggest improvements. After the errors are fixed, run regression tests to make sure the untouched functionality hasn’t been affected. This last step can repeat several times until no critical bugs are left.  To Sum Up System testing is the third step in the hierarchy of software testing levels. It comes after integration testing and reviews a system as a whole.  During system testing, we work with a software build that is potentially ready for release. As a rule, there are still critical bugs that deteriorate user experience. The main task of a QA team at this stage is to find as many errors as possible so that the team can move on to acceptance testing – the next and final level in this hierarchy.