Hiring new team members is an uneasy process. You want to be sure that a person can handle all the assigned tasks and challenges. We’ve come up with a list of questions that might be helpful to interview new members of a QA team. And if you are a QA engineer preparing for a job interview, you may find some useful information as well.  We’ve divided the questions into several categories for convenience. These categories cover general terminology, particularities of different testing types, and soft skills.  General Testing Questions Why is software testing essential?  There are several reasons that make software testing a must for every project. Here are some of the key points: Software testing evaluates product quality and compliance with business requirements. This compliance is crucial for product success. Testing is a way to assure that software works as expected and users aren’t going to encounter any serious defects. Testing implies the estimation of a product from the point of view of an end customer, so it helps deliver a positive user experience.  Testing allows finding bugs early to reduce the cost of a possible mistake in development.  It is also the way to ensure that the software meets international standards set for a certain niche.  What is the test data?  It is data used in the software testing process – from requirements and inputs based on them, as well as expected outputs and actual results.  What is the difference between quality assurance, quality control, and software testing?  The purpose of quality assurance is to ensure that a company provides the best product/service to customers. This process is based on multiple reviewing activities.  Quality control is about examining a product in development to assure it meets original specifications. It is a broad term that includes software testing as one of the ways to validate conformance with business requirements.  How to achieve 100% coverage of testing? 100% test coverage is impossible. You can come close to absolute coverage if you create at least one test case for every feature. Some user scenarios, however, are unpredictable. Testers can miss them, and that’s okay – they will make conclusions and use this experience in the future.  How to ensure 100% bug-free software?  100% bug-free software is a myth. The goal of a QA team is to make sure software is free of critical bugs. What is risk analysis?  It is a technique that identifies what can go wrong during software development. During risk analysis, a QA team identifies hidden issues and prioritizes the sequence of resolving those issues. There are issues of high, medium, and low importance. What are the best practices of quality assurance?  The best practices are different for every project. They greatly depend on the company’s business processes. Some of the universal positive practices are: continuous improvement; a smart combination of manual and automated testing on large projects; clear metrics and documentation; test-driven development; responsibility of team members. How testing techniques are different from testing tools?  A testing technique is a process we use to ensure that certain parts of a system function properly. Testing tools are software solutions we apply for testing when necessary. There are only several techniques and a vast variety of tools.  How to make sure you’ve created enough test cases to cover the product? There should be at least one test case for every requirement. Otherwise, test coverage is insufficient.  What is test coverage?  Test coverage estimates an amount of testing on a project. To measure it, we can list the features and see whether each has at least one test to run. Testing Types, Methods, and Techniques What is the difference between white-box and black-box testing?  White-box testing is the technique that counts on a QA team knowing how the back-end works. QA engineers design test cases to exploit the code for finding bugs. During black-box testing, the team estimates functionality from the business requirements perspective. Testers check the software like a user would experience it. What is static and dynamic testing?  For static testing, a QA engineer prepares a checklist to find errors. It is cost-efficient, covers more areas in a shorter time, and can start before a program is finalized.  Dynamic testing starts after code deployment. There is an actual application to check valid inputs and expected outputs.  What are the levels of testing?  We can group tests by the part of functionality they check or the stage of the software development process they are applied to.  The latter division defines four levels of testing:  Unit testing checks separate code components. It is conducted at the earliest stages, usually by developers.  Integration testing evaluates how different units are functioning together.  System testing examines the entire software system against the requirements.  Acceptance testing evaluates compliance with end-user requirements. What is functional testing?  Functional testing is meant to verify that actual product features match those described in the documentation. A testing team checks the behavior of software against the expected specifications and requirements. Functional tests cover all features and take into account the most probable types of mistakes. How is non-functional testing different from functional testing?  As you can guess from the name, functional testing covers the functionality – in other words, software features. Non-functional testing is about performance, usability, reliability, and other system parameters that indicate whether it is optimized for usage.  What types of performance testing do you know?  Performance testing focuses on software speed, stability, and scalability. Some of the performance testing types are:  Load testing – checks how the system behaves under an expected load.  Stress testing – inspects how the software handles the increase in traffic.  Spike testing – models sudden traffic rises and examines how software reacts to them. Volume testing – reviews system behavior when the volume of in-app data stored increases.  Scalability testing – estimates the ability of the software to scale up and down.  What is exploratory testing?  Exploratory testing is the simultaneous development and execution of test cases. During exploratory testing, a QA team doesn’t follow a plan and doesn’t have any predetermined testing procedures. It is effective when it is not evident what test should come next. Is regression testing and retesting the same thing?  Regression testing verifies that the latest code changes haven’t affected the existing functionality. Retesting verifies that bugs detected earlier have been fixed.  What test coverage techniques do you know?  There are statement coverage, decision coverage, and path coverage.  Statement coverage verifies that each end every line of source code has been tested.  Decision coverage verifies that every decision in source code has been tested.  Path coverage verifies that every possible route has been tested.  What is risk-based testing? Risk-based testing is an approach based on prioritizing test cases and software features taking into account their importance, the probability of breaking, and how critical probable mistakes are. Bugs, Defects, Errors Why does software have bugs?  To start with, there is always a human factor. It is not possible to write a perfect flawless code because units can work unpredictably together. Other frequent reasons for bugs are programming errors, miscommunication, changing requirements, tight deadlines, and software complexity.  Are testing and debugging the same thing?  Testing is a process of detecting bugs. Debugging implies looking into code and fixing what causes bugs. QA team is responsible for testing. Debugging is the task for developers.  What are the steps to follow after you find a defect? QA engineers only find bugs. Fixing bugs is a task for developers. So, after finding a software defect a tester reports it to a developer. It is important to document the bug well, mentioning the conditions, the number of times it occurs, and expected results. A complete bug report makes it easier to fix the issue. What is the difference between a defect and a failure?  A defect is a software issue you find and resolve before the software gets to production. A failure is a bug that reaches an end customer.  What is a latent and what is a masked defect?  A latent defect does not cause failure, because it is difficult or impossible to recreate the set of conditions that causes it. A masked defect does not cause a failure either, but only because there is another defect that prevents the execution of this part of the code. What are the main components of a defect report?  The essential components of a defect report are:  Project name  Date of detection  A tester who detected it ID and number of the defect Severity status and priority Snapshot Date when it was resolved A developer who resolved it  Is verification different from validation? Verification implies evaluating software to make sure it complies with the general or niche standards. Validation estimates software from the perspective of business requirements. Is there a difference between a bug and an error?  A bug is an issue in software detected during testing. This issue makes software function in a manner you don’t anticipate. An error is an issue that arises during testing because of a missing scenario in the requirements, mistakes in design or implementation. What states of a bug do you know?  There are several phases a bug goes through during software development. A bug can be:  new, when it’s just been detected; assigned, when it becomes a responsibility of a particular development;  open, when the work on fixing is in progress;  rejected, if an issue is marked as a bug by mistake;  deferred, if it is not critical and there are urgent issues to resolve; fixed, when a bug is finally resolved;  reopened, in case a tester is not satisfied with the solution;  closed, after the bug is finally verified and everything works well.  What is a blocker? A blocker is a bug of the highest severity and thus of the highest priority. It interferes with testing by blocking a large share of software features. It is different from a critical bug since the latter deteriorates user experience but doesn’t block testing. SDLC & STLC What is the software testing life cycle?  Similarly to SDLC that is a collective term for all activities that occur throughout the development process, STLC refers to all the activities performed during testing.  What is the place of testing in SDLC?  It depends on the type of SDLC the company sticks to. If we are talking about a traditional Waterfall approach, testing follows requirement collecting and analysis, designing, and coding stages. It comes right before the deployment.  In Agile methods, however, the place of testing isn’t that strictly defined by the order. It can start, for example, at the requirements stage. Documentation testing will help avoid gaps in product logic and contribute to project feasibility.  Testing can end after the product deployment if no further changes are planned or last through the product life cycle, which is a more frequent case.  When software testing should start and when should it stop? Testing can start at different stages:  after the code is ready; when code is still in development, but there is functionality that testers can work with; at the beginning of the project when documentation is created.  It depends on the project complexity, a company’s business processes, and approach to testing. The time to stop testing also varies. There are some milestones a QA team keeps in mind to decide upon this. So, it can happen: after test case execution; after all the high-priority issues have been fixed; if the interval between inherent failures is large;  when a large functionality is covered by automated tests and no further critical bugs appear.  Some decide to stop testing when deadlines are tight and there is no other option but to release software. This scenario is undesirable.  What is a test strategy?  A test strategy describes the approach to STLC. The goal of a test strategy is to provide rational planning for the entire testing process – from management goals to real test cases. Altogether, it focuses on building an efficient quality assurance process.  Test strategy considers and explains team roles and responsibilities, test scope, environment, tools, schedule, and associated risks.  What is a test plan and what does it cover?  A test plan is a document that describes testing goals, resources, and processes for a particular project in detail. It provides a comprehensive overview of the working process. Creating a test plan is a task for a Test Lead or a Test Manager. The document features test scope, objectives, environment, entrance and exit criteria, risk factors, and deliverables.  What is the difference between a test case and a test scenario?  A test case is a sequence of actions used to check a certain functionality. It helps detect and identify problems. Test cases verify specific flows with predetermined inputs, preconditions, and outputs.  A test scenario defines what exactly to test. You may need multiple tests to cover a single scenario.  Why is it important to split testing into stages? Splitting testing into stages makes it easier to manage tests and the process in general. Every test stage has a different purpose. Sometimes there is a need to run tests in different environments, and that is another reason to distinguish between several different stages.  Throughout different stages, testers focus on different aspects of functionality and performance. It positively affects overall product quality. What are the phases of STLC?  The division may depend on the business processes inside the company. Usually, it consists of the following phases:  requirement analysis and validation; test planning;  test design;  environment setup; test execution; test closure & reporting. What types of documents do you use across SDLC?  Documentation testing is useful but companies often skip it. Project documentation varies across the SDLC.  There is a project test plan that outlines the complete strategy. A team uses it from the beginning to the end.  An acceptance test plan comes in during the requirements phase. A team gradually completes it during the entire process, too.  A system test plan starts at the designing stage.  An integration test plan starts at the execution phase.  What is Agile testing?  Unlike testing in traditional SDLC models, it goes alongside with coding. A QA team can find bugs earlier and reduce the cost of mistakes. Agile also implies continuous communication between teams. It allows testers better understand the customer-ready product and test it efficiently from a user’s perspective.  Role Specific Questions Should a QA lead write test cases? Some are certain that a QA Lead should only review test cases, but that’s not true. The QA Lead should contribute to the test creation just like any other member of the team. When would you recommend automation and when not?  Test automation is meant to facilitate the work of a QA engineer. Thus, it is relevant to automate tests for large systems, but only in cases when they are stable.  Automation is useful if you deal with complex calculations, regression testing, and non-functional testing. It may be reasonable to automate smoke & sanity testing, GUI & API testing.  You shouldn’t automate tests that may lead to unpredictable results. You cannot cover unstable parts of software with automation. You shouldn’t use automation when its benefits don’t apply – for example, when scripts writing takes longer than a manual check. Some types of testing, like usability and exploratory, cannot be automated per se.  What is Scrum?  Scrum is a popular way to adopt an agile methodology. SDLC is based on short iterations (one-, two-, or three-week sprints) that end with a release of an updated version of a product.  All members of the agile team have daily meetings to discuss the process, share updates and insights. Scrum Master is usually a person who holds the meeting, but every team member reports on the work done, tasks planned, and results. The continuous communication makes development more efficient and helps deliver a product to end-users faster.  What were your role and responsibilities at the previous job?  The roles and responsibilities are quite flexible nowadays, just like job titles. A person should mention their key tasks and duties, both hard and soft skills. For more on this, check out one of our previous posts on QA team roles and responsibilities.  What testing tools do you use and how do you select them?  A person should list both testing and management tools, including bug-tracking systems, messengers, management apps, and other software resources a team applies. The list may be not default but vary depending on a client. So the criteria for selection of management tools may depend on a client’s preferences. When it comes to tech tools, it is important to: identify the features required in an automation tool for a project; evaluate open-source and commercial options; estimate cost (both license & training) and benefits;  consult with the team before making a final decision.  How to provide effective feedback for the team? It is essential to find a balance between praises and criticism. It may sound a bit too vague, but that’s how you provide an effective feedback.  If a person is performing well, make sure they understand their achievements and encourage them to continue the same way.  If a person doesn’t live up to your expectations, don’t be too harsh. Start with things they do well, admit the contribution they make, and then explain what things need to change. Set deadlines for improvement. Provide assistance if needed and try to encourage the team member so they don’t give up but become a better specialist.  How can you contribute to the project?  An interviewee should tell about challenging tasks they’ve managed to handle and share their ideas on improvements. A person can also mention their skills that would be useful for the entire team. An interview isn’t the best time to be humble. Good candidates are honest but not arrogant.  This question encourages to talk about different practices a person would like to implement – from specific documentation and metrics to teamwork, educational initiatives, and bonding. A good candidate will demonstrate the ability to take the initiative and inspire positive change.  What skills should a QA Lead have?  Technical and soft skills matter equally much. Even in case a candidate has to deal mainly with management tasks, it is essential to know that they understand the QA process.  Here’s a piece of advice for recruiters:  Make sure a person has passed all the previous career stages before becoming a QA Lead.  Expect to hear about the hard skills an interviewee obtained through the years.  Ask about the tools they can work with.  The efficiency of clear communication, emotional intelligence, and conflict resolution should be mentioned among the most important things, too.  How do you set objectives for team members?  It always depends on a specific project, as well as the competencies and experience of every team member. The more experience a person has, the higher are the manager’s expectations.  The best decision is to set objectives that are a bit challenging but realistic. You don’t want a person to become exhausted and unmotivated by requirements that are primarily impossible to meet.  Encourage healthy competition within the team so everyone can do their best and learn new things meanwhile.  How do you manage team conflicts?  Conflicts are inevitable because of diverse backgrounds, tempers, and experiences of every team member. Thus, you should prepare for conflicts beforehand. They shouldn’t catch you flatfooted.  Conflict resolution always starts with a conversation. You need to hear all parties and then bring them together for discussion. By that time, you should decide on some compromises to offer.  Ask team members to co-operate based on a shared goal. Explain that you all have to focus on what’s best for the project and advance arguments for what is the best decision.  To Sum Things Up The list of questions is far from complete yet. These are the basics a software testing professional should know. We are going to update this list from time to time to cover more aspects of testing and keep up with trends. Bookmark this page to make sure you don’t lose and come back for more questions. 

In a modern digitalized world, information is one of the most precious commodities. Most of the data today is stored on private servers and in public cloud facilities. If a corporate network is connected to internet, which is ubiquitous in modern business, sooner or later someone would try to breach it - out of sheer interest or looking for valuable data. Despite sizable efforts many companies and cloud vendors put into ensuring the safety of the data they store and operate upon, what one human built, another can break. Top it with cases where sensitive information is revealed thanks to human factor, like one of the biggest recent instances - First American Corporation breach - which left more than 850 million customer records exposed and you can get a picture. Any lucrative activity attracts the attention of people seeking to make easy money. Black hat hackers steal commercial and personal information to demand ransom, blackmail or sell to highest bidder. So, we all heard stories about successful data breaches and gruesome reputational and financial damages these bring about. Such accidents happen quite often lately, wreaking havoc even on blue-chip enterprises to say nothing of smaller fry The financial losses caused by data security breaches amount to six-digit numbers annually displaying a steady growth pattern despite the security measures taken by companies. The exposure of customers’ data cost Desjardins Group $53 million, Norsk Hydro spent up to $75 million to cover for the losses inflicted by a cyberattack, British Airways footed the total bill of $100 million that resulted from GDRP violation. Naturally, these are the top cases, however, according to the 2019 IBM report, the average sum that companies have to fork out to redress data breach ravages has risen to almost $4 million - an astounding 12% growth within just five years! Given that the scope of the calamity becomes ever more appalling, businesses try to come up with effective data-protection strategies. As an established data protection experts, we consider penetration testing a cornerstone of any successful protection strategy. Penetration Testing: What Is It? Known under a shorter name of pen testing and a euphemism 'ethical hacking', this automated or manual practice consists of tampering with a computer, a network, or an app. Why would anyone agree to have their hardware and software tapped (even if it is just make-believe)? The answer is simple: reliable and official white hat hackers who exercise the penetration can identify system vulnerabilities that real hackers may exploit. Alongside this master goal, penetration testing may have some collateral objectives including assessment of the corporate security policy as a whole, monitoring an organization’s compliance with legislation requirements, gauging the personnel awareness, and sounding out a company’s ability to detect and react to security issues. The diversity of penetration testing goals can be explained by the wide variety of systems it is applied to.   The results of a test are reflected in a report that serves as a guideline for the organization to modify its security investment schemes to forestall possible hacking attempts. If probing penetration was against an app, its security vulnerabilities are revealed to developers allowing them to introduce necessary corrections. The Benefits of Penetration Testing Penetration testing is beneficial in many respects: It allows revealing the security weaknesses of systems or software. What is important, not only machines and their functioning can become the object of scrutiny. Testing agents examine the actions and working habits of your employees that may pose the security risk. It imitates real-life cyberattacks. Experts in the field know what hackers aim at when tampering with your software and networks. Thus, not only can they point to the areas of concern but also dispel your apprehensions as to other elements of the system that you might have considered unsafe. It exposes your ability to react to the challenges. Any security issue must be eliminated promptly and properly. Ideally, the organization should have a detailed plan with systematic procedures of reacting to breach threats. Testing will let you see whether such plans work well or underscore a necessity to have one. It ensures smooth business proceedings. Ultimately, penetration testing is called to provide uninterrupted functioning of an organization. Any security breaches result in absence of network or software availability, which translates into unintended downtime that impacts business adversely. Therefore, penetration testing serves as a kind of business perpetuity audit. It presents a third-party opinion. As the old saying has it, no man is a prophet in his land. So if anyone within an organization detects a problem, the odds are that the management may not treat the warning seriously. If the same is done by outsiders (especially ones with adequate qualifications), the likelihood that executives will heed their recommendations increases exponentially. It lets you keep abreast of the legal norms. For instance, in PCI regulations and the ISO 27001 standard, systematic penetration tests and subsequent security reviews are mandatory for modern public businesses that seek to operate internationally. It fosters customer trust. The company reputed for its strict and consistent security policy instills trust, which is a solid foundation for long-standing loyalty. Penetration Testing: FAQ Here are brief answers to questions most often asked about penetration testing: How often should the testing be conducted? Routine testing is to be performed at least annually. However, additional tests may be recommended in case of major shifts within the organization, like adding new infrastructure or significant revamping of the existing one, opening new offices, introducing new security software, revising end-user policies, etc. Who should perform the testing? Conditions for selecting ethical hackers are simple: they should not be aware of the current security measures of the revised company and be trustworthy. The first condition makes it clear that outside agents should be involved in the testing so that it will be as similar to real attack as possible. The second condition urges you to approach the choice of pen testers with caution. You should hire experienced specialists with impeccable background and excellent reviews. In this respect, UTOR ticks all the boxes and can handle any security-related tasks to improve your cyber protection. How is the testing conducted? Typically, testing proceeds along the five following stages.   1. Preliminary Arrangements At this stage, the scope and the aim of testing are determined as well as the systems to test and methods to employ. When done, the testing team collects intelligence about the network, domain names, mail servers, and other elements of the tested infrastructure up to the smallest ones. 2. Static and Dynamic Scanning By scanning the tested infrastructure's objects (both in an isolated state and as a whole), the team can realize how the system reacts to intrusion attempts. 3. Breach This is the core stage of penetration testing. Using SQL injection, cross-site scripting, backdoors, and other hacking methods, the testing team seeks to reveal vulnerabilities. Discovered tears are leveraged to escalate privileges, intercept traffic, steal data, and do other deleterious things to fathom the possible damage. 4. Maintenance Persistent threats are bugs and breaches that endure within the system for months enabling repeated thefts of data. The testing team tries to see if any of the detected vulnerabilities are of such nature. 5. Report Compilation The final report is drawn upon the completion of testing. It includes the details about discovered vulnerabilities, sensitive data that was or could have been tapped into, terms of breaches, and time during which testers managed to stay undetected within the system. What kinds of pen testing are there?   There are several directions of pen testing: 1. External testing During such testing, websites, DNS (domain name server), email, and other externally accessible resources are targeted. In white-box testing, which is a subset of external testing, the would-be intruders are briefed on the security measures practiced by the company beforehand. 2. Internal Testing It simulates an attack by a wrongdoer from within the organization. 3. Blind Testing Also known as a black box testing, it is performed by security experts who only know the name of the tested company. It also serves as a real-life drill for security staff. 4. Double-Blind Testing This method is similar to the previous one but the company personnel is not warned of the attempt beforehand. This allows evaluating the preparedness of the internal security department and its ability to detect unexpected attacks and mitigate their results. 5. Targeted Testing It looks more like training than an attempted hacking because in targeted testing “the victim” and “the perpetrator” work in close cooperation evaluating each other’s actions. Such an approach allows the security staff to adopt hackers’ vantage points in assessing their company’s security policies and measures. Whatever methods the testers apply, they cannot do without testing tools. Penetration Testing Tools Today, a plethora of tools that can be used for penetration testing exist, so novices in the field may be at a loss which one to employ. Having accumulated considerable experience in this sphere, our team can narrow your choice to the most efficient pen-testing instruments that are a part of our technological stack. 1. Metasploit This is probably the most widely used tool. Its popularity is explained by two features: it is very powerful (allowing running several tests at once) and automated (little preliminary technical background is required to use it). Metasploit has several modules, each geared to scan a specific system for vulnerabilities and check if it is possible to exploit them. It operates based on an extensive exploit database that contains easy-to-load scripts. In essence, exploit is a code that can bypass security barriers by leveraging known vulnerability or several, and launch a payload - another code, that would provide access to the targeted software and hardware. Currently, Metasploit features over 1,500 exploits, has commercial support from Rapid7, and is compatible with Linux, Microsoft Windows, and Apple macOS. Among its other fortes are the availability of both command-line and graphical user interfaces. The major downside is quite a high usage cost. 2. Burp Suite An expensive application pen-testing suite aimed at security experts. The community edition is cheaper, yet it falls short of the enterprise version in functionality, which explains the latter’s high price (starting at $4,000 per year). In addition to scanning options, Burp Suite offers proxy capturing, command injection, and even an intrusion testing tool (albeit a limited one). It supports the same platforms as Metasploit, moreover, it allows connecting custom extensions created in Java, Python, and Ruby programming languages. Covering more than 100 vulnerabilities, Burp Suite can perform interactive application security tests as well, which, coupled with the previously mentioned benefits, makes it a very solid testing tool for professionals. 3. Nessus Vulnerability Assessment This tool is almost twice as cheap (a little over $2,000 a year) but lower price hints at its more limited nature. Its 450+ templates are honed to detecting vulnerabilities (including default passwords, obsolete patches, open email relays, and out-of-compliance configurations), yet you have to apply other tools to exploit them. However, within its competence Nessus reigns supreme, enabling IP scans, sensitive data searches, open ports analysis, malware detection, and other software flaw identification measures. The numerous assets of Nessus and the policy of encouraging feedback to be later used in optimizing the tool earned it a high repute within its province, with 2 million downloads and 27,000 companies using it worldwide. Its interface takes some time to master, but you can perform very fast scans that may serve as a preliminary stage for the PCI-DSS audit. 4. Apktool Apktool is very specific in its applicability. It is used to reverse engineer the Android binaries. With the help of apktool, a user can decode an app to its original form (including resources, XMLs, and manifests). This allows checking the efficiency of the employed obfuscation methods. The tool also has automation capabilities. Despite being a narrow-scope instrument, apktool is an excellent fit for its restricted niche. 5. Cutter Cutter is another reverse engineering tool. To be more precise, it is a platform based on Radare2. It is more universal than APKtool since it allows reverse engineering executables of all three major desktop platforms (Linux, macOS, and Windows) and some minor ones including BSD, Solaris, and Haiku. Moreover, it supports WinRAR archives, raw binaries, plus a dozen more file formats and over thirty instruction sets, most notably Intel x86 and ARM. One more Cutter’s edge is that it's free - the factor that can be crucial in opting for this tool.  Conclusion In the contemporary world, cybersecurity is one of the primary concerns both for individuals and for organizations. Penetration testing is designed to assess corporate defensive procedures, the preparedness of the staff, discover hidden vulnerabilities, devise counter-measures to probable hacking attempts, and safeguard data, software, and infrastructure. Our team is able to perform high-quality testing, highlight the problematic zones, and provide guidelines for improving the security policy of your company. Contact us to get a professional consultation.

When you start to research the basics of software testing, you often end up puzzled with a diversity of types of software testing. Manual and automated. Functional and non-functional. Load, stress, security, regression… It seems that the list will never come to an end. All types of testing have different purposes and tasks. But does it mean that you have to apply all those types at every project? Or maybe there is a useful cheat sheet that will help to skip something without giving up quality?  The good news is the types of testing needed always depend on a particular project. A good QA team will choose them for your individual project. As a software testing company, we can say that for sure. So here’s the list of methods and types of software testing we typically use for quality assessment and improvement. Each comes with a brief explanation – as brief as it may be, taking into account their diversity.  1. Automated testing  Software code always needs a thorough check, even if it seems perfectly written. You can test code by checking software features when they are ready for use – by installing an app, trying to log in, clicking on the links and buttons, and so on.  This process can be manual, when a person clicks on buttons to check everything, or automated, when a QA engineer uses a special program to run a check. Automated testing requires writing code for the check-up. Automation is good for complex applications with large functionality. Why do you need automated testing? Automated testing helps automate frequently repeated tasks and the largest part of software functionalities. The manual check in these cases is too time-consuming and thus inefficient.  The advantages of automated testing include:  Repeatability. All tests always work the same way, and you can run themthem anytime.  Fast execution. An automated script doesn’t need to look up into instructions and documentation, and this saves a lot of time. Lower support costs. As testers need less time to support the scripts and analyze tests, there are fewer working hours to pay for. Independence. A QA engineer can spend time doing other useful things while a test is in progress. Engineers often leave the tests to run during after hours when the load on local networks is low.  Automation can optimize resources. It helps save time, avoid human error, and maximize the performance of a testing team. When the specialists don’t have to deal with a large number of routine tasks, they can focus on more uncommon tasks that do require human input.  Who needs automated testing?  If you have a long-term project with opportunities for development and scaling, a QA team will need to repeat some tests frequently. The slightest code changes can cause new bugs, and a team has to check everything all over again.  After you write automated tests, you can quickly start them at any time and get highly accurate results. Automation simplifies project maintenance and reduces the costs of testing. When is it better than manual testing?  Remember that automated testing is not an alternative for manual testing. It is an efficiency booster, but not a replacement. A QA team applies always applies both for higher efficiency. The coverage of automation depends on a project type and the peculiarities of business processes in a company. Manual testing can be repetitive and boring. Automation can help to avoid these downsides, as a computer will do everything for a tester. Also, automation saves time, effort, and money. One can run an automated test over and over with minimal effort.  We would recommend automating the following types of testing (where appropriate): Feature testing Regression testing Stress testing Installation testing Configuration testing GUI testing To decide whether automation is relevant, you need to answer the question “do the benefits outweigh in this case?” If you cannot find a single part where automation is feasible and reasonable, it is better to stick to manual testing. Don’t go with automation for the sake of automation only. 2. Manual testing During manual testing, QA engineers don’t use any other tools but hands and documentation. Even before automating the testing of any application, you have to perform a series of tests manually. Without manual testing, we can’t be sure if automation is feasible at all.  So manual testing is a direct interaction of a QA engineer with an application. A person can get immediate feedback about the product, and that’s impossible if you use automated testing.  With manual testing, we can get information about the state of the product much faster. It takes time to write autotests and even more time to change and update them. Why is manual testing important? A QA team can get feedback from a product owner when development is in progress. Let’s say, a person notices some feature in an almost-ready product that is better to change before the release. If you already have prepared tests to check this feature, you need to rewrite them. Updating autotests and checking them will take up valuable time one could use to check the feature. When is manual testing enough? Manual testing fits for any software, while automation is good only for stable systems. Some types of testing, such as ad-hoc or exploratory testing, can only be performed manually.  Manual testing may be a time-consuming and lengthy process, but there is no need for automation on small short-term projects. Besides, automation can be quite expensive at the early stages of development as keeping the tests up to date requires resources. But what to do if you want to regularly add new functionality and keep up with the competitors? Before creating auto tests, always check the product capabilities manually. In this case, manual testing speeds up the process, especially for mobile development.  There is one more highlight of manual testing. When it comes to user interface design, the services are helpless. There is no software smart enough to estimate the nuances in color tones and their effect on a user. 3. Functional testing  The task of functional testing is to verify that the software features meet the functional requirements. Simply put, expectations and reality should match. Product developers provide is what was expected initially.  So when to perform functional testing? When you need to verify the requirements specified in the documentation.  When you need to verify that the app provides expected business processes. QA engineers conduct functional testing based on their personal experience, project documentation, communication with a client, and expectation of app behavior. The advantages of functional testing are the imitation of real-user behavior and wide coverage with a variety of tests. To cover these two aspects, we perform several types of software testing – feature, full-system, and regression testing. Feature testing At this stage, a tester interacts with an app like a real user. Sometimes software features meet the requirements but are difficult to understand for an end-user. To avoid this, a testing team follows a user path by clicking on all buttons and links, filling in forms, entering promo codes, interacting with popups, etc. The main task is to mimic user behavior as clear as possible. Full-system testing  It shows if the system as a whole meets the requirements. In other words, after separate modules of the system are integrated, it is necessary to make sure they are still working as supposed. Sometimes bug-free units are incompatible when brought together and start to act crazy, and we should make sure it is not the case.  Regression testing This is one of the key regular check-ups. Regression testing has a place after every code modification. Sometimes updates affect unrelated features. For example, adding a new barcode in a database of a calorie counter may break the layout of the weekly report page. The purpose of regression is to verify that the latest updates haven’t affected an unchanged part of the functionality.  4. Performance testing  Performance testing is a way to learn how quickly a system or a part of it works under a certain load. It also provides an insight into different attributes of the system quality – scalability, reliability, resource consumption, etc. All projects need performance testing. It is equally important for e-commerce and booking platforms, for fitness and gaming apps. It is essential for mobile, desktop, and web applications.  Users always want to have stable and flawlessly functioning apps. Performance testing is the way to make sure the software meets the expectations. There are several types of performance testings. Each focuses on different aspects of an app’s performance. Load testing  It is the basic form of performance testing. It estimates app behavior under an expected load – an average number of users, a number of simultaneous transactions, etc.  All types of projects need load testing. Before a release, a team should be certain an app will withstand a load. Monitoring databases, servers, and network help detect weak spots. Stress testing  It estimates software reliability during extreme or disproportionate loads and helps you understand the system capacity. For example, e-commerce platforms need to run stress tests before big sales. Streaming platforms needed to run stress tests when users went on the lockdown. Game apps need stress tests before the release of long-awaited updates. Stress testing shows if the system performance is sufficient in case the load greatly exceeds the expected standard. Volume (flood) testing  It allows a QA team to find out how many users a system can handle or what amount of data to process before it loses productivity and the stability will become just unacceptable. We are mainly interested in how an increase in the number of users or data stored affects the time of execution for operations.  It is essential to check the functionalities that process large volumes of data or use complex queries to reach data. For example, some learning apps make all finished lessons available and store them as user data. An app becomes heavy, but it should still be fast.  Stability (reliability) testing The purpose of this type of software testing is to ensure that an app can withstand an expected load for an extended time. A QA team models an increased load to check memory consumption and identify potential leaks. This is how you discover performance degradation that leads to slower information processing and/or longer response time. Failover testing It validates the ability of the system to allocate extra resources under pressure. If the load rises, the software should be able to apply back-up systems to support the functioning. The effect of server failures, reaching a performance threshold, and other reasons beyond our control on user experience should be minimal. It is necessary to keep things going for a while without manual intervention until the affected functionality is being fixed. 5. Security testing There is a difference between security testing in general and two of its types – penetration and compliance testing. It may seem puzzling for some, so let’s figure out what makes them different.  A QA team always conducts an iterative check of software functionality and infrastructure to detect the weak spots. Security testing starts at the initial stages of designing a product. It may include risk assessment, vulnerability scanning, code control, etc. Penetration testing  Penetration testing looks for the ways to outsmart an information system and circumvent its protection protocols. A team gets a report with a list of detected vulnerabilities, attack vectors, achieved results, and recommendations for improvement.  Penetration testing considers not only software and hardware, but also data stored, organizational activities, company documentation, and different business processes. The results of a pen test depend on hardware metrics, staff’s actions, and consistency of operational processes as much as code quality. By the way, we perform only penetration testing. Compliance testing It is used to verify that software meets the general standards. These standards are developed by large organizations like W3C (World Wide Web Consortium) or niche enterprises (for example, when it comes to healthcare software). The compliance check is not a must for all types of software. When the product is serious, this stage can involve a board of regulators and compliance experts working on the product analysis.  6. Usability testing  Usability testing is conducted to assess the convenience, intuitiveness, usefulness, and satisfaction of using the product for the end-user. There are three types of tests we recommend and provide.  User testing It is a close look at the personal experience of using an app or any other type of software. At this stage, a number of people model the behavior of real users to find the drawbacks QA professionals might have missed.  “Missed” is not about being uncompetitive or inattentive, but viewing an app from the point of view of development. Developers and users may have a different understanding of how a “good” app should work. In some way, it reminds a focus group, but user testing is performed on an individual basis. Accessibility testing As a rule, developers adapt an app for a wide range of users, including people with disabilities. Accessibility testing is testing software for compliance with the recommendations of the Web Content Accessibility Guidelines (WCAG) 2.1. to make sure it is suitable for users with disabilities.  Most often, applications are adapted for people with hearing or visual impairments, and sometimes for users who cannot use a keyboard or other manual input device. Accessibility assessment, however, is important not only for people with disabilities.  It is helpful for all uses in certain circumstances. For example, when you are in a noisy environment, you can watch videos without sound but with subtitles. Or on the contrary, when reading is inconvenient, you can use listening mode.  Usability testing People often confuse usability and UX that stands for “user experience”. What’s the difference? UX is the complete experience that comes with a product. It can go beyond features and move into a broader concept of customer experience, supported by all the marketing materials and presentations. Usability is about the product being simple and easy to use. It is usually measured by a number of issues that cause frustration during using a product (or their absence). There are five main criteria of usability: Usefulness shows how well a product solves users’ tasks. Efficiency determines how quickly and easily a user can perform the necessary actions. Learnability illustrates how easy it is for the user to understand the game or application. Satisfaction reveals positive emotions a person gets from the use of a game or an app and the desire to recommend it to friends. Accessibility identifies the ability to use the application for people with disabilities. Levels of testing Every type of testing includes a set of activities each with its own objective, strategy, and deliverables. The goal of each testing type if to verify its particular objective or send the code for re-work. In simple words, you run stress tests to make sure the system can withstand a sudden increase of users. If it can’t, it is necessary to make some changes.  Levels of testing are a different concept. A level of testing determines the area covered – an individual unit, a group of units, or the system as a whole. Testing on different levels takes place throughout the software development life cycle. It is key to successful product release and maintenance. There are four levels of testing:  Unit testing Integration testing System testing Acceptance testing Unit testing studies each separate element of the system. Usually, developers cover these tests, so let’s focus on the rest three levels that belong in our area of responsibility. Integration testing This next level of testing goes after unit testing. It helps figure out whether units function properly together. In other words, we test two or more related units to verify that the integration was successful and no critical bugs popped up. There are two approaches to integration testing that influence the way we perform it: Bottom to top testing. We put together all the low-level modules and test them. After that, the next level of units is assembled for testing. It goes on and on until the functionality is fully covered. This approach is good to use when all (or almost all) units of the level are ready. Top to bottom testing. The team starts to check high-level units and gradually adds low-level ones. QA engineers use module stubs if necessary to simulate missing units of the functionality if they are not ready yet. Later, ready active units replace these components. System testing  The name speaks for itself. On this level, we check the whole system for bugs in it. A tester investigates the relations between all hardware and software components. Then they check the way the system functions.  By this time, the software is ready for a potential release. A QA team conducts tests on all browsers or operating systems and runs different types of testing. We can divide all tests into groups: Functional testing is meant to detect the mismatches between actual and expected behavior of different features. These tests cover all implemented functions and take into account the most probable types of errors. Non-functional testing focuses on software characteristics that can be measured by different values – speed, capacity, etc. It is a set of tests that reveal how the system performs. Acceptance testing Last but not least, there is acceptance testing. This level comes directly before handing software to a customer. It is a final check verifying that a product meets customer’s requirements. Either a QA team or a client’s team can run this check.   There are three main types of  acceptance testing, each with its specific goal:  Sanity testing looks into software performance in detail every time after we get a relatively stable build. It validates that all important parts of the functionality work at a low level. For example, during sanity testing, you check if a user can install an app and log in after an update.  Smoke testing is executed every time we get a new project build that is yet unstable. We need to make sure that all the critical features of the software under test work as expected. The idea is to identify serious problems as early as possible and reject a current build at an early stage of testing without wasting time on obviously defective software. Ad-hoc/monkey testing is an ‘informal’ check that doesn’t require any documentation or planning. It is an improvisation, so a QA team doesn’t need to prepare test cases and scenarios. Ad-hoc is helpful when the deadlines are tight and there is little time for consistent testing. A QA engineer relies on their general understanding of the application and common sense. Bottom Line The goal of software testing is to cover the maximum amount of testing scenarios. The more bugs you detect, the higher chances are to release flawlessly functioning software. There are some types of software testing, like user testing, that any company can handle without professional QA expertise. The rest, however, requires a more serious approach. We hope our article will help you understand better what exactly a QA company checks and keep up with the improvements during the testing stage of SDLC. 

Delivering a product to the market usually includes testing. It is done to ensure that software or application behaves according to the technical requirements and does not contain bugs. There are two types of testing you can choose for your project: manual and automation testing. In this post, we are going to talk about the differences between them and how to select the type that suits your project best. What is manual testing? First of all, let us clarify what manual testing is. This is the procedure of checking that everything in the program works as planned that requires prepared specialists. They click links and input values into fields, verify outputs, and press buttons manually. Their job is to find bugs or other issues and report them in a way so that software developers could find and fix them.  Types of manual testing There are several types of manual testing that you might need for your project: Exploratory testing. In simple terms, exploratory testing is when the development and execution of tests happen at the same time. A tester doesn’t have a predefined script and carries out the procedure without strict accordance with a plan. For example, it is up to the tester to decide what details in the program functionality allow to say that it works incorrectly or choose the speed of filling in the forms. Exploratory testing is all about discovery, where the creative intuition of the tester plays a crucial role.  Ad-hoc testing. The previous type of testing is often confused with ad-hoc testing. Ad hoc is improvisation, spontaneous search for errors. It is done without planning and documentation and can be applied for both research and commercial projects. Usability testing. Usability testing checks whether the product is convenient to interact with. It is essential to conduct usability testing at every stage of development to see how real users will perceive your product. During the experiment, a test group is asked to complete typical tasks to see whether the product behaves according to their needs and expectations. User acceptance testing. User Acceptance Testing (UAT) is conducted by the end users of the system to make a decision whether the product is ready to be launched. It checks whether the system works according to the user scenarios. This is the final stage of testing. The UAT should happen with real users, the stakeholders don’t fit in this group. The process of getting approval from the client is called stakeholder feedback. As you can see, manual testing can be applied to a wide range of cases. Now let us see where exactly it is the most beneficial. The benefits of manual testing MANUAL TESTING Pros Cons easy hiring (anyone can test) may not detect all defects versatile, can be applied to many test cases for complex cases demand high expertise close to real user feedback takes a lot of time flexible, easily adapts to changes in the UI lower quality in case of large volume tests demands almost zero investments not reusable Manual testing is a classical approach to software quality assurance. However, there are cases when human presence strongly outperforms scripted testing: Costs-efficiency  In the short run, manual testing is less expensive and more cost-efficient. The threshold of this profession is low. Companies can hire beginners and train them in a small amount of time. Almost anyone can become a manual tester. With this kind of testing, you don’t have to invest in expensive software and a team of qualified software developers to get good results. Versatility Human judgment and intuition allow checking the components of the software that require expertise a computer cannot provide. In difficult cases, humans do much better than machines.  Real user feedback Automation testing has many benefits, but there are some areas where they become useless. Human expertise is valuable to provide real feedback about the usability and convenience of the user interface. You can get the opinion about the product approximate to one of the users if you attract testers to the process.  The disadvantages of manual testing There are cases when manual testing has some drawbacks. In some situations, you will have to assess the pros and cons of each testing approach and choose the more cost-efficient one. Human factor This is the reason why automation testing appeared in the first place. The chance of error or not finding bugs is higher when you’re working with people. They might be biased, tired, or unqualified. You need to invest in the development of your personnel or look for qualified specialists if you want to be sure of your software quality. Limitation in volume  In case you need to test a large-scale software system, manual testers will not be able to deal with such volumes. You can expand your team and hire more manual testers. However, if we are talking about products that are comparable to a federal healthcare system or aircraft control systems, it is more cost-efficient to automate at least part of your tests.  Time-consuming If you only use manual testing for all quality assurance work, it might take a lot of time. The employees will have to check every element and report problems, which takes more time than just running a script. What is automated testing? Now let us see what we understand by automation testing. Unlike manual testing, it happens with the help of some programs called scripts. They conduct the validation of programs automatically and do not require manual control. However, they imitate the manual way of validating applications. Their goal is to make sure that test execution is completed in a minimal amount of time.  Automation testing needs software programmers to work on the project. They write the tests suitable for a particular project and monitor its execution.  Types of automated testing Here are some common types of automation testing: Data-driven testing. A data-driven approach to testing implies that the test data is stored separately from scripts in a database or Excel file. Imagine that you need to create ten different records with the help of your test application, i.e., perform the same actions ten times but with different data. You can put the data in an array or a separate file. The second approach is very convenient: all the data is stored in one place. When you need to use it, you simply switch between different data, rather than search for it in the project file. Load testing. Load testing is done to check whether the system behaves normally under both normal and peak load conditions. You can use it to find out the maximum operating capacity of your system and eliminate any bottlenecks and problems in the early stages. Performance testing. Performance testing is conducted to check the speed of response and stability of a website or application. It also allows you to access essential parameters like usability and scalability. Regression testing is software testing when you select some or all the test cases that were already performed and perform them again to make sure that works fine. The pros of automation testing Automation testing provides many benefits that are important for software development companies and their clients. Speed Automated software testing allows us to perform testing faster and check larger volumes of data. One script can do the work of multiple manual testers in one working day. When you need to deliver the product to the market as fast as possible (for example, in the conditions of high competitiveness), automation testing is your best choice. Accuracy and reliability Automation testing software that was written by professional doesn’t miss bugs and reports all issues in a way that is easy to find. Human testers might be inattentive or limited in knowledge. However, the algorithm will always behave according to strict requirements. From this point of view, you can say that automated testing is more accurate because it allows finding standard bugs easier and not to miss a thing. Reusability It is possible to use the same testing scripts on different software or software components with no (or almost no) alterations. Automation software requires investments of time and money at the beginning but, in the long run, has hired reusability than manual testing.  Effectiveness for large-scale applications As we have mentioned above, the software allows testing data of larger volume while there is a limit to human possibilities.  The cons of automation testing  There are two major disadvantages of working with automation testing: Lack of flexibility Tests should be entirely changed if there was a change in user interfaces. Humans are flexible; however, if you want to get accurate results after introducing changes to the interface, rework the script. Lack of creativity Automation doesn’t work well with nontrivial problems. Some problems are too complicated to script and require human interference.  Manual vs Automation testing: what to choose?  The moral is that every approach to testing has both strong and weak points.    Factor Manual testing Automation testing Costs Cheaper in the short run Cost-efficient in the long run Accuracy Can be less accurate and error-prone due to human factor Scripts allow to more accurately detect and track bugs Speed More time-consuming Fast and ideal for repeated testing/ large-volume testing Area of use Works for non-trivial issues where human expertise is required  Does not guarantee high-quality user experience Resources Focus on human resources Focus on automation Flexibility Very flexible and can be used in a wide variety of cases Might not be effective when there are changes in the interface (requires changes in the script) When go for automated testing Automation testing provides high reliability and accuracy. It allows us to test software systems of any volume, and we cannot go without it when working on large projects. Automation testing is an excellent solution to streamline the time to delivery without compromising on quality.  However, you will never be able to automate every task. When you need manual testing Manual testing shines when you need to work on a wide variety of situations. For complicated cases and usability testing, human participation will always be required.  Summing up  In this post, we’ve covered the pluses and minuses of manual and automation testing and discussed the cases when each of them works best. Stay tuned to our blog for more mind-blowing materials from the world of software quality assurance. ---  If you are looking for qualified software testers who are proficient at both kinds of testing, contact us. We will analyze your case and propose testing strategies that suit your situation.