Improving the Testing Process for the Versational AI Platform
Client
Inteligems offer Versational, an AI sales platform
Client Goals
Improve efficiency of the testing process
Increase test coverage
Speed up releases
Reduce the number of bugs in the release version of the application
Project overview
Client
Inteligems
Region
USA
Industry
Sales and marketing with AI
Type of project
Web app
Scope of work
QA Audit
Company size and stage
Midsize Business
Company profile
Inteligems offers an AI platform called Versational that analyzes recorded virtual meetings and recommends how to improve them. Versational goes beyond automated meeting notes to guide sales and account teams in communicating positive customer value during buyer conversations.
Technical Product Manager Jonah Kingery and his team wanted to improve the QA process for the AI platform to reduce bugs in the product.
We are looking for a software testing engineer to help us shore up our processes and release us from technical debt. We know generally what we want and how it will fit into our process but also really appreciate any industry expertise that you have to give us advice on how to improve as well. Our main goals are to increase test coverage and reduce bugs in the next month.
Jonah Kingery
Technical Product Manager at Inteligems
1 project manager
Agile approach
3 developers
No CI/CD process
1 designer
No QA strategy
Challenges
While working with Inteligems, we faced several challenges in that Inteligems lacked.
A QA strategy
QA estimates: requirements validation, test design estimation, deep testing estimation, automation coverage estimation
Bug analysis and, as a result, an increasing number of bugs
Collaboration process
On the initial call, we talked with Jonah about his goals and offered a QA audit as a starting point for our collaboration.
During a meeting with the Inteligems team, our QA Analyst interviewed them and identified the high-level quality challenges that the team experienced.
To create a strategy for improving the testing process, our team conducted a QA audit and explored their current QA processes.
Summary after the initial call of UTOR CEO and Inteligems Technical Product Manager
Comments that Jonah left on the presentation
After the QA audit, our QA Analyst prepared a report that discussed the following elements and presented it to the Inteligems team.
- Teamwork and communication
- Test Strategy
- Environments
- Monitoring & Alerting
- Task and bug flow
- DevOps
- Test Management
- QA Automation
- Test Documentation
- Non-functional types of testing (security, performance, usability)
Solutions
We created test documentation based on product and project specifications. The test documentation included:
- A testing strategy
- Defining which test types to provide and their domains:
- Functional testing (UI/UX acceptance testing, cross-browser testing, “happy flows,” end-to-end flows, negative flows, user-based testing)
- Exploratory testing (boundary values testing, negative testing, error handling, simulating critical errors)
- Automated testing (UI and API)
- Performance testing
- Daily task checklists
- Daily checklist for debugging
- Bug regression suite to test most critical and reoccurring bugs
To reduce an increasing number of bugs in the production environment, we established a bug management strategy:
- Update the bug flow and definition of done for QA activities related to tasks and bugs
- Define tools to monitor performance, testing, and management.
- Create a sophisticated test data library & test user mapping (personification).
- Extend automated testing coverage for regression suites and consider creating a list of manual regression tests.
- Combine replication with bug analysis to understand patterns and bug types.
- Implement a Zoho dashboard for organizing reports and better transparency.
We created an action plan for improving the Versational QA process. It included all aspects of the QA audit and was the basis for further testing.
To implement the QA activities from our action plan, we assigned one full-time QA to conduct manual testing and one full-time QA with knowledge of Cypress to conduct automated testing.
Results
Critical bugs were reduced to 30%
Increased test coverage from 0% to 90%
Stable releases every two weeks according to sprints
It was very helpful to have this audit process at the beginning of our contract because we knew we had some problems to solve on the testing side but not what they were or how to solve them. If we had just started hiring testers and giving them work to do, I wouldn’t have been sure what our overall plan was. But assessing our workflow and urging us to get this documentation together and what our problems could be was very helpful, along with summarizing all of the findings in a nice presentation.
Jonah Kingery
Technical Product Manager at Inteligems
Ready to extend your QA team with our software testing experts?
Schedule a call so we can discuss your project.