Understanding Quality Assurance Metrics vs Quality Engineering Metrics | mabl

Nothing can be improved until it’s measured, which is why metrics are such a critical part of managing quality. Unless quality teams have the tools to consistently measure and communicate the state of their product, the development process will slow and users will ultimately have a worse user experience. 

Metrics also play a critical role in supporting digital transformations like DevOps adoption and the transition to quality engineering. Considering that most changes stall due to cultural resistance and people’s reluctance to embrace change, the right metrics can help convince stakeholders to continue learning new processes. For quality teams, framing their journey to quality engineering in metrics can be the key to long-term success. 

Avoiding the QA versus Quality Engineering Mindset

Quality assurance versus quality engineering isn’t an all-or-nothing proposition. Many teams will find themselves balancing aspects of both disciplines as they identify new opportunities for transformation along the development lifecycle. The two even have shared goals and outcomes: 

Quality assurance is a systematic process of determining whether a website or application meets designated requirements through a manual, hybrid, or automated testing strategy. It’s chiefly concerned with ensuring a seamless user experience by recreating all possible customer journeys. 

Quality engineering is the practice of manual and automated testing throughout the development lifecycle, with the purpose of delivering a positive user experience that will help companies satisfy, retain, and acquire new customers. 

Both QA and QE are deeply concerned with the user experience and actively work to improve customer satisfaction with each new release. This common ground is why metrics like test coverage are relevant for both disciplines, provided the quality team has a strategy in place for ensuring test coverage reflects the customer experience. But how they accomplish this goal is another matter – and another set of metrics. 

QA: Measuring the End Result

Since QA is primarily concerned with measuring quality against a set of specified requirements, QA metrics are more closely tied to overall product quality and how users perceive quality. To assess the success of a QA strategy, testing teams might consider how long it takes to run a test plan, how often QA is holding back releases, and customer-centric metrics like churn rates,  or satisfaction rates. Rather than measure quality as code moves through the SDLC, QA is concerned with measuring how testing impacts the near-finished product and customer happiness. Though these metrics give an accurate picture of how testing impacts user satisfaction, they aren’t enough when working to enable DevOps or other digital transformations. 

QE: Measuring the Process

In contrast, quality engineering has a wider focus that seeks to understand quality as code moves through the SDLC. By shifting testing to the left, QE is able to quantify issues earlier in development in a more nuanced way. 

To that end, QE metrics are embedded into development pipelines and aligned with the entire SDLC. Automated tests of all types (functional, integration, E2E, performance, load, API, etc.) are created to find defects earlier in the pipeline so issues can be fixed at a much lower cost, compared to escaped bugs that are first reported by customers. This way, the entire defect discovery and resolution process – issue identification, the delivery of bug fixes, and ensuring new features don’t break existing features –  is accelerated. Developers and quality engineers alike then have more time to dedicate to building new features and improving overall quality. 

Capturing this holistic approach to quality requires a wider set of metrics, chiefly test coverage across high-demand user journeys, the time it takes to resolve bugs, what types of tests are failing, and longer term performance trends. 

Quality Engineering is a Journey, Not a Destination 

The difference between QA and quality engineering is steps, not miles. As QA teams expand their automated testing strategy, enhance their manual testing, and shift testing to the left, they’re able to measure new aspects of quality. When the quality is better quantified across the SDLC, quality teams are empowered to embrace quality engineering and enable DevOps adoption. 

Start your quality engineering journey with mabl’s two week free trial.