Quality gates

A high level guide of setting up quality gates in software development process

Intro

Quality gates is a part of the development process consists of setting up specific, measurable and achievable criteria of success for each SDLC phase and restricting project to proceed without passing these criteria.

Enabling quality gates enforces the improvement of each development stage, making the process more transparent and traceable in terms of quality.

Also, it helps to avoid waste of time, pushes to test earlier and minimize the price of bugs fixing.

Diagram of quality gates in SDLC phases

In general, SDLC looks like represented below.

The output of each phase is a set of artefacts which then serves as an input for a later stage.

And each of these artefacts should have their quality gates.

Examples and explanation of such gates:

Examples of quality gates in each phase

Let’s go through the phases and see what can be there in quality gates.

Requirements

Requirements are one of the first in SDLC phases. It explains how functionality should look like, and the quality of requirements is impacting the quality of a product.

Because of that, it should be refined and clarified by many parties, such as PO, developers, testers, and checked by many criteria.

That will help to avoid implementation of wrong functionality and waste of time.

As example criteria may include and not limited by: prioritized, verifiable (acceptance tests), realizable, understandable, unambiguous, consistent (with other requirements), complete, necessary to a user, technology-independent, concise.

Additional reading https://www.pjsrivastava.com/a-short-guide-to-writing-software-requirements

Design

Design is a process of planning of how the software will solve the task from requirements.

It includes a selection of platforms, frameworks, libraries, protocols, patterns. It does not include code writing but instead defines a skeleton/architecture of needed components/systems.

Quality gates for design may include and not limited by: selected tools/frameworks/libraries are tested for the goal (functionality/performance/security), the design is realizable, understandable, unambiguous, consistent (with the rest of software), concise, SOLID, KISS, DRY, solution should be testable, observable (logging/monitoring).

Additional reading: https://en.wikipedia.org/wiki/Software_design

https://www.intertech.com/Blog/principles-of-good-software-design/

https://www.freecodecamp.org/news/how-to-write-a-good-software-design-document-66fcf019569c/

Code

Software product consists of program code, and it’s quality directly impact the quality of the overall product. In comparison with other QG code gates can be mostly automated in CI/CD pipelines and thus less probably forgotten.

Code quality gates may include and not limited by: static analysis and composition analysis (Sonarqube, Snyk, Blackduck, Veracode, Checkmarx, linters, style checkers, etc.), code coverage, mutation coverage, peer reviews.

Example of quality gates in SonarQube:

Additional reading: https://docs.sonarqube.org/latest/user-guide/quality-gates/

https://en.wikipedia.org/wiki/Static_program_analysis

https://en.wikipedia.org/wiki/Code_coverage

https://opensource.com/article/19/9/mutation-testing-example-definition

https://medium.com/palantir/code-review-best-practices-19e02780015f

Acceptance Testing

After the code is written and tested from a technical perspective, it can be built into a releasable artefact and tested from end to end perspective. Also, the stage may include UAT, release planning, sign off procedure.

Code quality gates may include and not limited by: E2E tests are automated, builds are available in artifactory, the release date is planned and agreed with others (teams/stakeholders/releasers/support), sign off procedure is followed, and documents are published in common places.

Additional reading: https://www.browserstack.com/guide/end-to-end-testing

https://softwarehut.com/blog/it-project-management/project-sign-off-document

Release

After everything is ready it is a time to release the software.

And after release it make sense to make sure that the released features are working and maintainable.

Release quality gates may include: feature test on production, synthetic monitoring, logs monitoring, APM.

Additional reading: https://en.wikipedia.org/wiki/Synthetic_monitoring

https://www.datadoghq.com/blog/introducing-synthetic-monitoring/

https://en.wikipedia.org/wiki/Application_performance_management

Where to start with the gates and which are more important?

There is one of the main software testing principles saying that testing should start as early as possible.

According to that requirements seem to be the first candidate for this initiative.

Also, it would not much help if you have great quality gates in one or a few stages, and others are left empty.

Look at this as a task of securing a house. If you have super strong big gates and no fence around it, it will not work.

So the end goal is to have proportional, balanced, secure gates on each stage and refine them periodically to avoid holes.

Additional readings: https://www.guru99.com/software-testing-seven-principles.html