Mục lục

DevOps— Software Quality Control Using Jenkins CI/CD Pipeline

Essential Tips To Software Quality Assurance for Java Technology

Photo by Joel Fulgencio on Unsplash

Building automated unit tests as part of application code development is undeniably a great practice as you can run the tests frequently to verify system logic whenever you change the code. Unfortunately, testing alone is just one of the dimensions of software quality. Taking care only a single dimension does not guarantee a high quality deliverable.

Testing mainly covers software functionalities while there are other perspectives such as reliability, security, maintainability and more. Your application would become faulty if those areas are ignored. Furthermore, your test cases might not fully cover all scenarios as the result of complicated system logic or time constraint during test case development.

In this article, I’m going to share the way how to tackle software quality by making a set of tools in addition to testing. I will demonstrate the whole setup in a CI/CD pipeline for an example Java application.

There is a number of tools available in the community. Some are mature and widely adopted in the industry. You will find the implementation for software quality control is worthwhile as the setup not a difficult task but the benefit is immense regardless of your project size from a personal sole project to a team development.

At A Glance

Here is the list of topics to be covered in this article for software quality control:

Unit Test Coverage — Find out which system logic is not yet covered by your test cases.
Static Code Analysis — Inspect your code for possible issues related vulnerability, security and maintainability.
Frequent Code Style Check — Check your code style frequently while you’re writing code.
Automated Pipeline With Quality Gates — Build and analyze your code in a central place, a gate keeper of code quality prior to delivery.

Tooling

For Java technology, the following tools will be used in pipeline implementation:

Maven — It is the main tool for source build and test execution.
JaCoCo — It gathers unit test coverage information and generate report
Sonarqube — It plays the key role in source code analysis and pipeline process will be terminated if source code fails to meet quality gate standards.

Unit Test Coverage — JaCoCo

White Box Testing

Instead of treating a component as a black box to examine its functionalities, unit test is usually a kind of “White Box Testing” for developers to inspect all internal program flow and validate their code logic. It is an effective and exhaustive method to make sure all system logic are well tested.

Test Coverage

How do you know whether your unit test cases fully cover all system logic? It is a time consuming task to check especially there is a number of branches in your program logic. JaCoCo is a great tool for the coverage inspection.

To check the coverage of your unit test case, you can simply run unit tests and JaCoCo automatically captures program instruction of test execution behind the scene. As a result, a HTML report is generated showing percentage and other statistics regarding test coverage. You can easily identify the areas with low coverage and then navigate down to class source code level to see exactly which parts are not yet covered.

Application level overview

JaCoCo Report — Application Level (Screenshot captured on my workstation)

Class level with list of methods

JaCoCo Report — Class level with list of methods (Screenshot captured on my workstation)

Source code level with coverage highlighted

JaCoCo Report — Source Code with Coverage Highlighted (Screenshot captured on my workstation)

The setup is pretty simple, it is just a plugin configuration in Maven pom.xml. Thus, you can enable it by adding the following lines to your Maven pom.xml and you will find the report in target/site/jacoco after test case execution.

JaCoCo — Maven Configuration

Comprehensive Code Analysis — Sonarqube

The root cause of software defects is due to not following a good coding practice or misuse of certain method calls. Some inappropriate coding practice would lead to potential security issues. However, it is a massive effort to review source code manually because there would be several hundred coding practice to be aware of.

Hence, you probably need a server side component with the capability to carry more advanced analysis and store the result. Sonarqube is popular tool definitely fits for this purpose.

Once you’ve submitted code to Sonarqube for scanning, you can access to the report on its web portal. The example below shows the results in categories — Reliability, Security, Security, Maintainability, Code Duplication as well as Test Coverage.

It comes with quality gate which indicates the overall status Pass or Failed. Maintainability status is an interesting figure which illustrates the indicative effort to fix code smells. The team can perhaps take it as a reference to plan backlogs for technical debt clean up.

Sonarqube Report — Overview (Screenshot captured on my workstation)

As Sonarqube stores all previous scan result, it clearly shows the quality trend of recent commits. It is an important information to the team and let them which area they should pay more attention to.

In the sample diagram below, new code commits in beginning introduced substantial technical debt followed by some improvements and the situation became stable in latter code deliveries.

Sonarqube Report — Historical View (Screenshot captured on my workstation)

Similar to SonarLint, it displays and highlights issues in source code level.

Sonarqube Report — Issues in Source Code Level (Screenshot captured on my workstation)

It is a headache if you get a list of issues without any clues how to handle them. No worry, the scanner provides explanation and suggested solution for your convenience.

Sonarqube Rule Explanation & Suggested Solution (Screenshot captured on my workstation)

Beside issues highlighted in source code level and a list of issues, Sonarqube provides other diagrams which would give you an insight in different perspectives.

Sonarqube Report — Insights of Different Perspectives (Screenshot captured on my workstation)

Setup

Deploying components using docker is a handy way and it can be done in a few minutes. Please refer to docker compose definition below:

gavinklfong/docker-compose-collection

A collection of docker compose for various cases. Contribute to gavinklfong/docker-compose-collection development by…

github.com

You can clone git repository and run docker compose command to to spin up the environment:

> git clone https://github.com/gavinklfong/docker-compose-collection.git > cd cicd-stack
> docker-compose up

Once your sonarqube is up and running, all you need to do is to add configuration to Maven and submit source code files for scanning. Then, the result will be available on Sonarqube portal in a few minutes. You can refer to link below for detail of Maven configuration:

SonarScanner for Maven

The SonarScanner for Maven is recommended as the default scanner for Maven projects. The ability to execute the…

docs.sonarqube.org

Frequent Code Style Check — SonarLint

The earlier you can spot an issue, the easier you can get it fixed and a smaller impact to your project. You can run code analysis frequently as you writes code on IDE using SonarLint prior to your code submission to sonarqube.

SonarLint is part of Sonarqube, it is an IDE plugin which is available to many popular IDEs such as Eclipse, IntelliJ and Visual Studio Code. Obviously, IDE is perfect tool to alert you of any issues while you are writing code, it would just take you minutes to rectify any mistakes. Nevertheless, this tool is your coding companion but it is not to replace Sonarqube which offers a more comprehensive code analysis.

Here is the sample screenshot on Eclipse with issue highlighted in source code:

SonarLint Issue & Source Code View (Screenshot captured on my workstation)

Automated Pipeline with Quality Gates — Jenkins

Enforcing quality control is essential to ensuring delivery up to standard. By setting up automated CI/CD pipeline with quality gates, the output quality is guaranteed as quality standard is strictly enforced and no substandard code can be deployed without passing the quality control.

Pipeline Design

Let’s make it clear the requirement about what steps are going to be executed in the pipeline. Exit criteria of the current step should be met before the execution of the next task in the pipeline. Here is the exit criteria of each step:

Unit Test — All unit test case passed
Code Analysis — All quality standard of all dimensions met. The following dimensions are involved in code analysis: Reliability, Security, Maintainability and Unit Test Coverage.
Integration Test — All integration test case passed
End-to-end Test — All end-to-end test case passed

Quality Gates in CI/CD Pipeline (Diagram created by myself)

Pipeline Implementation

Pipeline steps are defined using Groovy script, the script is declarative and intuitive, so you will be able to pick up the logic quickly.

Pipeline Definition

As you can see in the Groovy script, dedicated Maven profiles are defined for each pipeline step so that the target group of test cases will executed accordingly.

Unit Test — Run all test cases with tag “UnitTest” by Maven surefire plugin
Integration Test — Run all test cases with tag “IntegrationTest” by Maven surefire plugin
End-to-end Test — Run target test case java class by Maven failsafe plugin. It is a cucumber test, it will automatically execute all feature files.

Despite Jenkins offers a user friendly GUI for the pipeline setup, I highly recommend the use of script as you can check-in the script file to Git repository for the benefit of version control.

Jenkins Pipeline Configuration — Pipeline script from Git (Screenshot captured on my workstation)

In my example, you can find the Jenkins groovy script in Git repository as part of application source.

Pipeline groovy script in Git (Screenshot captured on my workstation)

Pipeline In Actions

This example demonstrates the scenario that source code does not pass quality gate at Sonarqube. Since quality gate is NOT passed, so you can see that the pipeline abort all subsequent steps.

Jenkins Pipeline — Execution Failed, Quality Gate NOT Passed (Screenshot captured on my workstation)

To fix the code issues, let’s inspect the detail on Sonerqube report. It offers a convenient way to navigate to the problematic source code with the issues in detail

Sonerqube — Issues are highlighted in source code level (Screenshot captured on my workstation)

Once the issues have been fixed, we re-run the pipeline, you can see that quality gate is passed and the pipeline can be fully executed.

Jenkins Pipeline — Successful Execution with Quality Gate Passed (Screenshot captured on my workstation)

Git Repository

You can refer to this git repository for the example application with Jenkins groovy script and Maven pom.xml if you are interested in pipeline implementation

gavinklfong/reactive-spring-forex-trade

This repository was created with the aim of demonstrating the prevailing non-blocking technology of Spring framework…

github.com

Final Thoughts

No matter the scale of your development project, taking multiple dimensions into consideration is essential to delivery of high quality digital products. In other words, functionality is just a single perspective and you should look into security, maintainability as well as coverage of your test cases.

There are many great tools available such as code scanning and automated CI/CD pipeline with tollgate control. Therefore, you can establish and automate the quality control process quickly without a huge investment on the setup. More importantly, quality control process together with a team spirit with mindset of achieving high quality products will definitely make a difference.

If you are interested in building pipelines on a cloud-based platform, then the article below is for you as it covers comprehensively how to build the same pipeline with quality gates on a popular cloud-based platform — CircleCI.

CircleCI — The Modern Cloud-Based Platform of Pipeline Automation

A Journey from Jenkins to CircleCI on cloud computing

medium.com