SonarQube is a server that allows to track coverage statistics, find bugs in your code and more. It is language-agnostic and can be installed on premises, and you can integrate it easily with Buddy.
For the sake of example, in this article we will use JavaScript as a sample code language.
This article will guide you step by step through the configuration process. If you already have a repository with tests/coverage set up, you can skip to the SonarQube configuration part. If you have SonarQube installed as well – you may skip to the integration part.
Actions used in this guide:
Node.js
Set up code and tests with coverage
First, let's create the repository on Buddy. It will be a simple Git repo.
Create a new Buddy repository
Now let's clone it:
$ git clone https://app.buddy.works/yourname/yourrepo
$ cd yourrepo
Copy
Bash
Let's quickly set up something testable to have a coverage to upload. We will use jest as our test command:
$ npm init # use 'jest --coverage' as test command and 'src/index.js' as entry point
Copy
Bash
Now, let's install some dependencies:
$ npm install jest @types/jest sonar-scanner --dev
Copy
Bash
Jest is a test/coverage tool, and Sonar Scanner is a tool that uploads the coverage. Also, we installed Jest types for better code completion as all major IDEs support it.
Now let's make a sample code & test:
$ mkdir src
Copy
Bash
Create a file src/index.js with the following code:
// src/index.js
exports.fn = arg => {
if (arg < 0) return 0;
return arg + 1;
};
Copy
JavaScript
Create a test file src/index.test.js:
// src/index.test.js
const {fn} = require('.');
describe('fn', () => {
test('adds 1', () => {
expect(isCovered(1)).toEqual(2);
});
});
Copy
JavaScript
Now, if you run the following command:
$ npm test
Copy
Bash
You will get an output like this:
> buddy-sonar@1.0.0 test /buddy-sonar
> jest --coverage
PASS src/index.test.js
isCovered
✓ adds 1 (3ms)
----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files | 75 | 100 | 50 | 100 | |
index.js | 75 | 100 | 50 | 100 | |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: 1.331s
Ran all test suites.
Copy
Plain Text
Also, note that the coverage directory has been created in the root of your cloned repo.
Now let's create the Sonar Scanner config file sonar-project.properties:
sonar.projectKey=buddy
sonar.projectName=Buddy
sonar.sourceEncoding=UTF-8
sonar.sources=src
sonar.exclusions=**/*.test.ts
sonar.tests=src
sonar.test.inclusions=**/*.test.js
sonar.javascript.coveragePlugin=lcov
sonar.javascript.lcov.reportPaths=coverage/lcov.info
Copy
Plain Text
Let's add the coverage folder to .gitignore along with Node modules:
coverage
node_modules
Copy
Plain Text
The next step is adding our files to Git and pushing them to master:
$ git add .gitignore src/index.js src/index.test.js package.json package-lock.json
$ git commit -m Init
$ git push origin:master
Copy
Bash
Install SonarQube
For the sake of simplicity, we will use a local installation of SonarQube using Docker and put it online using Ngrok service. This kind of installation can be easily repeated elsewhere if you have a Docker instance deployed somewhere.
Thie first thing is installing Docker if you haven't done that already. Docker is a virtual machine manager that allows running virtual images with specific software installed as if it is a physical computer. Installation is very simple – just follow the docs on the site.
The next step is to run the SonarQube Docker image:
$ docker run -d --name sonarqube -p 9000:9000 sonarqube
Copy
Bash
You will see the following output:
$ docker run -d --name sonarqube -p 9000:9000 sonarqube
Unable to find image 'sonarqube:latest' locally
latest: Pulling from library/sonarqube
8d691f585fa8: Pull complete
3da6fe7ff2ef: Pull complete
e22147996cc0: Pull complete
8df48a2d4467: Pull complete
06eb74af83c0: Pull complete
a642409dc81e: Pull complete
778617ae58c7: Pull complete
78e3d611ddbb: Pull complete
ec0d78b01f70: Pull complete
Digest: sha256:03681e6bb9de5ca4192e9c9b5035e0cc84404dbc107bb7069ca95152dca5f945
Status: Downloaded newer image for sonarqube:latest
854ae293f9003011fae39b757b8bf6f4d0fbbb7f7eb6a0a30f53d1aa1dfd0d19
Copy
Bash
If you see no errors it means that the server is up and running.
Once this is done, follow these steps:
Open http://localhost:9000 in your browser
Click Log In and use admin as the username and password
Click + in the upper right corner → Create new project
Enter the project key buddy and the project name Buddy and click Set Up to proceed
Give a name to the token: buddy-token and click Generate
Copy the created token, it will look like this: buddy-token: xxxxxxxxxx, you will need it later
Click Continue to proceed
Now, we need to install Ngrok. It will expose your 9000 port. You might need to sign up for the account if you haven't done that already.
Use it with care, it makes your local installation accessible. It is recommended to change the admin's password at the very least.
Now we can expose the Sonar Server by running this command:
$ ngrok http 9000
Copy
Bash
Your console will produce an output like this:
ngrok by @inconshreveable
Session Status online
Account Kirill Konshin (Plan: Free)
Version 2.3.29
Region United States (us)
Web Interface http://127.0.0.1:4040
Forwarding http://yyyy.ngrok.io -> http://localhost:9000
Forwarding https://yyyy.ngrok.io -> http://localhost:9000
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00
Copy
Bash
Copy the URL https://xxxx.ngrok.io – you will need it later.
Integrate SonarQube with Buddy
Now we're ready to configure a pipeline that will automatically run the tests on every push to the repository.
You can skip to the summary if you don't want to configure the pipeline via the GUI or would rather use a ready-to-use config file instead.
Buddy will automatically detect the type of files in the repository and label the project with corresponding logos.
Repository overview
Click Add a new pipeline to begin. Enter the name of the pipeline, select the branch that you want to test, set it to run on push:
Pipeline configuration
Configure the action
The pipeline is ready, now we need to add some actions to it. Click Node.js from the action roster:
Adding the Node.js action
A modal will launch with configuration details. Add npm run coverage and set up SonarQube credentials in addition to the default scripts :
npm install
npm test
npm run coverage -- -Dsonar.login=$SONAR_LOGIN \
-Dsonar.host.url=$SONAR_HOST_URL \
-Dsonar.links.homepage=$SONAR_LINKS_HOMEPAGE \
-Dsonar.links.ci=$SONAR_LINKS_CI -Dsonar.links.scm=$SONAR_LINKS_SCM
Copy
Bash
The action should look like this:
Action configuration
To use the "sonar.branch.name" property and analyze branches, the "Developer Edition" or above is required. See https://redirect.sonarsource.com/doc/branches.html for more information. You can omit it to only use the master branch. If you want to enable it add -Dsonar.branch.name=$BUDDY_EXECUTION_BRANCH.
Set up environment
Sonar Scanner needs Java to run, so we will have to use a custom Docker image with both Node.js and Java. To do so, switch the tab to Environment and set the image to ringcentral/web-tools and the image version to alpine:
Action environment configuration
You may remove everything from the Customize environment section as we won't be needing that.
Set up cache
Sonar Scanner's performance can be improved by enabling the action cache. Switch to the Cache tab and add the following paths to the Additional cache section.
/buddy/sonar/.scannerwork
/root/.sonar/cache
Copy
Bash
The tab should look like this:
Managing cache
Set up variables
Now we need to configure the ENV variables needed for the coverage to upload. Switch to the Variables tab and add the following variables:
SONAR_LOGIN — the SonarQube token (buddy-token's value xxxxxxxxxx) that you've obtained earlier
SONAR_HOST_URL — the Ngrok URL https://yyyy.ngrok.io that you've obtained earlier
SONAR_LINKS_HOMEPAGE — the URL of your Git repo at Buddy: https://app.buddy.works/yourname/yourrepo
SONAR_LINKS_SCM — the URL of your Git repo at Buddy: https://app.buddy.works/yourname/yourrepo
SONAR_LINKS_CI — the URL to the pipelines section of your Buddy project: https://app.buddy.works/yourname/yourrepo/pipelines
Adding a variable
For security reasons, you should encrypt sensitive data like SONAR_TOKEN.
The complete page should look like this:
Buddy variables page
Add npm script
Now let's go back to the code editor and add a few things to make the setup work. In order to upload the coverage, we need to create an npm script. Add the following to your package.json:
{
"scripts": {
"test": "jest --coverage",
"coverage": "sonar-scanner"
}
}
Copy
JSON
Running Pipeline
With everything in place, we're ready to give the pipeline a test ride. Make a push to the associated branch or click the Run button to initiate the pipeline. A progress bar will appear:
Pipeline progress
You can click the actions within to take a look at how the execution is going, as well as browse execution logs once it's over:
Action logs
A proper console output should look like this:
npm run coverage -- -Dsonar.login=******ENCRYPTED****** -Dsonar.host.url=https://b4170e1e.ngrok.io -Dsonar.links.homepage=https://app.buddy.works/kirillkonshin/sonar -Dsonar.links.ci=https://app.buddy.works/kirillkonshin/sonar/pipelines -Dsonar.links.scm=https://app.buddy.works/kirillkonshin/sonar
> buddy-sonar@1.0.0 coverage /buddy/sonar
> sonar-scanner "-Dsonar.login=******ENCRYPTED******" "-Dsonar.host.url=https://b4170e1e.ngrok.io" "-Dsonar.links.homepage=https://app.buddy.works/kirillkonshin/sonar" "-Dsonar.links.ci=https://app.buddy.works/kirillkonshin/sonar/pipelines" "-Dsonar.links.scm=https://app.buddy.works/kirillkonshin/sonar"
INFO: Scanner configuration file: /buddy/sonar/node_modules/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /buddy/sonar/sonar-project.properties
INFO: SonarQube Scanner 3.1.0.1141
INFO: Java 1.8.0_202 Oracle Corporation (64-bit)
INFO: Linux 4.15.0-1045-aws amd64
INFO: User cache: /root/.sonar/cache
INFO: SonarQube server 7.9.1
INFO: Default locale: "en_US", source code encoding: "UTF-8"
WARN: SonarScanner will require Java 11+ to run starting in SonarQube 8.x
INFO: Load global settings
INFO: Load global settings (done) | time=173ms
INFO: Server id: BF41A1F2-AW4ZVHr_lBig5s92hKuf
INFO: User cache: /root/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=107ms
INFO: Load/download plugins (done) | time=79244ms
INFO: Process project properties
INFO: Execute project builders
INFO: Execute project builders (done) | time=2ms
INFO: Project key: buddy
INFO: Base dir: /buddy/sonar
INFO: Working dir: /buddy/sonar/.scannerwork
INFO: Load project settings for component key: 'buddy'
INFO: Load project settings for component key: 'buddy' (done) | time=125ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=186ms
INFO: Load active rules
INFO: Load active rules (done) | time=4885ms
INFO: Indexing files...
INFO: Project configuration:
INFO: Excluded sources: **/*.test.ts, **/*.spec.js
INFO: Included tests: **/*.spec.js
INFO: Load project repositories
INFO: Load project repositories (done) | time=100ms
INFO: 2 files indexed
INFO: 0 files ignored because of inclusion/exclusion patterns
INFO: 0 files ignored because of scm ignore settings
INFO: Quality profile for js: Sonar way
INFO: ------------- Run sensors on module Buddy
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=149ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=2ms
INFO: Sensor SonarJS [javascript]
INFO: 2 source files to be analyzed
INFO: Sensor SonarJS [javascript] (done) | time=141ms
INFO: Sensor ESLint-based SonarJS [javascript]
INFO: 2/2 source files have been analyzed
INFO: 2 source files to be analyzed
INFO: Sensor ESLint-based SonarJS [javascript] (done) | time=5539ms
INFO: Sensor SonarJS Coverage [javascript]
INFO: 2/2 source files have been analyzed
INFO: Analysing [/buddy/sonar/coverage/lcov.info]
INFO: Sensor SonarJS Coverage [javascript] (done) | time=9ms
INFO: Sensor JavaXmlSensor [java]
INFO: Sensor JavaXmlSensor [java] (done) | time=0ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=8ms
INFO: ------------- Run sensors on project
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=6ms
INFO: SCM provider for this project is: git
INFO: 2 files to be analyzed
WARN: Shallow clone detected, no blame information will be provided. You can convert to non-shallow with 'git fetch --unshallow'.
INFO: 0/2 files analyzed
WARN: Missing blame information for the following files:
WARN: * src/index.js
WARN: * src/index.test.js
WARN: This may lead to missing/broken features in SonarQube
INFO: 2 files had no CPD blocks
INFO: Calculating CPD for 0 files
INFO: CPD calculation finished
INFO: Analysis report generated in 42ms, dir size=73 KB
INFO: Analysis report compressed in 6ms, zip size=12 KB
INFO: Analysis report uploaded in 240ms
INFO: ANALYSIS SUCCESSFUL, you can browse https://yyyy.ngrok.io/dashboard?id=buddy
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at https://yyyy.ngrok.io/api/ce/task?id=zzz
INFO: Analysis total time: 13.147 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 1:48.966s
INFO: Final Memory: 14M/104M
INFO: ------------------------------------------------------------------------
Copy
Bash
Checking coverage
If the pipeline has finished successfully, you can open http://localhost:9000 and then fire up your project. You will see that the coverage has been properly collected:
Sonarqube overview page
You can drill down to src/index.js stats to see which lines were covered:
Sonarqube coverage page
Thanks for reading and good luck with setting up the pipeline!
Bonus: YAML configuration
If you prefer, you can flick the YAML switch and commit the following as buddy.yml file to avoid setting everything up in GUI:
- pipeline: "Sonar"
trigger_mode: "ON_EVERY_PUSH"
ref_name: "master"
ref_type: "BRANCH"
clone_depth: 1
trigger_condition: "ALWAYS"
actions:
- action: "Execute: npm run coverage"
type: "BUILD"
working_directory: "/buddy/sonar"
docker_image_name: "ringcentral/web-tools"
docker_image_tag: "alpine"
execute_commands:
- "npm install"
- "npm test"
- "npm run coverage -- -Dsonar.login=$SONAR_LOGIN -Dsonar.host.url=$SONAR_HOST_URL -Dsonar.links.homepage=$SONAR_LINKS_HOMEPAGE -Dsonar.links.ci=$SONAR_LINKS_CI -Dsonar.links.scm=$SONAR_LINKS_SCM"
cached_dirs:
- "/buddy/sonar/.scannerwork"
- "/root/.sonar/cache"
mount_filesystem_path: "/buddy/sonar"
shell: "BASH"
trigger_condition: "ALWAYS"
Copy
YAML
To learn more about configuration-as-code in Buddy, check out the section on YAML configuration