Continuous Integration: Releases Jenkins Pipeline

From Gcube Wiki
Revision as of 04:15, 4 September 2019 by Manuele.simi (Talk | contribs) (Jenkins Pipeline Configuration)

Jump to: navigation, search


Jenkins Pipeline is a combination of plugins that support the integration and implementation of continuous delivery pipelines using Jenkins. A pipeline has an extensible automation server for creating simple or complex delivery pipelines "as code," via pipeline DSL (Domain-specific Language).

gCubeRelease Pipeline Project

In gCube we use a Pipeline to trigger the builds of jobs forming a gCube Release. The pipeline project is available at: https://jenkins.d4science.org/job/gCube-Release/

Parameters

Jenkins pipeline params.png

Triggers

No triggers are defined because the pipeline is expected to be manually launched by the Release Manager:

Jenkins pipeline triggers.png

It can be changed according to the release needs and the availability of a sufficient number of dedicate agents in Jenkins.

Git

The pipeline definition is maintained in a Git repository. This section connects the project to the Git repository.

Jenkins pipeline git.png

Jenkins Pipeline Definition

Git Repository

The definition of the gCube release pipeline is maintained in this Git Repository: https://code-repo.d4science.org/gCubeCI/gCubeRelease

Requirements

  • Jenkins configured with a JDK named 'OpenJDK 8'
  • One or more Jenkins agents labeled as 'pipeline-agent'

Basic Structure

Jenkinsfile defines a Declarative Pipeline. The Pipeline’s code defines the entire build process of a gCube Release.

This is the stub of the gCubeRelease pipeline.

  
  // manage options and settings

  // load and parse the release file
  String releaseURL = "<git repo url>/releases/gcube-${gCube_release_version}.yaml"
  def text = releaseURL.toURL().getText()
  def jsonConfig = new Yaml().load(text)

  // pipeline
  pipeline {
        // run only on agents with the label
        agent { label 'pipeline-agent' }

        // expected input parameters
        parameters {
            string(name: 'myInput', description: 'Some pipeline parameters')
        }


        stages {
            stage('preliminary steps') {
               //prepare the environment for the builds
                steps {
                       //execute steps here if needed
                }
            }
        
            stage('build <group1 name> components') {
                steps {
                  withMaven(jdk: "${maven_jdk}", mavenLocalRepo: "${maven_local_repo_path}", mavenSettingsFilePath: "${maven_settings_file}") {
                    buildComponents items: jsonConfig.gCube_release.Components.<group1 name>.collect { "${it}" }
                  }
                  echo "Done with <group1 name> components"
               }
            }

            stage('build <group2 name> components') {
                steps {
                  withMaven(jdk: "${maven_jdk}", mavenLocalRepo: "${maven_local_repo_path}", mavenSettingsFilePath: "${maven_settings_file}") {
                    buildComponents items: jsonConfig.gCube_release.Components.<group2 name>.collect { "${it}" }
                  }
                  echo "Done with <group2 name> components"
               }
            }
        }
	// post-build actions
	post {
		always {
			echo 'This will always run'
		}
		success {
			echo 'This will run only if successful'
		}
		failure {
			echo 'This will run only if failed'
		}
		unstable {
			echo 'This will run only if the run was marked as unstable'
		}
		changed {
			echo 'This will run only if the state of the Pipeline has changed'
			echo 'For example, if the Pipeline was previously failing but is now successful'
		}
	}

    }

    def buildComponents(args) {
       parallel args.items.collectEntries { name -> [ "${name}": {
        if (!"NONE".equalsIgnoreCase(name))
            build name
        }]}
    }

Reference Documentation

Pipeline Syntax

Jenkins Pipeline Configuration

Each gCube release requires a configuration file (called release file) written in the YAML format.

The file must be placed in the /release folder of the Git repository and named as gcube-<version>.yaml (e.g. gcube-4.14.5.yaml). The file must report the gcube release version and the list of Jenkins jobs to build grouped in logical sets.

This is a sample release configuration with 3 groups (SmartGear, Enabling, Data):

gCube_release:
  Version: 4.14.5
  Components:
    SmartGear:
      - maven-parent
      - gcube-bom
      - maven-smartgears-bom
      - authorization-client
      - gxRest

    Enabling:
      - information-system-bom
      - information-system-model
      - resource-registry-api
      - resource-registry-client

    Data:
      - NONE

Do note that if a logical group is added, a correspondent stage must be added to the pipeline definition.

Executing the release pipeline with the previous release file results in something like the following on the Jenkins interface: JenkinsPipelineProgress.png

Back to the CI guide.