Jenkins workflow templates

Every jenkins pipeline does pretty much the same thing - at least in a small team with multiple projects.

Build (from the same source code repo) -> run tests -> publish artifacts (also artifact repo)

We create many new projects and they all have a very similar life cycle. Is it possible to create a template pipeline from which I can create specific pipleines and make the necessary changes to the jobs?


source to share

2 answers

There are several approaches that I use for myself and my team well.

Part 1) is to determine which orchestration plugins are best for Jenkins. The plugins and approaches that worked well for me were:

a) Use It abstracts the definitions of jobs and threads using a higher level library. This allows you to define jobs in YAML, which is pretty simple and supports most common use cases (jobs, templates, threads). These yaml files can then be used by the jenkins-jobs-builder python cli utility with an orchestration tool like indispensable puppet chef. You can use YAML anchors to replace blocks that are common across multiple jobs, or ever templates them from the templating engine (erb, jinja2)

b) Use the workflow plugin, The workflow plugin allows you to have a single workflow in groovy, not a set of jobs that are chained together.

"For example, to check out and build multiple repos in parallel, each on its own slave:

parallel repos.collectEntries {repo -> [/* thread label */repo, {
    node {
        dir('sources') { // switch to subdir
            git url: "${repo}"
            sh 'make all -Dtarget=../build'



If you create these workflow definitions from a templating engine (ERB, jinja2) and combine them using a configuration management tool (again, chef, puppet). It becomes much easier to make small and large changes that affect one or all tasks. For example, you can use a template that some jenkins boxes collect, publish and deploy artifacts to a development environment, while others just deploy artifacts to a QA environment. All of this can be achieved from the same template using if and then statements and macros in jinja2 / erb.

Ex (abstraction):

if ($environment == dev=) then compile, publish, deploy($environment)
elif ($environment== qa) then deploy($environment)


part2) is to make sure all jenkins config for all jobs and threads is stored in source control and make sure that job definition change in source control is automatically pushed to jenkins server (again) unmarried, puppet, chef) ... Or even have a jenkins job that monitors its own job definition repo and updates itself automatically

When you reach # 1 and # 2, you should be in a position where you can, with some confidence, allow all members of your team to make changes to their jobs / projects, giving you information on who changed what and when, and be able to easily roll changes from change control when things go wrong.

its pretty much about getting jenkins to deploy code from a series of templated jobs that were themselves defined in the code.



Another approach we followed was to manage jobs using Ansible templates. We started out before the jenkins_job module became available and uses the url module to communicate with jenkins, but the general approach will be the same:

  • j2 templates created for different tasks
  • loop covers project descriptions and updates jobs and views in jenkins
  • the default is a generic definition and a minimal description is required:

          template: build.xml.j2
        Release: ...
          repository: git://../..
          a: b
          c: d
            template: custom.j2



All Articles