Jenkins workflow templates

There are several approaches that I use for myself and my team well.

Part 1) is to determine which orchestration plugins are best for Jenkins. The plugins and approaches that worked well for me were:

a) Use http://ci.openstack.org/jenkins-job-builder/ It abstracts the definitions of jobs and threads using a higher level library. This allows you to define jobs in YAML, which is pretty simple and supports most common use cases (jobs, templates, threads). These yaml files can then be used by the jenkins-jobs-builder python cli utility with an orchestration tool like indispensable puppet chef. You can use YAML anchors to replace blocks that are common across multiple jobs, or ever templates them from the templating engine (erb, jinja2)

b) Use the workflow plugin, https://github.com/jenkinsci/workflow-plugin The workflow plugin allows you to have a single workflow in groovy, not a set of jobs that are chained together.

"For example, to check out and build multiple repos in parallel, each on its own slave:

parallel repos.collectEntries {repo -> [/* thread label */repo, {
    node {
        dir('sources') { // switch to subdir
            git url: "https://github.com/user/${repo}"
            sh 'make all -Dtarget=../build'
        }
    }
}]}

      

        
        
        
      

    

"

If you create these workflow definitions from a templating engine (ERB, jinja2) and combine them using a configuration management tool (again, chef, puppet). It becomes much easier to make small and large changes that affect one or all tasks. For example, you can use a template that some jenkins boxes collect, publish and deploy artifacts to a development environment, while others just deploy artifacts to a QA environment. All of this can be achieved from the same template using if and then statements and macros in jinja2 / erb.

Ex (abstraction):

if ($environment == dev=) then compile, publish, deploy($environment)
elif ($environment== qa) then deploy($environment)

      

        
        
        
      

    

part2) is to make sure all jenkins config for all jobs and threads is stored in source control and make sure that job definition change in source control is automatically pushed to jenkins server (again) unmarried, puppet, chef) ... Or even have a jenkins job that monitors its own job definition repo and updates itself automatically

When you reach # 1 and # 2, you should be in a position where you can, with some confidence, allow all members of your team to make changes to their jobs / projects, giving you information on who changed what and when, and be able to easily roll changes from change control when things go wrong.

its pretty much about getting jenkins to deploy code from a series of templated jobs that were themselves defined in the code.