Jenkins declarative pipeline, run groovy script on slave agent
I have a Jenkins declarative pipeline that I used in Jenkins craft and it works great. However, now that I have switched to doing this on a slave node, groovy scripts called in the pipeline cannot access files in the workspace.
My jenkinsfile looks like this ...
pipeline {
agent {
label {
label "windows"
customWorkspace "WS-${env.BRANCH_NAME}"
}
}
stages {
stage('InitialSetup') {
steps {
"${env.WORKSPACE}/JenkinsScripts/myScript.groovy"
}
}
}
I can see on the slave that it is creating a workspace by checking out from git and executing the script correctly. However, if something in the script tries to interact with files in the workspace, it fails.
If I have something simple ...
def updateFile(String filename) {
echo env.NODE_NAME
filename = "${env.WORKSPACE}/path/to/file"
def myFile = new File(filename)
<do other things with the file>
}
... it says it cannot find the file specified. It gives me the path it is looking for and I can confirm that the file exists and that the code only works by building on the master.
Why can't the script find files this way when it can only run on the main node? I added the command "echo env.NODE_NAME" to my groovy file and it says the script is being executed on the correct node.
Thank.
source to share
It turns out that Groovy File commands are considered unsafe, and while they will run on the master, they won't run on the slave. if you call them from a script where the agent is configured on a different node, it will still execute the command just fine, only on the master node and not on the agent. Here is an excerpt from the article https://support.cloudbees.com/hc/en-us/articles/230922508-Pipeline-Files-manipulation
The operation with the File class is done on master, so it only works if build is done on master, in this example I create a file and check if I can access it on a node with a method, it doesn’t exist because it new File(file)
is done on master so that to check it out, I am looking for a folder Users
that exists in my master but is not present in the node.
stage 'file move wrong way'
//it only works on master
node('slave') {
def ws = pwd()
def context = ws + "/testArtifact"
def file = ws + '/file'
sh 'touch ' + file
sh 'ls ' + ws
echo 'File on node : ' + new File(file).exists()
echo 'Users : ' + new File('/Users').exists()
sh 'mv ' + file + ' ' + context
sh 'ls ' + ws
}
We recommend using your own commands to execute the file manipulation command.
This is a simple example of shell operations
stage 'Create file'
sh 'touch test.txt'
stage 'download file'
def out='$(pwd)/download/maven.tgz'
sh 'mkdir -p ./download'
sh 'curl -L http://ftp.cixug.es/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -o ' + out
stage 'move/rename'
def newName = 'mvn.tgz'
sh 'mkdir -p $(pwd)/other'
sh 'mv ' + out + ' ' + newName
sh 'cp ' + newName + ' ' + out
}
source to share
I've implemented code that automatically sets Groovy to slave (for pipeline scripting). Perhaps this solution is a bit cumbersome, but pipelines do not offer any other way to achieve the same functionality as in "Executing Groovy Script" from old Jenkins, because the plugin https://wiki.jenkins.io/display/JENKINS/Groovy+ plugin is not yet supported for the pipeline.
import hudson.tools.InstallSourceProperty;
import hudson.tools.ToolProperty;
import hudson.tools.ToolPropertyDescriptor;
import hudson.tools.ToolDescriptor;
import hudson.tools.ToolInstallation;
import hudson.tools.ToolInstaller;
import hudson.util.DescribableList;
import hudson.plugins.groovy.GroovyInstaller;
import hudson.plugins.groovy.GroovyInstallation;
/*
Installs Groovy on the node.
The idea was taken from: https://devops.lv/2016/12/05/jenkins-groovy-auto-installer/
and https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/configMavenAutoInstaller.groovy
COMMENT 1: If we use this code directly (not as a separate method) then we get
java.io.NotSerializableException: hudson.plugins.groovy.GroovyInstaller
COMMENT 2: For some reason inst.getExecutable(channel) returns null. I use inst.forNode(node, null).getExecutable(channel) instead.
TODO: Check if https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.helpers.step.MultiJobStepContext.groovyCommand
works better.
*/
def installGroovyOnSlave(String version) {
if ((version == null) || (version == "")) {
version = "2.4.7" // some default should be
}
/* Set up properties for our new Groovy installation */
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def proplist = new DescribableList<ToolProperty<?>, ToolPropertyDescriptor>()
def installers = new ArrayList<GroovyInstaller>()
def autoInstaller = new GroovyInstaller(version)
installers.add(autoInstaller)
def InstallSourceProperty isp = new InstallSourceProperty(installers)
proplist.add(isp)
def inst = new GroovyInstallation("Groovy", "", proplist)
/* Download and install */
autoInstaller.performInstallation(inst, node, null)
/* Define and add our Groovy installation to Jenkins */
def descriptor = Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy")
descriptor.setInstallations(inst)
descriptor.save()
/* Output the current Groovy installation path, to verify that it is ready for use */
def groovyInstPath = getGroovyExecutable(version)
println("Groovy " + version + " is installed in the node " + node.getDisplayName())
}
/* Returns the groovy executable path on the current node
If version is specified tries to find the specified version of groovy,
otherwise returns the first groovy installation that was found.
*/
def getGroovyExecutable(String version=null) {
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def channel = node.getComputer().getChannel()
for (ToolInstallation tInstallation : Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy").getInstallations()) {
if (tInstallation instanceof GroovyInstallation) {
if ((version == null) || (version == "")) {
// any version is appropriate for us
return tInstallation.forNode(node, null).getExecutable(channel)
}
// otherwise check for version
for (ToolProperty prop in tInstallation.getProperties()) {
if (prop instanceof InstallSourceProperty) {
for (ToolInstaller tInstaller: prop.installers) {
if (
(tInstaller instanceof GroovyInstaller) &&
(tInstaller.id.equals(version))
)
return tInstallation.forNode(node, null).getExecutable(channel)
}
}
}
}
}
return null
}
/* Wrapper function. Returns the groovy executable path as getGroovyExecutable()
but additionally tries to install if the groovy installation was not found.
*/
def getGroovy(String version=null) {
def installedGroovy = getGroovyExecutable(version)
if (installedGroovy != null) {
return installedGroovy
} else {
installGroovyOnSlave(version)
}
return getGroovyExecutable(version)
}
Just put these 3 methods in your pipeline script and you can get the path to the Groovy executable using the getGroovy () method. If it is not already installed, then the installation will be done automatically. You can test this code with a simple pipeline like:
// Main
parallel(
'Unix' : {
node ('build-unix') {
sh(getGroovy() + ' --version')
processStages(arch_unix)
}
},
'Windows' : {
node ('build-win') {
bat(getGroovy() + ' --version')
processStages(arch_win)
}
}
)
For me, the output was:
[build-unix] Groovy Version: 2.4.7 JVM: 1.8.0_222 Vendor: Private Build OS: Linux
[build-win] Groovy Version: 2.4.7 JVM: 11.0.1 Vendor: Oracle Corporation OS: Windows 10
source to share