GitLab CI enable SCP
I am currently using one of the shared contributors at GitLab.com. Is it possible to install the .gitlab-ci.yaml file so that the build can transfer SCP files from the remote server to the runner? My target is SCP files, which are required dependencies for my build, but they are not tracked in any Git repos.
I have marked the line where I would like to transfer, but I do not know how to express it correctly.
Note. CodeA has dependencies in CodeB and CodeC that need to be built before CodeA can compile, so I need to have access to CodeB and CodeC in order to build them on the ubuntu image first.
image: ubuntu:12.04
before_script:
build_CodeC:
stage: build
allow_failure: true
script:
-->- scp user@remoteServer:/home/user/file.tar . <---
- sh ./continuous_integration/build_CodeC_dependency.sh
build_CodeB:
stage: build
script:
- sh ./continuous_integration/build_CodeB_dependency.sh
build_CodeA:
stage: build
script:
- sh ./continuous_integration/build_CodeA.sh
source to share
From your question here , I think fetching your dependencies via http is not possible, so here's what you need to do in order to use scp
:
- Generating a key pair
- Copy the private key to the gitlab CI variable (call it
SSH_PRIVATE_KEY
) - Copy the public key to the server. gitlab will connect and add it to your
~/.ssh/authorized_keys
file - Tell your CI pipeline to use the private key that is stored in the Gitlab CI variable
To take this final step, simply add the following .gitlab-ci.yml
in the script or before_script section of the task of interest:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
You can also specify that CodeA depends on B and C. For this to work, build_CodeB and build_CodeC must be at a different stage than build_CodeA.
Also, you need a way to migrate the built files from the build_CodeB and build_CodeC jobs to the build_CodeA job. One way to do this is using artifacts .
In the end, your file .gitlab-ci.yml
should look something like this:
image: ubuntu:12.04
stages:
- deps
- build
build_CodeC:
stage: deps
allow_failure: true
script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
- scp user@remoteServer:/home/user/file.tar .
- sh ./continuous_integration/build_CodeC_dependency.sh
artifacts:
paths:
- path_to_built_codeC
build_CodeB:
stage: deps
script:
- sh ./continuous_integration/build_CodeB_dependency.sh
artifacts:
paths:
- path_to_built_codeB
build_CodeA:
stage: build
dependencies:
- build_CodeB
- build_CodeC
script:
- sh ./continuous_integration/build_CodeA.sh
I only set the SSH key installation part in build_CodeC because that's where you are using scp
. You will need to copy this to any job that requires use scp
. I think you might need to do this in build_codeB as your tar file will not be carried over to the build_CodeB job.
source to share