Gitlab CI: setting dynamic variables

I do not know where your expectation, but is trivial to verify there is no special meaning to $

, _

'/' and :

, if not followed by a space in the YAML. Maybe in gitlab, but I highly doubt it is as you expect.

To formalize your expectation, you assume that any key (from the same mapping) that is preceded $

and ends at the end of the scalar with _

or /

will be "expanded" to that key value. _

must be such a terminator, otherwise $PROJECT_$PACKAGE

it will not expand correctly.

Now let's look at adding a key-value pair:

 BREAKING_TEST: $PACKAGE_PATH

      

        
        
        
      

    

it is supposed to expand to:

 BREAKING_TEST: /opt/project/build/package/bundle

      

        
        
        
      

    

or follow the rule of being _

a terminator and simply expanding to:

 BREAKING_TEST: project_PATH

      

        
        
        
      

    

To prevent similar programs with ambiguity, such as bash

using quotes around variable names to be expanded ( "$PROJECT"_PATH

vs. $PROJECT_PATH

), but a smarter and more modern solution is to use clamping start and end characters (e.g., {

and }

, $%

and %

,) with some special rule to use the clamping symbol as plain text.

So, it won't work as you indicated that you are indeed doing something wrong.

There is no need to preprocess the YAML file, and this can be done in Python for example (but note that {

has a special meaning in YAML), possibly with jinja2: load the variables, then expand the source using the variables until the replacements will no longer be done.

But it all starts with a wise choice of separators. Also keep in mind that although your "variables" appear to be ordered in YAML text, there is no such guarantee when they are constructed as dict / hash / mapping in your program.

You can, for example, use <<

and >>

:

variables:
  PROJECT: project
  PACKAGE: package
  PROD: <<PROJECT>>_<<PACKAGE>>
  STAGE: <<PROD>>_stage
  PACKAGE_PATH: /opt/<<PROJECT>>/build/<<PACKAGE>>
  BUILD_PATH: /opt/<<PROJECT>>/build/<<PACKAGE>>/bundle
  CONTAINER_IMAGE: registry.example.com/<<PROJECT>>/<<PACKAGE>>:e2

      

        
        
        
      

    

which with the following program (which does not handle escaping <<

to keep it normal) generates your original, extended, YAML exactly.

import sys
from ruamel import yaml


def expand(s, d):
    max_recursion = 100
    while '<<' in s:
        res = ''
        max_recursion -= 1
        if max_recursion < 0:
            raise NotImplementedError('max recursion exceeded')
        for idx, chunk in enumerate(s.split('<<')):
            if idx == 0:
                res += chunk  # first chunk is before <<, just append
                continue
            try:
                var, rest = chunk.split('>>', 1)
            except ValueError:
                raise NotImplementedError('delimiters have to balance "{}"'.format(chunk))
            if var not in d:
                res += '<<' + chunk
            else:
                res += d[var] + rest
        s = res
    return s


with open('template.yaml') as fp:
    yaml_str = fp.read()
variables = yaml.safe_load(yaml_str)['variables']
data = yaml.round_trip_load(expand(yaml_str, variables))
yaml.round_trip_dump(data, sys.stdout, indent=2)