Python YAML parameter (get value from another parameter)
I am trying to use a YAML config file in my Python script, one of the problems I have found is that I cannot access other attributes, so I need to duplicate a lot of content.
for example
root_path: /root
script_path: root_path + /scripts
This of course doesn't exist, but is there a way to achieve this? Because there is a lot of content that I cannot duplicate, because when someone changes, I need to change it everywhere ...
I also looked at creating my own connection function
root_path: &ROOT /root
script_path: !join [*ROOT, '/scripts']
def join(loader, node):
seq = loader.construct_sequence(node)
return ''.join([str(i) for i in seq])
yaml.add_constructor('!join', join)
But I have to set every time &VARNAME
, it would be nice to automatically set the parameter key as a reference object ...
source to share
Ok, I was working on a solution because nothing was what I wanted and everything was so hard, I wanted to die ...
These solutions convert string %something%
to value for something
.
It works great, this is an example
root_path: /root
script_path: "%root%/scripts"
With this method, it script_path
becomes /root/scripts
.
def replace_values(yaml_file):
def _get(dict, list):
return reduce(lambda d, k: d[k], list, dict)
def _replace(obj):
for k, v in obj.iteritems():
if isinstance(v, dict):
_replace(v)
if isinstance(v, str):
match = re.match(r'%(.*)%', v)
if match:
reference = match.group(1).split('.')
replace = _get(yaml_file, reference)
obj[k] = re.sub(r'%.*%', replace, v)
_replace(yaml_file)
return yaml_file
Usage is easy, just load the file in the Yaml
usual way and replace the call.
with open(config_file.get(options.env), 'r') as ymlfile:
config = yaml.load(ymlfile)
config = replace_values(config)
Then our new one will config
keep the replaced values, it won't overwrite the original file of .yml
course. I hope you find it useful, I am actually doing this because this is exactly what I need.
I used the percentage token %
, but you can change it to whatever you want and change the method to make it work with regex (some tokens are used by regex, so I used %
)
source to share
You cannot do this with YAML. For a config file, you can instead use ConfigParser , which allows you to interpolate values so that your example looks like this:
root_path: /root
script_path: %(root_path)s/scripts
source to share
I would reinforce the YAML parser a bit ¹:
import ruamel.yaml as yaml
from ruamel.yaml.comments import CommentedMap
yaml_str = """\
root_path: /root # use this key for expansion of values
script_path: root_path + /scripts
"""
def set_item(self, key, value):
split_val = value.split(' + ', 1)
if len(split_val) > 1:
alt_key = split_val[0].strip()
if alt_key in self.keys():
value = self.get(alt_key) + split_val[1]
self._org__setitem__(key, value)
CommentedMap._org__setitem__ = CommentedMap.__setitem__
CommentedMap.__setitem__ = set_item
data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader)
print yaml.dump(data, Dumper=yaml.RoundTripDumper)
will provide you with:
root_path: /root # use this key for expansion of values
script_path: /root/scripts
Note that both comment and key order are preserved using the RoundTripLoader / Dumper combination. data
works like regular Python dict
.
¹ This was done using ruamel.yaml , of which I am the author. It is a superset of PyYAML primarily for storing comments and other crawling information.
source to share