Inevitable - iterate over several hashes in one game
I have a data structure in a json file that is loaded into an additional variables file and looks something like this:
"files_from_ftp":
{
"file1":
{
"full_name":"very-long-file-name-1"
"url":"ftp://.....very-long-file-name-1.zip"
},
"file2":
{
"full_name":"very-long-file-name-2"
"url":"ftp://.....very-long-file-name-2.zip"
}
},
"files_from_network":
{
"file3":
{
"full_name":"very-long-file-name-3"
"network_path":"file:///SOMEWHERE/ON/THE/NETWORK/very-long-file-name.zip"
}
},
"files_from_s3":
{
"file4":
{
"full_name":"very-long-file-name-4"
"url":"s3://some.s3.bucket/very-long-file-name-4.tar"
}
}
These three sections list files that are downloaded from different sources.
They come from different sources and some of them are known to be different types (tar files come from s3, zip from local network and ftp server), so I have three different pieces to handle each download. But at the end of every game, there is a file whose name is the value full_name
(all entries in these hashes will have an item named full_name
) in the directory /home/ubuntu/downloaded_files
.
Once loaded, they are all treated the same way. Right now I have three separate pieces that do the same thing but repeat over a different hash, like this:
- name: process files from ftp
shell: some_command -f /home/ubuntu/downloaded_files/{{ item.value.full_name }}
with_dict: files_from_ftp | default({})
- name: process files from network
shell: some_command -f /home/ubuntu/downloaded_files/{{ item.value.full_name }}
with_dict: files_from_network | default({})
- name: process files from s3
shell: some_command -f /home/ubuntu/downloaded_files/{{ item.value.full_name }}
with_dict: files_from_s3 | default({})
I would like to have one game that handles all these hashes, but I'm not sure if this is possible. I cannot understand the syntax. I'm thinking of hypothetical (and probably syntactically invalid) code:
- name: process all downloaded files
shell: some_command -f /home/ubuntu/downloaded_files/{{ item.value.full_name }}
with_dict: ( files_from_s3 | default({}) )
and ( files_from_ftp | default({}) )
and ( files_from_network | default({}) )
In this code example, I imagine I with_dict
will iterate over the concatenation of three other hashes.
source to share
I personally think you should go from using a file hash to a list of hashes. However:
In Ansible 2.0, you can do hash_merge
this:
- shell: echo "{{ item.value.full_name }}"
with_dict: files_from_s3|hash_merge(files_from_ftp)|hash_merge(files_from_network)
Which you can already customize by adding this hash_merge filter filter
source to share