Extract JSON value for shell variable using jq
I have a json response as shown below:
[
{"id":10,
"list_file":["/var/a.txt",
"/dev/b.txt"]}
]
I need to extract the values of list_file and store the shell variable as an array. I tried to do this by wading through and reading in the values.
#!/bin/bash
x=()
while read -r value
do
#echo "$value"
x+=("$value")
done < <(jq -r '.[] | .list_file' input.json)
But the extracted values in the array also contain quotes, parentheses, and a comma.
[
"/var/a.txt",
"/dev/b.txt"
]
Can you please help me change the code so that the array contains only /var/a.txt and / dev / b.txt entries. Also, I tried readarray and map, but they won't work on Mac Osx. Any help would be really appreciated.
To handle all values correctly, you need to use the command declare
to include the output jq
in the array assignment.
Some corner cases to worry about can be entered here: space, newline, and globe.
$ cat input.json
[
{"id":10,
"list_file":[
"/var/a b.txt",
"/dev/c\nd.txt",
"*"]}
]
A jq
command that extracts and prints the correctly quoted lines for use by the shell:
$ $ jq -r '.[] | .list_file[] | @sh' input.json
'/var/a b.txt'
'/dev/c
d.txt'
'*'
And a shell command that can use the output:
$ declare -a "x=($(jq -r '.[] | .list_file[] | @sh' input.json))"
Proof of correct operation:
$ printf '==%s==\n' "${x[@]}"
==/var/a b.txt==
==/dev/c
d.txt==
==*==
You can use @tsv
string format combined with / and split the output into an array in tabs: --raw-output
-r
bash
$ IFS=$'\t'
$ x=($(jq -r '.[] | .list_file | @tsv' input.json))
$ for xx in "${x[@]}"; do echo "$xx"; done
/var/a.txt
/dev/b.txt
For -r
will strip the outer quotes (useful for making filters jq
talk to non-JSON systems) and @tsv
outputs the array as a sequence of tab-separated strings (tabs, newlines, etc.).
Alternatively, you use a filter @sh
that outputs the array as a sequence of whitespace-separated strings. However, to interpret such output, you must eval
uate it:
$ eval "x=($(jq -r '.[] | .list_file | @sh' input.json))"
-
On a Mac it is easy enough to install bash with
readarray
(for example, using homebrew :)brew install bash
, so in the next one I will assume it is available, but you can just as well use the methodwhile read -r value
to read the values in stages. (One of the benefits of these line-oriented methods is that you don't have to mess with IFS.) -
For a given problem, there might be situations where it would be better to keep the JSON representation of special characters, such as "\ n" for NEWLINE, "\ u0000" for NUL, and so on. At any rate, a common alternative to use
-r
is to use the command line option-c
as shown below:
$ readarray -t x < <(jq -n -c '("\u0001\nb", "c\td", "e\u0000f")'
| sed -e 's/^\"//' -e 's/\"$//' )
$ printf ":%s:\n" "${x[@]}"
:\u0001\nb:
:c\td:
:e\u0000f: