Why is "<<<" filtering null bytes in gdb where "<()" is not?
I recently discovered that the trick <<<
in gdb
order to feed the stdin
program being debugged is filtering null bytes from the stream.
Here's a small example (which everyone should be able to reproduce at home) to prove that it filters out null bytes:
$> python -c 'print("A\x00" * 10)' | cat -A A^@A^@A^@A^@A^@A^@A^@A^@A^@A^@$ $> gdb /bin/cat ... gdb license prelude ... snip... (gdb) r -A <<< $(python -c 'print("A\x00" * 10)') Starting program: /bin/cat -A <<< $(python -c 'print("A\x00" * 10)') /bin/bash: warning: command substitution: ignored null byte in input AAAAAAAAAA$ [Inferior 1 (process 3798) exited normally]
Where, using bash-specific <()
process substitution , leave null bytes available to the program stdin
internally gdb
:
(gdb) r -A < <(python -c 'print("A\x00" * 10)') Starting program: /bin/cat -A < <(python -c 'print("A\x00" * 10)') A^@A^@A^@A^@A^@A^@A^@A^@A^@A^@$ [Inferior 1 (process 3804) exited normally]
So, I always thought that <<<
, and <()
do the same thing, now obviously wrong. I would like to know what are the differences between these two methods and explain the explanation for the bash
cryptic error message:
/bin/bash: warning: command substitution: ignored null byte in input
Any help is more than welcome!
source share
So, I always though
<<<
and<()
did the same thing, that now is clearly incorrect.
The same ones do nothing in common. The operator <<<
redirects the string "here" to the appropriate input stream of the process. Process substitution via <()
expands to a file name (usually FIFO or similar) from which the standard output of the given command can be read.
What you mean is that a combination of <<<
command substitution (via $()
or backticks) does about the same thing as a combination of regular standard input redirection via <
and process substitution. This is true, but as you discovered, the semantics are not entirely equivalent.
I would like to know what are the differences between these two methods and there is an explanation about bash mysterious error message
The key difference is that redirecting from a string here requires first creating a string as a value stored in the shell (which you do with command substitution), whereas redirecting process substitution involves redirected output that is read directly by the associated process.
Ultimately, the diagnostics you receive indicate that the unexpected behavior you are experiencing is due to command substitution behavior, not behavior <<<
. While I did not find it explicitly documented, I am not at all surprised that bash strips null characters from program output when processing command substitution, as I expect its internal representation of shell strings to be C strings. C strings are null terminated and therefore can represent character sequences that include a null character.
Update:
Note also that, as @sorontar pointed out in the comments to another answer, POSIX says that if the output of a command in command substitution contains empty bytes, then no result is specified. bash can thus strip null bytes - or really do more or less of what it wants when it sees them - without sacrificing POSIX compliance. Other shells can do different things in this regard than bash. This is a great reason to avoid command substitutions where a null byte appearing in the command output is visible.
source share
As mentioned, these two methods are not the same (although in some cases they can be used for the same reason)
This one <<<
is the line here and is subject to the extension extension rules. Bash does not allow null bytes on variable values.
Process substitution, on the other hand, is <()
treated as a file, and null characters are allowed in files.
Thus, the differences you noticed are related to this Bash behavior. With a different shell, this limitation may not be valid.
Additional tests:
$ echo -en "A\x00A\x00A" |od -t x1c 0000000 41 00 41 00 41 A \0 A \0 A 0000005 $ a=$(echo -en "A\x00A\x00A");echo "$a" |od -t x1c bash: warning: command substitution: ignored null byte in input 0000000 41 41 41 0a A A A \n 0000004 $ cat <(echo -en "A\x00A\x00A") |od -t x1c #this is treated as file 0000000 41 00 41 00 41 A \0 A \0 A 0000005 $ cat <<<$(echo -en "A\x00A\x00A") |od -t x1c #this is considered a variable bash: warning: command substitution: ignored null byte in input 0000000 41 41 41 0a A A A \n 0000004
source share