How can I set a byte string from an environment variable in Python?
Say you have a byte string generated with os.urandom(24)
,
b'\x1b\xba\x94(\xae\xd0\xb2\xa6\xf2f\xf6\x1fI\xed\xbao$\xc6D\x08\xba\x81\x96v'
and you want to store this in an environment variable,
export FOO='\x1b\xba\x94(\xae\xd0\xb2\xa6\xf2f\xf6\x1fI\xed\xbao$\xc6D\x08\xba\x81\x96v'
and extract the value from the Python program with os.environ
.
foo = os.environ['FOO']
The problem is that it foo
has a string literal value here '\\x1b\\xba\\x94...
instead of a sequence of bytes b'\x1b\xba\x94...
.
What is the correct value export
to use, or facility os.environ
to treat foo
as byte strings?
source to share
You can "unescape" your bytes in Python with:
import os
import sys
if sys.version_info[0] < 3: # sadly, it done differently in Python 2.x vs 3.x
foo = os.environ["FOO"].decode('string_escape') # since already in bytes...
else:
foo = bytes(os.environ["FOO"], "utf-8").decode('unicode_escape')
source to share
One more parameter, just set it as binary data in Bash. This is using ANSI string quoting :
export FOO=$'\x1b\xba\x94(\xae\xd0\xb2\xa6\xf2f\xf6\x1fI\xed\xbao$\xc6D\x08\xba\x81\x96v'
source to share
With zwer's answer, I tried the following
first from bash (this is the same binary literal given by ybakos)
export FOO='\x1b\xba\x94(\xae\xd0\xb2\xa6\xf2f\xf6\x1fI\xed\xbao$\xc6D\x08\xba\x81\x96v'
then i run python shell (i have python 3.5.2)
>>> import os
>>> # ybakos original binary literal
>>> foo = b'\x1b\xba\x94(\xae\xd0\xb2\xa6\xf2f\xf6\x1fI\xed\xbao$\xc6D\x08\xba\x81\x96v'
>>> # ewer python 3.x solution
>>> FOO = bytes(os.environ["FOO"], "utf-8").decode('unicode_escape')
>>> foo == FOO
False
>>> ^D
The last line foo == FOO should return true, so the solution doesn't work correctly.
I noticed there is an os.envirnb dictionary , but I couldn't figure out how to set the environment variable to a binary literal, so I tried the following alternative which uses base64 encoding to get the ASCII version of the binary literal.
First launch of python shell
>>> import os
>>> import base64
>>> foo = os.urandom(24)
>>> foo
b'{\xd9q\x90\x8b\xba\xecv\xb3\xcb\x1e<\xd7\xba\xf1\xb4\x99\xf056\x90U\x16\xae'
>>> foo_base64 = base64.b64encode(foo)
>>> foo_base64
b'e9lxkIu67Hazyx4817rxtJnwNTaQVRau'
>>> ^D
Then in bash shell
export FOO_BASE64='e9lxkIu67Hazyx4817rxtJnwNTaQVRau'
Then back to the python shell
>>> import os
>>> import base64
>>> # the original binary value from the first python shell session
>>> foo = b'{\xd9q\x90\x8b\xba\xecv\xb3\xcb\x1e<\xd7\xba\xf1\xb4\x99\xf056\x90U\x16\xae'
>>> dec_foo = base64.b64decode(bytes(os.environ.get('FOO_BASE64'), "utf-8"))
>>> # the values match!
>>> foo == dec_foo
True
>>> ^D
The last line shows that the 2 results are the same !!
What we do is first get the binary value from os.urandom () and Base64 encoding it. We then use the Base64 encoded value to set the environment variable. Note: base64.b64encode () returns a binary value, but it will only contain printable ASCII characters.
Then in our program, we read the Base64 encoded string value from the environment variable, convert the string to its binary form, and finally Base64 decode it back to its original value.
source to share