Differences in sys.path when python2.7 is called normally or via subprocess

Question

Why does it exist python2.7

when called using a subprocess via python3

not having the same sys.path as python2.7

which is called normally? In particular, python2.7

the subprocess does not have a "/path/to/site-packages/"

directory in sys.path.

Context

I would like to use fabric

to deploy a Django application that I am writing. My problem is that I wrote the app in python3

, but fabric

doesn't have explicit python3

support yet. My workaround, while fabric

not fully compatible with yet python3

, is to call the fab

script with subprocess

.

For some reason, when I call python2.7

using subprocess

via python3

, I don't have access to any modules in site-packages

.

python2.7 checks

I have python2.7

u fabric==1.10.0

installed via Enthought.

$ which python
/Users/.../Library/Enthought/Canopy_32bit/User/bin/python

$ python --version
Python 2.7.6 --  32-bit

$ which fab
/Users/.../Library/Enthought/Canopy_32bit/User/bin/fab

$ fab --version
Fabric 1.10.0
Paramiko 1.15.1

      

        
        
        
      

    

checks subprocess

I have no problem calling fab

from python2.7

using a subprocess.

$ python
Enthought Canopy Python 2.7.6 | 32-bit | (default, Apr 11 2014, 12:06:39)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.check_output('fab --version', shell=True)
'Fabric 1.10.0\nParamiko 1.15.1\n'

      

        
        
        
      

    

I also have no problem calling python2.7

from python3

using a subprocess.

$ python3
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 00:54:21)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.check_output('which python', shell=True)
b'/Users/.../Library/Enthought/Canopy_32bit/User/bin/python\n'
>>> subprocess.check_output('python --version', shell=True)
Python 2.7.6 --  32-bit
b''

      

        
        
        
      

    

DistributionNotFound: Fabric == 1.10.0

However, while my subprocess python2.7

can "find" the fab script, I cannot call it.

# python3
>>> subprocess.check_output(['which', 'fab'])
b'/Users/.../Library/Enthought/Canopy_32bit/User/bin/fab\n'
>>> subprocess.check_output(['fab', '--version'])
Traceback (most recent call last):
  File "/Users/.../Library/Enthought/Canopy_32bit/User/bin/fab", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/Applications/Canopy.app/appdata/canopy-1.4.0.1938.macosx-x86/Canopy.app/Contents/lib/python2.7/site-packages/pkg_resources.py", line 2877, in <module>
    working_set.require(__requires__)
  File "/Applications/Canopy.app/appdata/canopy-1.4.0.1938.macosx-x86/Canopy.app/Contents/lib/python2.7/site-packages/pkg_resources.py", line 698, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/Applications/Canopy.app/appdata/canopy-1.4.0.1938.macosx-x86/Canopy.app/Contents/lib/python2.7/site-packages/pkg_resources.py", line 596, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Fabric==1.10.0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['fab', '--version']' returned non-zero exit status 1

      

        
        
        
      

    

site-packages not in sys.path

It looks like python2.7

when called using a subprocess through python3

does not have the same sys.path as that python2.7

called normally.

As expected, sys.path did not have an Enthoughtt directory "site-packages"

that contains the module fabric

.

# python3
>>> subprocess.check_output('python -c "import sys; print sys.path"', shell=True)
## does not contain '/path/to/Enthought/python2.7/site-packages'

      

        
        
        
      

    

Manually add site packages to sys.path

To confirm that this is possible: when I manually add the correct one "site-packages"

, I can import successfully fabric

.

# python3
>>> subprocess.check_output('python -c\
    "import sys; sys.path.append(\'/path/to/Enthought/site-packages\');\
    from fabric import version; print version.get_version()"',\
    shell = True)
b'1.10.0\n'

      

        
        
        
      

    

Other options?

There should be a better way to ensure that python2.7, when called through a subprocess from python3, has the same sys.path as it python2.7

is called normally. Can anyone more familiar with subprocess weigh in?

Additional thoughts

It is very interesting what python2.7

could spawn another one python2.7

via a subprocess and that the subprocess has the correct site-packages directory in sys.path.

$ python
>>> import subprocess
>>> subprocess.check_output('python -c "import sys; print sys.path"', shell=True)
## contains "/path/to/Enthought/python2.7/site-packages"

      

        
        
        
      

    

I also compared sys.path

python3, python3 with python3 sub-process, and python3, python2.7 sub-processor, and was a little surprised to find that all three resulted in the same sys.path

.