In setup.py involving Cython, if install_requires, then how from library can import something?
It doesn't make sense to me. How can I use setup.py to install Cython and then also use setup.py to compile the library proxy?
import sys, imp, os, glob
from setuptools import setup
from Cython.Build import cythonize # this isn't installed yet
setup(
name='mylib',
version='1.0',
package_dir={'mylib': 'mylib', 'mylib.tests': 'tests'},
packages=['mylib', 'mylib.tests'],
ext_modules = cythonize("mylib_proxy.pyx"), #how can we call cythonize here?
install_requires=['cython'],
test_suite='tests',
)
Hereafter: python setup.py build
Traceback (most recent call last):
File "setup.py", line 3, in <module>
from Cython.Build import cythonize
ImportError: No module named Cython.Build
This is because cython is not installed yet.
Oddly enough, so much has been written. A quick github search shows so much: https://github.com/search?utf8=%E2%9C%93&q=install_requires+cython&type=Code
source to share
One solution is to not require Cython to be built and instead distribute the Cython generated files C
with your package. I'm sure there is a simpler example somewhere, but this is what it does pandas
- it imports Cython conditionally, and if not, then it can be created from c files.
https://github.com/pandas-dev/pandas/blob/3ff845b4e81d4dde403c29908f5a9bbfe4a87788/setup.py#L433
Edit: The doc link from @danny has a simpler example. http://docs.cython.org/en/latest/src/reference/compilation.html#distributing-cython-modules
source to share
All in all, it doesn't make sense. This is, as you suspect, an attempt to use something that (possibly) has yet to be installed. If you are testing a system that already has a dependency installed, you may not notice this defect. But run it on a system where your dependency is missing and you will definitely notice.
There is one argument keyword setup()
, setup_requires
which may be used in parallel to appear in the form of install_requires
, but this is an illusion. While it install_requires
runs the beautiful ballet of unattended installations in environments that lack dependency names, there is setup_requires
more documentation than automation. It won't automatically install, and it certainly won't magically return in time to automatically install modules that have already been called in instructions import
.
There's more on the setuptools file , but the quick answer is that you are right to confuse a module trying to automatically set its own setup prerequisites.
For a practical workaround, try installing cython
separately, and then run this setup. While this will not remove the metaphysical illusions of this script setup, it will solve the requirements and let you move on.
source to share
As far as I understand, this is where PEP 518 comes in - see also some clarification from one of its authors.
The idea is that you add another file in your project / Python package: pyproject.toml
. It is intended to contain information about the dependencies of the build environment (among other things, in the long run). pip
(or any other package manager) can look into this file and set the required build environment before running setup.py (or any other build script). So it pyproject.toml
might look like this:
[build-system]
requires = ["setuptools", "wheel", "Cython"]
This is a fairly recent development and as of now (January 2019) it is not finalized / endorsed by the Python community, although (limited) support was added to pip in May 2017 / 10.0 release.
source to share
When you use setuptool you must add cython
to setup_requires
(and also install_requires
if cython is used during installation), i.e.
# don't import cython, it isn't yet there
from setuptools import setup, Extension
# use Extension, rather than cythonize (it is not yet available)
cy_extension = Extension(name="mylib_proxy", sources=["mylib_proxy.pyx"])
setup(
name='mylib',
...
ext_modules = [cy_extension],
setup_requires=["cython"],
...
)
Cython
not imported (it is not yet available when it setup.py
starts), but setuptools.Extension
used instead to cythonize
add the Cython extension to install.
This should work now. Reason: will try to import cython after : setuptools
setup_requires
...
try:
# Attempt to use Cython for building extensions, if available
from Cython.Distutils.build_ext import build_ext as _build_ext
# Additionally, assert that the compiler module will load
# also. Ref #1229.
__import__('Cython.Compiler.Main')
except ImportError:
_build_ext = _du_build_ext
...
This gets more complicated if your Cython extension uses numpy
, but it is possible - see this post .
source to share