How to properly handle additional functions in python
I am working on python packages that implement scientific models and I am wondering how best to handle additional functionality. Here's the behavior I would like: If some optional dependencies cannot be imported (for example, a headless machine build module), I would like to disable features by using those modules in my classes, alert the user if he tries to use them, and that's it that doesn't break execution. so the script will work anyway:
mymodel.dostuff()
mymodel.plot() <= only plots if possible, else display log an error
mymodel.domorestuff() <= get executed regardless of the result of the previous statement
So far, the options I see are as follows:
- check the box
__init __.py
for available modules and save the list (but how to use it correctly in the rest of the package?) - for every function that relies on optional dependencies has an operator
try import ... except ...
- placement of functions depending on a specific module in a split file
These parameters should work, but they all seem rather hacky and difficult to maintain. what if we want to give up addiction entirely? or make it mandatory?
source to share
The simplest solution, of course, is to just import the optional dependencies in the body of the required function. But always-right PEP 8
says:
Imports are always placed at the top of the file, immediately after any module comments and docstrings, and also before module globals and constants.
Not wanting to go against the best wishes of the python masters, I take the following approach, which has several advantages ...
First import with try-except
Say one of my features foo
is required numpy
and I want to make it an optional dependency. At the top of the module, I put:
try:
import numpy as _numpy
except ImportError:
_has_numpy = False
else:
_has_numpy = True
Here (in the exclusion block) there will be a place to print the warning, preferably with a module warnings
.
Then highlight the exception in the function
What if the user calls foo
and doesn't have numpy? I am throwing an exception and documenting this behavior.
def foo(x):
"""Requires numpy."""
if not _has_numpy:
raise ImportError("numpy is required to do this.")
...
Alternatively, you can use a decorator and apply it to any function that requires this dependency:
@requires_numpy
def foo(x):
...
This prevents duplication of code.
And add it as an additional dependency to your install script
If you are redistributing code, see how to add an additional dependency to your config setting. For example, with help setuptools
I can write:
install_requires = ["networkx"],
extras_require = {
"numpy": ["numpy"],
"sklearn": ["scikit-learn"]}
This indicates something that is networkx
absolutely required during installation, but the additional functionality of my module requires numpy
and sklearn
that are optional.
Using this approach, here are the answers to your specific questions:
- What if we want to make a dependency mandatory?
We can simply add our additional dependency to our list of tools to configure the required dependencies. In the above example, we are moving numpy
to install_requires
. All code existence checking numpy
can be removed, but removing it will not break your program.
- What if we want to completely remove dependencies?
Just remove dependency checking in any function that previously required it. If you've done dependency checking with a decorator, you can just change it so that it just passes the original function unchanged.
This approach has the advantage of placing all imports at the top of the module so that I can see at a glance what is required and what is optional.
source to share
I would use mixin style to create the class. Keep additional behavior in separate classes, and subclass those classes in your main class. If you find that optional behavior is not possible, create a dummy mixin class instead. For example:
model.py
import numpy
import plotting
class Model(PrimaryBaseclass, plotting.Plotter):
def do_something(self):
...
plotting.py
from your_util_module import headless as _headless
__all__ = ["Plotter"]
if _headless:
import warnings
class Plotter:
def plot(self):
warnings.warn("Attempted to plot in a headless environment")
else:
class Plotter:
"""Expects an attribute called `data' when plotting."""
def plot(self):
...
Or, alternatively, use decorators to describe when a function might not be available.
eg.
class unavailable:
def __init__(self, *, when):
self.when = when
def __call__(self, func):
if self.when:
def dummy(self, *args, **kwargs):
warnings.warn("{} unavailable with current setup"
.format(func.__qualname__))
return dummy
else:
return func
class Model:
@unavailable(when=headless)
def plot(self):
...
source to share