Clever Geek Handbook

Handling flexible function arguments in Python

TL; TR Look for idioms and patterns for unpacking positional and key arguments into an ordered sequence of positional arguments based on a simple specification like. list of names. The idea seems to be similar to analysis like scanf.

I am wrapping functions in a Python module called someapi

. Functions someapi

only expect positional arguments , which in most cases have the pain of number . I would like to enable callers with flexibility in how they can pass arguments to my wrappers. Below are examples of the wrapper prompts that I would like to allow:

# foo calls someapi.foo()
foo(1, 2, 3, 4)
foo(1, 2, 3, 4, 5) # but forward only 1st 4 to someapi.foo
foo([1, 2, 3, 4])
foo([1, 2, 3, 4, 5, 6]) # but forward only 1st 4 to someapi.foo
foo({'x':1, 'y':2, 'z':3, 'r':4})
foo(x=1, y=2, z=3, r=4)
foo(a=0, b=0, x=1, y=2, z=3, r=4) # but forward only x,y,z,r someapi.foo

I see no need to support the collapsed case of mixed positional and keywords:

foo(3, 4, x=1, y=2)

Here is my first stab at implementing the handling of such arguments for the shell foo

calling someapi.foo

:

def foo(*args, **kwargs):
    # BEGIN arguments un/re-packing
    a = None
    kwa = None
    if len(args) > 1:
        # foo(1, 2, 3, 4)
        a = args
    elif len(args) == 1:
        if isinstance(args[0], (list, tuple)) and len(args[0]) > 1:
            # foo([1, 2, 3, 4])
            a = args[0]
        if isinstance(args[0], dict):
            # foo({'x':1, 'y':2, 'z':3, 'r':4})
            kwa = args[0]
    else:
        # foo(x=1, y=2, z=3, r=4)
        kwa = kwargs

    if a:
        (x, y, z, r) = a
    elif kwa:
        (x, y, z, r) = (kwa['x'], kwa['y'], kwa['z'], kwa['r'])
    else:
        raise ValueError("invalid arguments")
    # END arguments un/re-packing

    # make call forwarding unpacked arguments 
    someapi.foo(x, y, z, r)

It performs the job as expected as far as I can tell, but there are two problems:

  • Can I do it better in a more idiomatic Python style ?
  • I have a dozen functions someapi

    to wrap around, so how do I avoid copying and setting the entire block between the BEGIN / END marks in each wrapper?

I don't know the answer to question 1 yet.

Here, however, I am trying to solve problem 2.

So, I've defined a generic argument handler based on a simple specification names

. names

to specify a few things depending on the actual call to the shell:

  • How many arguments are unpacked from *args

    ? (see below len(names)

    test)
  • What keyword arguments are expected in **kwargs

    ? (see generator expression returning a tuple below)

Here's the new version:

def unpack_args(names, *args, **kwargs):
    a = None
    kwa = None
    if len(args) >= len(names):
        # foo(1, 2, 3, 4...)
        a = args
    elif len(args) == 1:
        if isinstance(args[0], (list, tuple)) and len(args[0]) >= len(names):
            # foo([1, 2, 3, 4...])
            a = args[0]
        if isinstance(args[0], dict):
            # foo({'x':1, 'y':2, 'z':3, 'r':4...})
            kwa = args[0]
    else:
        # foo(x=1, y=2, z=3, r=4)
        kwa = kwargs
    if a:
        return a
    elif kwa:
        if all(name in kwa.keys() for name in names):
            return (kwa[n] for n in names)
        else:
            raise ValueError("missing keys:", \
                [name for name in names if name not in kwa.keys()])
    else:
        raise ValueError("invalid arguments")

This allows me to implement wrapper functions like this:

def bar(*args, **kwargs):
    # arguments un/re-packing according to given of names
    zargs = unpack_args(('a', 'b', 'c', 'd', 'e', 'f'), *args, **kwargs)
    # make call forwarding unpacked arguments 
    someapi.bar(*zargs)

I think I have achieved all the advantages over the version foo

above that I was looking for:

  • Include callers with the required flexibility.

  • Compact shape, cut for copy and paste.

  • Flexible protocol for positional arguments: bar

    can be called with 7, 8 or more positional arguments, or a long list of numbers, but only the first 6 are considered, for example, this would allow iterations to process a long list of numbers (for example, think about geometry coordinates):

    # meaw expects 2 numbers
    n = [1,2,3,4,5,6,7,8]
    for i in range(0, len(n), 2):
        meaw(n[i:i+2])
  • Flexible protocol for keyword arguments: more keywords can be specified than are actually used, or a dictionary can have more elements than are used.

Going back to question 1 above, can I do better and make it more Pythonic?

Also, I would like to ask you to review my solution: do you see any errors? Did I miss something? how to improve it?

+3


source to share


1 answer


Python is a very powerful language that allows you to manipulate code the way you want, but understanding what you are doing is difficult. You can use a module for this inspect

. An example of using a function in someapi

. I'll only cover positional arguments in this example, you can figure out how to proceed with this. You can do it like this:

import inspect
import someapi

def foo(args*):
    argspec = inspect.getargspec(someapi.foo)

    if len(args) > len(argspec.args):
        args = args[:len(argspec.args)]

    return someapi.foo(*args)

This will determine if there are enough arguments given for foo

, and if so, it will get rid of the extra arguments. On the other hand, if there are too few arguments, it just won't do anything and let it foo

handle errors.

Now to make it more pythonic. An ideal way to wrap many functions using the same pattern is to use decorator syntax (assuming familiarity with this object is assumed if you want to learn more, then look at the docs at http://www.python.org/doc ). Although, since the decorator syntax is mainly used for functions that are in development and not to port another API, we will create a decorator, but just use it as a factory pattern for our API. To make this a factory, we'll use a module functools

to help us out (so the wrapped function looks like it should). Therefore, we can include our example in:

import inspect
import functools
import someapi

def my_wrapper_maker(func):
    @functools.wraps(func)
    def wrapper(args*):
        argspec = inspect.getargspec(func)

        if len(args) > len(argspec.args):
            args = args[:len(argspec.args)]

        return func(*args)
    return wrapper

foo = my_wrapper_maker(someapi.foo)


Finally, if it someapi

has a relatively large API that can change between versions (or we just want to make our source file more modular so that it can wrap any API), then we can automate the application my_wrapper_maker

to whatever is exported by the module someapi

. We will do it like this:

__all__ = ['my_wrapper_maker']

# Add the entire API of someapi to our program.
for func in someapi.__all__:
    # Only add in bindings for functions.
    if callable(getattr(someapi, func)):
        globals()[func] = my_wrapper_maker(getattr(someapi, func))
        __all__.append(func)

This is probably considered the most pythonic way to implement this, it takes full advantage of Python's metaprogramming resources and allows the programmer to use this API wherever they want, depending on the specific one someapi

.

Note. ... This is the most idiomatic way to do it, it really is an opinion. I personally think that this follows the philosophy set out in the Zen of the Python quite well and therefore it is very idiomatic for me.

+4


source






More articles:


All Articles