Python: Typehints for argparse.Namespace objects
Is there a way for Python static analyzers (e.g. in PyCharm, other IDEs) to pick Typehints on objects argparse.Namespace
? Example:
parser = argparse.ArgumentParser()
parser.add_argument('--somearg')
parsed = parser.parse_args(['--somearg','someval']) # type: argparse.Namespace
the_arg = parsed.somearg # <- Pycharm complains that parsed object has no attribute 'somearg'
If I remove the type declaration in the inline comment, PyCharm doesn't complain, but it doesn't receive invalid attributes either. For example:
parser = argparse.ArgumentParser()
parser.add_argument('--somearg')
parsed = parser.parse_args(['--somearg','someval']) # no typehint
the_arg = parsed.somaerg # <- typo in attribute, but no complaint in PyCharm. Raises AttributeError when executed.
Any ideas?
Update
Inspired by Austin 's answer below, the simplest solution I could find is to use namedtuples
:
from collections import namedtuple
ArgNamespace = namedtuple('ArgNamespace', ['some_arg', 'another_arg'])
parser = argparse.ArgumentParser()
parser.add_argument('--some-arg')
parser.add_argument('--another-arg')
parsed = parser.parse_args(['--some-arg', 'val1', '--another-arg', 'val2']) # type: ArgNamespace
x = parsed.some_arg # good...
y = parsed.another_arg # still good...
z = parsed.aint_no_arg # Flagged by PyCharm!
While this is satisfactory, I still don't like repeating argument names. If the argument list grows significantly, it will be tedious to update both locations. The ideal would be to somehow extract the arguments from the object parser
like this:
parser = argparse.ArgumentParser()
parser.add_argument('--some-arg')
parser.add_argument('--another-arg')
MagicNamespace = parser.magically_extract_namespace()
parsed = parser.parse_args(['--some-arg', 'val1', '--another-arg', 'val2']) # type: MagicNamespace
I haven't been able to find anything in a module argparse
that would make this possible, and I'm still not sure if any static analysis tool can be smart enough to get these values and not cause the IDE to stop grinding.
Search more ...
Update 2
In hpaulj's post, the closest I could find in the above method that "magically" extracts the attributes of the parsed object is one that extracts the attribute dest
from each of the _action
s parsers .
parser = argparse.ArgumentParser()
parser.add_argument('--some-arg')
parser.add_argument('--another-arg')
MagicNamespace = namedtuple('MagicNamespace', [act.dest for act in parser._actions])
parsed = parser.parse_args(['--some-arg', 'val1', '--another-arg', 'val2']) # type: MagicNamespace
But this still does not result in attribute errors being flagged in static analysis. This is also true if I am passing namespace=MagicNamespace
in a call parser.parse_args
.
source to share
Consider defining an extension class for argparse.Namespace
that provides the types of hints you want:
class MyProgramArgs(argparse.Namespace):
def __init__():
self.somearg = 'defaultval' # type: str
Then use namespace=
to navigate to parse_args
:
def process_argv():
parser = argparse.ArgumentParser()
parser.add_argument('--somearg')
nsp = MyProgramArgs()
parsed = parser.parse_args(['--somearg','someval'], namespace=nsp) # type: MyProgramArgs
the_arg = parsed.somearg # <- Pycharm should not complain
source to share
I don't know anything about how PyCharm handles these types, but understands the code Namespace
.
argparse.Namespace
- simple class; essentially an object with multiple methods that make it easier to see attributes. And for ease of unittesting it has a method __eq__
. You can read the definition in the file argparse.py
.
parser
is reacted with a namespace most common way - getattr
, setattr
, hasattr
. This way you can use almost any string dest
, even the ones that you cannot access using the syntax .dest
.
Make sure you don't confuse the parameter add_argument
type=
; what a function.
Using your own class Namespace
(from scratch or subclassing) as suggested in another answer might be a better option. This is briefly described in the documentation. Namespace object . I have not seen this much, although I have asked him to handle special storage requirements on several occasions. Therefore, you will have to experiment.
Using subparameters may break using custom namespace class, http://bugs.python.org/issue27859
Note the handling of default values. The default for most actions argparse
is None
. It's handy to use this after parsing to do something special if the user hasn't provided this option.
if args.foo is None:
# user did not use this optional
args.foo = 'some post parsing default'
else:
# user provided value
pass
This can affect the type of prompts. Regardless of which solution you try, pay attention to the defaults.
A namedtuple
won't work like Namespace
.
First, the correct use of the custom namespace class:
nm = MyClass(<default values>)
args = parser.parse_args(namespace=nm)
That is, you started an instance of this class and passed it as a parameter. The one returned args
will be the same instance, with the new attributes set by parsing.
Secondly, namedtuple can only be created, it cannot be changed.
In [72]: MagicSpace=namedtuple('MagicSpace',['foo','bar'])
In [73]: nm = MagicSpace(1,2)
In [74]: nm
Out[74]: MagicSpace(foo=1, bar=2)
In [75]: nm.foo='one'
...
AttributeError: can't set attribute
In [76]: getattr(nm, 'foo')
Out[76]: 1
In [77]: setattr(nm, 'foo', 'one') # not even with setattr
...
AttributeError: can't set attribute
The namespace must work with getattr
and setattr
.
Another problem with namedtuple
is that it doesn't set any information about type
. It just defines the names of the fields / attributes. So nothing is needed for static input.
While it is easy to get the expected attribute names from parser
, you cannot get the expected types.
For a simple parser:
In [82]: parser.print_usage()
usage: ipython3 [-h] [-foo FOO] bar
In [83]: [a.dest for a in parser._actions[1:]]
Out[83]: ['foo', 'bar']
In [84]: [a.type for a in parser._actions[1:]]
Out[84]: [None, None]
Actions dest
are the normal name of the attribute. But type
it is not the expected static type for this attribute. It is a function that may or may not convert the input string. Here None
means that the input string is saved as is.
Since static typing and argparse
require different information, there is no easy way to generate one from the other.
I think the best you can do is create your own parameter database, perhaps in a dictionary, and create both a namespace class and a parsecr for them using your own utility functions.
Let's say dd
- a dictionary with the necessary keys. Then we can create an argument with:
parser.add_argument(dd['short'],dd['long'], dest=dd['dest'], type=dd['typefun'], default=dd['default'], help=dd['help'])
You or someone will have to come up with a namespace class definition that sets default
(simple) and static type (hard?) From such a dictionary.
source to share
The typed parser argument was created for this very purpose. Wrapping argparse
. Your example is implemented as:
from tap import Tap
class ArgumentParser(Tap):
somearg: str
parsed = ArgumentParser().parse_args(['--somearg', 'someval'])
the_arg = parsed.somearg
Here's a photo of it in action.
Full disclosure: I am one of the creators of this library.
source to share