Improve error messages for argparse choices using enum

[shangxiao]

BPO	42501
Nosy	@rhettinger, @miss-islington, @tirkarthi, @shangxiao
PRs	bpo-42501: Revise the usage note for Enums with the choices #23563 [3.9] bpo-42501: Revise the usage note for Enums with the choices (GH-23563) #23573

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/rhettinger'
closed_at = <Date 2020-11-30.21:22:19.709>
created_at = <Date 2020-11-29.10:20:31.016>
labels = ['type-bug', '3.10', 'docs']
title = 'Improve error messages for argparse choices using enum'
updated_at = <Date 2020-11-30.21:22:19.708>
user = 'https://github.com/shangxiao'

bugs.python.org fields:

activity = <Date 2020-11-30.21:22:19.708>
actor = 'rhettinger'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2020-11-30.21:22:19.709>
closer = 'rhettinger'
components = ['Documentation']
creation = <Date 2020-11-29.10:20:31.016>
creator = 'shangxiao'
dependencies = []
files = []
hgrepos = []
issue_num = 42501
keywords = ['patch']
message_count = 8.0
messages = ['382051', '382054', '382058', '382063', '382094', '382096', '382168', '382193']
nosy_count = 5.0
nosy_names = ['rhettinger', 'paul.j3', 'miss-islington', 'xtreak', 'shangxiao']
pr_nums = ['23563', '23573']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42501'
versions = ['Python 3.10']

[shangxiao]

Summary
-------

The argparse module mentions that it will happily accept an enum for the choices argument option [1]. There are currently 2 issues with this:

1. The usage displays non user-friendly values for the enum;
2. The error message for invalid values is not consistent with those provided by iterable-based choices in that it doesn't show the list of available choices. This becomes a problem when you specify the metavar as no choices are displayed at all.

Example of 1
------------

For example using the GameMove example in the argparse documentation:

 class GameMove(Enum):
 ROCK = 'rock'
 PAPER = 'paper'
 SCISSORS = 'scissors'

 parser = argparse.ArgumentParser(prog='game.py')
 parser.add_argument('move', type=GameMove, choices=GameMove)
 parser.print_help()

Gives the usage:

usage: game.py [-h] {GameMove.ROCK,GameMove.PAPER,GameMove.SCISSORS}

positional arguments:
 {GameMove.ROCK,GameMove.PAPER,GameMove.SCISSORS}

optional arguments:
 -h, --help show this help message and exit

As you can see, the string representations of the valid enum members is used instead of the values.

Example of 2
------------

Below is an example of the error message shown for invalid enum values, with metavar specified:

 parser = argparse.ArgumentParser(prog='game.py')
 parser.add_argument('move', metavar='your-move', type=GameMove, choices=GameMove)
 parser.parse_args(['asdf'])

Gives the following output:

usage: game.py [-h] your-move
game.py: error: argument your-move: invalid GameMove value: 'asdf'

This is in contrast with using standard iterable-based choices:

 parser = argparse.ArgumentParser(prog='game.py')
 parser.add_argument('move', metavar='your-move', choices=[x.value for x in GameMove])
 parser.parse_args(['asdf'])

Gives the following output:

usage: game.py [-h] your-move
game.py: error: argument your-move: invalid choice: 'asdf' (choose from 'rock', 'paper', 'scissors')

Analysis of 2
--------------

The reason for this behaviour is because argparse attempts to convert to the correct type before checking the choices membership and an invalid enum value results in the first error message.

It would be helpful (& consistent) if the choices are also displayed along with the invalid value message.

Possible Solution
-----------------

I believe that both issues could be solved by using the enum member values in the choices and then checking for choice membership _before_ the value is converted to an enum (ie for enums only).

[1] https://docs.python.org/3/library/argparse.html#choices

[rhettinger]

I'm inclined to state in the docs that choices is designed to work with collections of strings and that there is no special support for enums.

Paul, what do you think?

[tirkarthi]

https://bugs.python.org/issue25061 also had some discussion over error message display for enums

[shangxiao]

Oh apologies, I had "open" selected when I searched for prior issues.

Upon reading that issue I agree with Mr Hettinger's points about enum values not being a concern of the parser.

The solution for my needs was simple enough: I made my own action which simply set choices based on member values (I'm using user-friendly strings) and converted to the correct type upon being called [1].

Instead of removing any mention of enums from the docs - would a small example showing how to deal with them be worthwhile?

[1]
def enum_action_factory(enum_class):

 class EnumAction(argparse.Action):
 def __init__(self, option_strings, dest, **kwargs):
 kwargs["choices"] = [member.value for member in enum_class]
 super().__init__(option_strings, dest, **kwargs)

 def __call__(self, parser, namespace, values, option_string):
 if isinstance(values, str):
 converted_values = enum_class(values)
 else:
 converted_values = [enum_class(value) for value in values]
 setattr(namespace, self.dest, converted_values)

 return EnumAction

[paulj3]

choices is fine for a few strings, but quickly becomes awkward with other types and large numbers. The testing isn't an issue, since it just does a simple in/contains test. But display, whether in usage, help or error, is problematic if you try anything too fancy. metavar gets around some of those issues, but doesn't change the error messages.

Custom type or action is the best alternative.

I'm in favor omitting the enums mention in the docs, since it seems to be more confusing than helpful.

[rhettinger]

How does PR 23563 look to you all?

[rhettinger]

New changeset 7f82f22 by Raymond Hettinger in branch 'master':
bpo-42501: Revise the usage note for Enums with the choices (GH-23563)
7f82f22