Test imaplib API on all methods specified in RFC 3501

[zvyn]

BPO	22137
Nosy	@pitrou, @bitdancer, @soltysh, @zvyn
Files	imaplib_test_rfc3501.patch imaplib_test_rfc3501v2.patch new_imap_tests.diff: Initial proposal for imap tests

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-08-05.00:17:13.020>
labels = []
title = 'Test imaplib API on all methods specified in RFC 3501'
updated_at = <Date 2015-12-14.16:35:47.177>
user = 'https://github.com/zvyn'

bugs.python.org fields:

activity = <Date 2015-12-14.16:35:47.177>
actor = 'maciej.szulik'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2014-08-05.00:17:13.020>
creator = 'zvyn'
dependencies = []
files = ['36263', '36271', '41264']
hgrepos = ['325']
issue_num = 22137
keywords = []
message_count = 11.0
messages = ['224790', '224865', '224873', '241016', '241043', '241169', '241770', '241784', '256072', '256331', '256393']
nosy_count = 5.0
nosy_names = ['pitrou', 'r.david.murray', 'jesstess', 'maciej.szulik', 'zvyn']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue22137'
versions = []

Linked PRs

• gh-66335: Add tests for imaplib command methods #152872
• [3.15] gh-66335: Add tests for imaplib command methods (GH-152872) #152873
• [3.14] gh-66335: Add tests for imaplib command methods (GH-152872) #152874
• [3.13] gh-66335: Add tests for imaplib command methods (GH-152872) #152875
• gh-66335: Test uppercase IMAP4 command names #152876
• [3.15] gh-66335: Test uppercase IMAP4 command names (GH-152876) #153080
• [3.14] gh-66335: Test uppercase IMAP4 command names (GH-152876) #153081
• [3.13] gh-66335: Test uppercase IMAP4 command names (GH-152876) #153082

[zvyn]

I finished writing tests for all methods which are specified in RFC 3501 but left out extensions for now. The attached patch will trigger many resource warnings. Do you have an idea why the sockets aren't closed?

[pitrou]

Perhaps you need to close the client connection explicitly? (i.e. shutdown())

[zvyn]

Okay that was stupid from me (it was 5am when I submitted it), sorry.

[soltysh]

Milan, thanks for your patch, I've added comments to it (look for review link next to submitted file), can you address them please?
Other option is, if you don't mind, I'll take over this issue and finish on top of your work.

[soltysh]

One more thing I didn't mention before is, please update it to match current default, as it's failing patching, right now.

[soltysh]

Milan one more thing to change in your patch, in all your TCs you're modifying client state, I mean this part:

# lets cheat a bit here:
client.state = 'SELECTED'

It's OK to do that on the mocked server, but this is not acceptable doing on the tested library. It's better to call client.login() to change the state to 'AUTH' properly and only after that, call the actual method under test.

[zvyn]

Thanks a lot for reviewing my patch! I'm currently travalling so I'm totally
fine with you taking over this issue (would take me a few weeks till I can
work on it).

[soltysh]

Thanks Milan for the info, I was about to ping you about that. I have a working prototype for the tests that I've been working on during PyCon sprints with David. I'll try to submit new patch within a couple of next days.

[soltysh]

Resubmitting from http://bugs.python.org/issue25591, which I filled in unnecessarily.

Here are the rewritten test case I've developed during PyCon sprints along with David. Once we'll have the entire test suite in place it'll be safer to introduce any changes I'd like to cover, including updating the implementation to newest RFC 3501 version.

[bitdancer]

I'm afraid I don't remember the details of our conversations at PyCon. Specifically, I don't remember what problem(s) it is you are trying to solve with this patch.

What is the reason for not just adding your new tests to the ThreadedNetworkTests test case? That way they would be run both in regular mode and SSL mode. You seem to have copied bits of the code from that test into yours, but it mostly moves the code away from our usual conventions. That is, we generally put the support classes at the top level and then subclass them in test methods that need variations, rather than defining them in setup methods (where they would inaccessible unless assigned to self). Also, the existing reap_server method is to be preferred over a chain of addCleanups. Finally, we are moving all tests to using unittest.main() so that they can be called using unittest only instead of requiring regrtest, so the change in the __main__ block is incorrect.

Now, all that said, I can see several things I'd do to refactor the existing tests to make them clearer and easier to modify. For example, there are better ways to get reap_threads called on all test methods than wrapping each one individually (ex: tearDownClass). And I prefer setup methods that do addCleanup (in this case, addCleanup(self.reap_server), rather than the existing context manager approach, though I don't know how widespread my preferred pattern is in our test suite yet.

All of which has little to do with the title of this issue :). So, perhaps we should open a new issue for refactoring the tests, and then come back to this one once we are both satisfied with the refactoring? We can cite this issue as the motivation for the refactoring.

[soltysh]

Per the discussion we've had with David on IRC here's what happening. The last patch I've submitted is meant to clean up the tests by:

1. having single _setup method
2. the _setup method should addClenup to clean the environment
3. have stubs with implemented mostly used methods, all others should be implemented ad-hoc in test methods
4. use unittest.main() for running tests

The current idea is to create tests along the current one, to show how the new improve the readability. For this work I'm reopening previously submitted issue http://bugs.python.org/issue25591

The current issue will be on dependent on it and no further work should be done unless we have the tests cleared.

[serhiy-storchaka]

PR #152703 adds command-level tests for essentially every imaplib method that takes command arguments — create, delete, rename, subscribe, unsubscribe, list, select, status, copy, fetch, store, search, sort, thread (and their UID variants), login, enable, the ACL/QUOTA/ANNOTATION commands, and proxyauth — using request/response samples taken from RFC 3501 and the relevant extension RFCs (e.g. RFC 5256 for SORT/THREAD).

This covers most of what this issue asks for. Argument-less commands (NOOP, CHECK, CLOSE, EXPUNGE, CAPABILITY, …) are not all exercised yet, so it doesn't fully close the issue, but it's a large step in that direction.