dumbdbm should not commit if in read mode

[joanthanng]

BPO	28847
Nosy	@vadmium, @serhiy-storchaka, @joanthanng
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	dbm_dumb_readonly.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-12-07.09:12:22.519>
created_at = <Date 2016-12-01.03:50:06.052>
labels = ['3.7', 'type-feature', 'library']
title = 'dumbdbm should not commit if in read mode'
updated_at = <Date 2017-03-31.16:36:22.839>
user = 'https://github.com/joanthanng'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:22.839>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-12-07.09:12:22.519>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2016-12-01.03:50:06.052>
creator = 'Jonathan Ng'
dependencies = []
files = ['45722']
hgrepos = []
issue_num = 28847
keywords = ['patch']
message_count = 13.0
messages = ['282133', '282158', '282166', '282167', '282170', '282218', '282219', '282264', '282265', '282266', '282269', '282271', '282602']
nosy_count = 4.0
nosy_names = ['python-dev', 'martin.panter', 'serhiy.storchaka', 'Jonathan Ng']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28847'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

[joanthanng]

Or at the very least, if there is an OSError in _update, an error should be raised instead of ignoring this error.

In the current state of the code, if there was an OSError while reading the dirfile, the dirfile would be overwritten with a blank file when it is closed.

[serhiy-storchaka]

Do you have concrete example when not ignoring an OSError in _update causes an issue? This is needed for writing tests.

[joanthanng]

I'm not sure how to create an OSError.

But perhaps something like this:

'''
from dbm import dumb
import os

db = dumb.open('temp', flag='n')
db['foo'] = 'bar'
db.close()

db = dumb.open('temp', flag='r')
print(len(db.keys()))
db.close

os.rename('temp.dir', 'temp_.dir') # simulates OSError
db = dumb.open('temp', flag='r')
os.rename('temp_.dir', 'temp.dir')
db.close()

db = dumb.open('temp', flag='r')
assert len(db.keys()) > 0
'''

[serhiy-storchaka]

This example is too artificial.

But there is a real issue: opening read-only files in read mode. Currently this causes a PermissionError on closing.

For backward compatibility flags 'r' and 'w' are ignored. I.e. opening with 'r' and 'w' creates a file if it is not existing, and opening with 'r' allows modifying the database. Since 3.6 this emits deprecation warnings (bpo-21708). In future versions this will be an error.

Proposed patch makes two changes:

1. The index file no longer written if the database was not modified. This increases performance and adds a support of read-only files.

2. A deprecation warning is raised when the index file is absent in 'r' and 'w' modes. In future versions this will be an error.

May be the first change can be backported.

[joanthanng]

#1 makes sense to be backported.

On Thu, Dec 1, 2016 at 8:41 PM, Serhiy Storchaka <report@bugs.python.org>
wrote:

Serhiy Storchaka added the comment:

This example is too artificial.

But there is a real issue: opening read-only files in read mode. Currently
this causes a PermissionError on closing.

For backward compatibility flags 'r' and 'w' are ignored. I.e. opening
with 'r' and 'w' creates a file if it is not existing, and opening with 'r'
allows modifying the database. Since 3.6 this emits deprecation warnings
(bpo-21708). In future versions this will be an error.

Proposed patch makes two changes:

1. The index file no longer written if the database was not modified. This
   increases performance and adds a support of read-only files.

2. A deprecation warning is raised when the index file is absent in 'r'
   and 'w' modes. In future versions this will be an error.

May be the first change can be backported.

----------
keywords: +patch
stage: -> patch review
type: behavior -> enhancement
versions: +Python 3.7
Added file: http://bugs.python.org/file45722/dbm_dumb_readonly.patch

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue28847\>

---

[python-dev]

New changeset 0516f54491cb by Serhiy Storchaka in branch '2.7':
Issue bpo-28847: dubmdbm no longer writes the index file in when it is not
https://hg.python.org/cpython/rev/0516f54491cb

[serhiy-storchaka]

The first part is committed in 2.7. I'll commit it in 3.5-3.7 after releasing 3.6.0.

[vadmium]

Serhiy: The Windows buildbots are having trouble removing read-only files. Maybe restore the write mode for the end of the test, or fix support.rmtree()? See <https://docs.python.org/3/library/shutil.html#rmtree-example\>.