Push batch / transaction onto a thread-local stack inside 'with'.

[tseaver]

Prep. for #514.

[coveralls]

Coverage remained the same when pulling e5e4518 on tseaver:push_batch_xact_on_stack_in_context_mgr into 77cfe10 on GoogleCloudPlatform:master.

[coveralls]

Coverage remained the same when pulling 8190f4c on tseaver:push_batch_xact_on_stack_in_context_mgr into 77cfe10 on GoogleCloudPlatform:master.

[tseaver]

Nesting batches "works": it just "suspends" the outer batch until the inner one exits. It does mean that the "inner" changes will be sent to the back-end before the "outer" ones, but folks using Batch aren't supposed to care about that, right?

The problem I was trying to solve was to make discovery of any current batch / transaction straightforward, even if the application does weird stuff like nesting (maybe accidentally).

If nesting is not a desirable feature, then we can close this PR and do something like your suggestion instead.

[tseaver]

Motivating example:

from gcloud.datastore.batch import _BATCHES, Batch
def put(*entities):
current = _BATCHES.top
in_batch = current is not None
if not in_batch:
current = Batch()
for entity in entities:
current.put(entity)
if not in_batch:
current.commit()
def delete(*keys):
current = _BATCHES.top
in_batch = current is not None
if not in_batch:
current = Batch()
for key in keys:
current.delete(key)
if not in_batch:
current.commit()

which will all Just Work(TM) even in the face of nesting.

[dhermes]

I keep confusing myself. The key focus here is that with must be called and when __exit__ occurs the stack is happy. Hence there is no such thing as interleaving.

That said, I still think the maintenance cost of a stack is not worth the minimal gain in functionality (which many users may never see). Users motivated enough to want nested / conditional transactions can accomplish the same thing with the API provided byBatch. The complexity becomes their burden, not ours.

I'm torn between "it already works as intended" and "let's not add code that we don't need to".

[tseaver]

That's actually why I made _BATCHES private (only capitalized as a sop to pylint :): Nobody but us chickens should touch it.

[dhermes]

It's not us touching it that worries me, it's bugs from exotic usage in the wild that we don't / won't have bandwidth or desire to deal with.

For example, what if someone calls a method that uses a context manager under the hood, and they expect

withTransaction() astxn:
funky_save()

to save in txn when instead it spawns a new transaction and doesn't save there.

As I said, I don't think the burden of a larger surface area justifies the small set of use cases that this enables syntactically.

I'm much more comfortable

• Clearly documenting nesting is not allowed
• Throwing an error if nesting is attempted

[tseaver]

I don't see how your example is problematic: if funky_save manages its own transaction, separate from the surrounding one, their changes will still be saved, but just not "atomically" with txn's changes: that would be what they want, presumably (or they wouldn't code funky_save that way). We aren't doing damage by supporting nesting.

@silvolu Do the other mappings support nesting?

[dhermes]

I'm not saying the functionality is damaging, I'm saying the maintenance burden on us is not worth it.

[tseaver]

Hmm, the maintenance / explainability of the consistent / nested one seems easier to me than maintaining / explaining the "Highlander" case.

[dhermes]

I suppose you're right. I'm probably just being paranoid. Let's hope it doesn't bite us.

[dhermes]

LGTM