BUG: pickle RootLogger object

[lesteve]

It seems like #94 only fixed the case for logging.Logger objects and not logging.RootLogger objects (e.g. the logger you get when calling logging.getLogger()):

To reproduce:

importloggingimportcloudpicklelogger=logging.getLogger()
logging.basicConfig()
cloudpickle.dumps(logger)

Error:

TypeError: can't pickle _thread.RLock objects

Full traceback on master

TypeError Traceback (most recent call last)
<ipython-input-1-7f7ba95099c6> in <module>()
4 logger = logging.getLogger()
5 logging.basicConfig()
----> 6 cloudpickle.dumps(logger)
/home/local/lesteve/dev/cloudpickle/cloudpickle/cloudpickle.py in dumps(obj, protocol)
889 try:
890 cp = CloudPickler(file, protocol=protocol)
--> 891 cp.dump(obj)
892 return file.getvalue()
893 finally:
/home/local/lesteve/dev/cloudpickle/cloudpickle/cloudpickle.py in dump(self, obj)
266 self.inject_addons()
267 try:
--> 268 return Pickler.dump(self, obj)
269 except RuntimeError as e:
270 if 'recursion' in e.args[0]:
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in dump(self, obj)
407 if self.proto >= 4:
408 self.framer.start_framing()
--> 409 self.save(obj)
410 self.write(STOP)
411 self.framer.end_framing()
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
519 520 # Save the reduce() output and finally memoize the object
--> 521 self.save_reduce(obj=obj, *rv)
522 523 def persistent_id(self, obj):
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_reduce(self, func, args, state, listitems, dictitems, obj)
632 633 if state is not None:
--> 634 save(state)
635 write(BUILD)
636 /home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
474 f = self.dispatch.get(t)
475 if f is not None:
--> 476 f(self, obj) # Call unbound method with explicit self
477 return
478 /home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_dict(self, obj)
819 820 self.memoize(obj)
--> 821 self._batch_setitems(obj.items())
822 823 dispatch[dict] = save_dict
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in _batch_setitems(self, items)
845 for k, v in tmp:
846 save(k)
--> 847 save(v)
848 write(SETITEMS)
849 elif n:
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
474 f = self.dispatch.get(t)
475 if f is not None:
--> 476 f(self, obj) # Call unbound method with explicit self
477 return
478 /home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_list(self, obj)
779 780 self.memoize(obj)
--> 781 self._batch_appends(obj)
782 783 dispatch[list] = save_list
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in _batch_appends(self, items)
806 write(APPENDS)
807 elif n:
--> 808 save(tmp[0])
809 write(APPEND)
810 # else tmp is empty, and we're done
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
519 520 # Save the reduce() output and finally memoize the object
--> 521 self.save_reduce(obj=obj, *rv)
522 523 def persistent_id(self, obj):
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_reduce(self, func, args, state, listitems, dictitems, obj)
632 633 if state is not None:
--> 634 save(state)
635 write(BUILD)
636 /home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
474 f = self.dispatch.get(t)
475 if f is not None:
--> 476 f(self, obj) # Call unbound method with explicit self
477 return
478 /home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_dict(self, obj)
819 820 self.memoize(obj)
--> 821 self._batch_setitems(obj.items())
822 823 dispatch[dict] = save_dict
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in _batch_setitems(self, items)
845 for k, v in tmp:
846 save(k)
--> 847 save(v)
848 write(SETITEMS)
849 elif n:
/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
494 reduce = getattr(obj, "__reduce_ex__", None)
495 if reduce is not None:
--> 496 rv = reduce(self.proto)
497 else:
498 reduce = getattr(obj, "__reduce__", None)
TypeError: can't pickle _thread.RLock objects

[lesteve]

I tried to use pytest.mark.parametrize but I could not make it work with test classes deriving from unittest.TestCase (see this SO answer which seems to indicate that this is not possible)

[codecov-io]

Codecov Report

Merging #160 into master will increase coverage by 0.09%.
The diff coverage is 100%.

@@ Coverage Diff @@## master #160 +/- ##
==========================================
+ Coverage 83.73% 83.82% +0.09% 
==========================================
Files 1 1 Lines 535 538 +3 Branches 96 96 ==========================================
+ Hits 448 451 +3 
Misses 63 63 Partials 24 24

Impacted Files	Coverage Δ
cloudpickle/cloudpickle.py	83.82% <100%> (+0.09%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f14d19d...8551606. Read the comment docs.

[lesteve]

Since logging.RootLogger derives from logging.Logger I was wondering whether it would be worth to walk the class __mro__ in order to find a custom reduce function. I went for the simpler option.

[pitrou]

This looks fine to me, thanks.

[rgbkrk]

Given the last few, do you want to make another release @HyukjinKwon?

[lesteve]

Great to see this merged, thanks everyone!

[HyukjinKwon]

@rgbkrk Sure, will do soon later. I am a bit busy for relocating now :-).