Add support for pickle protocol special methods

[warsaw]

It might be kind of cool for @attr.s to optionally support __getstate__() and __setstate__(). I'm using the following and it seems to work well:

    def __getstate__(self):
        state = super().__getstate__()
        # Don't recurse when getting a pickle-able dictionary in case any of
        # our attributes is also an @attr.s and we want to retain its
        # object nature, instead of turning it into a dictionary which
        # can't be unpickled correctly.
        state.update(attr.asdict(self, recurse=False))
        return state

    def __setstate__(self, state):
        super().__setstate__(state)
        for field in attr.fields(self.__class__):
            setattr(self, field.name, state.get(field.name, field.default))

[Tinche]

Doesn't this work already? See #79 and the related pull request.

[warsaw]

If I remove my class's __getstate__() and __setstate__() I get the following failure:

Traceback (most recent call last):
  File "/home/barry/projects/ubuntu/allsnappy/ubuntu-image/ubuntu_image/state.py", line 73, in __next__
    step()
  File "/home/barry/projects/ubuntu/allsnappy/ubuntu-image/ubuntu_image/builder.py", line 301, in prepare_filesystems
    volumes = self.gadget.volumes.values()
AttributeError: 'Attribute' object has no attribute 'volumes'

(This is open source code, but in a currently unpushed branch.) In any case, the important thing is that self.gadget is an attr.s decorated class too, and self.gadget.volumes is an attr.ib instance. So the problem could be the recursion in the pickling class. I specifically call attr.asdict(self, recurse=False).

I suppose it's also possible that the missing super call causes the state of subclasses to not be preserved. (I'm just visually inspecting the attrs code and don't see a super call.) My pickle-able class is a subclass so the super class state needs to be preserved too.

[Tinche]

Hm, if you could prepare a small test case and give us your Python version, I can take a look.

(Yes, attrs never calls super()).

[warsaw]

Yep, thanks. I'll try to craft one up.

[hynek]

So do we have a executive summary of what’s going on here? :)

[warsaw]

@hynek I am currently unable to produce a boiled down example, so feel free to close this for now. If I can craft a small reproducer I'll submit and reopen.

[immerrr]

Here's a short reproducer that exploits the fact that __reduce__ seems to take precedence over __getstate__:

import attr
import pickle


class B:
    def __new__(cls, *args, **kwargs):
        retval = object.__new__(cls)
        retval.args = args
        return retval

    def __reduce__(self):
        return type(self), self.args

    def __repr__(self):
        return repr(self.args)


@attr.s
class C(B):
    foo = attr.ib()
    bar = attr.ib()

b1 = B(1, 2)
b2 = B(foo=1, bar=2)
c1 = C(1, 2)
c2 = C(foo=1, bar=2)

print('attr.__version__', attr.__version__)
print('b1', pickle.loads(pickle.dumps(b1)))  # works fine
print('b2', pickle.loads(pickle.dumps(b2)))  # works fine
print('c1', pickle.loads(pickle.dumps(c1)))  # works fine
print('c2', pickle.loads(pickle.dumps(c2)))  # fails

Here's the output I see locally:

$ python ~/test_attrs.py 
attr.__version__ 17.4.0
b1 (1, 2)
b2 ()
c1 C(foo=1, bar=2)
Traceback (most recent call last):
  File "/home/immerrr/test_attrs.py", line 32, in <module>
    print('c2', pickle.loads(pickle.dumps(c2)))  # fails
TypeError: __init__() missing 2 required positional arguments: 'foo' and 'bar'

I hit this when inheriting from Exception, which appears to be implemented in a similar manner in Py3.

[hynek]

This should be solved by #642.