Add PEP484 stubs (python-attrs#238)

* Add PEP484 stubs

* Deploy .pyi stubs alongside .py files.

This is the recommended approach for 3rd party stubs.
See: python/typing#84 (comment)

* Add support for the new type argument.

* Add tests for stubs and address a few issues.

* Improve declaration of private vs public objects in stubs

* More stub tests

* Separate the stub tests into their own tox env

it does not make sense to test the stubs in multiple python *runtime* environments (e.g. python 3.5, 3.6, pypy3) because the results of static analysis wrt attrs is not dependent on the runtime.  Moreover, mypy is not installing correctly in pypy3 which has nothing to do with attrs.

* Update the manifest with stub files

* Remove mypy from the dev requirements

* Allow _CountingAttr to be instantiated, but not Attribute.

* Incorporate defaults into attr.ib typing

* Fix a bug with validators.and_

* Add more tests

* Remove _CountingAttr from public interface

It is crucial to ensure that make_class() works with attr.ib(), as a result we no longer have any functions that care about _CountingAttr.

* Lie about return type of Factory

this allows for an abbreviated idiom: `x: List[int] = Factory(list)`

* Add tox stubs env to travis

* used the wrong comment character in mypy tests

* Improve overloads using PyCharm order-based approach

overloads are pretty broken in mypy.  the best we can do for now is target PyCharm, which is much more forgiving.

* Remove features not yet working in mypy. Document remaining issues.

* Test stubs against euresti fork of mypy with attrs plugin

Copied the pytest plugin from mypy for testing annotations: It is not an officially supported API and using the plugin from mypy could break after any update.

* Add some types and TypeVars to some types. Make tests pass

* Suppress warnings about named attribute access from fields()

e.g.  fields(C).x
Eventually it would be good to add support for returning NamedTuple from the mypy plugin

* Add WIP mypy-doctest plugin

* Deal with a few remaining type issues in the docs

* sphinx doctest: don't turn warnings into errors.

doing so makes the tests abort after the first failed group.

* Update "type: ignore" comments to reflect issues fixed in mypy plugin

* doctest2: improve output formatting

* Update manifest

* static tests: use inline error declarations

* More tests

* Tests passing (with notes about remaining issues)

* Attempt to get latest plugin from euresti working

* Issues fixed.

Had to place calls to attr.ib under a class definition.

* Deal with a PyCharm bug

* Minor test improvements

* Make tests prettier

* Use 2 decorators instead of 3

* doctest2: add support for skipping mypy tests

* Add tests for inheritance, eq, and cmp

* Add fixmes and todos

* Rename convert to converter

* Attribute.validator is always a single validator

* Conform stubs to typeshed coding style

And add auto_attrib kw

* backport style fixes from typeshed

* Add test cases to cover forward references and Any

* Add fixes for forward references and Any

* Address typeshed review notes

* Use Sequence instead of List/Tuple for validator arg

list and tuple are invariant and so prevent passing subtypes of _ValidatorType

* backports changes from typeshed #1914

* backport changes from typeshed #1933

* Prevent mypy tests from getting picked up

Evidently the discovery rules changed recently for pytest.

* make our doctest extension compatible with latest sphinx

* Adjustments to the tests

* Fix flake and manifest tests (hopefully)

* Fix tests on pypy3 (hopefully)

* Update stubs from typeshed

Also update tests.

* Make PEP 561-compliant

* minor cleanup

* Consolidate stub support files into stub directory

In preparation for removing them from the pyi_stubs branch.

* Get tests passing

This is a final test of the current stubs before moving the stub tests to a new branch.

* Revert stub test additions

Replace with a simple mypy pass/fail test

* get pre-commit passing

* Address review feedback

* Move typing test up in tox envlist

.gitignore

@@ -3,6 +3,7 @@
 .cache
 .coverage*
 .hypothesis
+.mypy_cache
 .pytest_cache
 .tox
 build

.pre-commit-config.yaml

@@ -4,6 +4,9 @@ repos:
     hooks:
       - id: black
         language_version: python3.6
+        # override until resolved: https://github.com/ambv/black/issues/402
+        files: \.pyi?$
+        types: []
 
   - repo: https://github.com/asottile/seed-isort-config
     rev: v1.0.1

.travis.yml

@@ -46,6 +46,8 @@ matrix:
       env: TOXENV=readme
     - python: "3.6"
       env: TOXENV=changelog
+    - python: "3.6"
+      env: TOXENV=typing
 
   allow_failures:
     - python: "3.6-dev"

MANIFEST.in

@@ -3,6 +3,10 @@ include LICENSE *.rst *.toml .readthedocs.yml .pre-commit-config.yaml
 # Don't package GitHub-specific files.
 exclude .github/*.md .travis.yml codecov.yml
 
+# Stubs
+include src/attr/py.typed
+recursive-include src *.pyi
+
 # Tests
 include tox.ini .coveragerc conftest.py
 recursive-include tests *.py

setup.py

@@ -115,4 +115,5 @@ def find_meta(meta):
         classifiers=CLASSIFIERS,
         install_requires=INSTALL_REQUIRES,
         extras_require=EXTRAS_REQUIRE,
+        include_package_data=True,
     )

src/attr/__init__.pyi

@@ -0,0 +1,240 @@
+from typing import (
+    Any,
+    Callable,
+    Dict,
+    Generic,
+    List,
+    Optional,
+    Sequence,
+    Mapping,
+    Tuple,
+    Type,
+    TypeVar,
+    Union,
+    overload,
+)
+
+# `import X as X` is required to make these public
+from . import exceptions as exceptions
+from . import filters as filters
+from . import converters as converters
+from . import validators as validators
+
+_T = TypeVar("_T")
+_C = TypeVar("_C", bound=type)
+
+_ValidatorType = Callable[[Any, Attribute, _T], Any]
+_ConverterType = Callable[[Any], _T]
+_FilterType = Callable[[Attribute, Any], bool]
+# FIXME: in reality, if multiple validators are passed they must be in a list or tuple,
+# but those are invariant and so would prevent subtypes of _ValidatorType from working
+# when passed in a list or tuple.
+_ValidatorArgType = Union[_ValidatorType[_T], Sequence[_ValidatorType[_T]]]
+
+# _make --
+
+NOTHING: object
+
+# NOTE: Factory lies about its return type to make this possible: `x: List[int] = Factory(list)`
+# Work around mypy issue #4554 in the common case by using an overload.
+@overload
+def Factory(factory: Callable[[], _T]) -> _T: ...
+@overload
+def Factory(
+    factory: Union[Callable[[Any], _T], Callable[[], _T]],
+    takes_self: bool = ...,
+) -> _T: ...
+
+class Attribute(Generic[_T]):
+    name: str
+    default: Optional[_T]
+    validator: Optional[_ValidatorType[_T]]
+    repr: bool
+    cmp: bool
+    hash: Optional[bool]
+    init: bool
+    converter: Optional[_ConverterType[_T]]
+    metadata: Dict[Any, Any]
+    type: Optional[Type[_T]]
+    def __lt__(self, x: Attribute) -> bool: ...
+    def __le__(self, x: Attribute) -> bool: ...
+    def __gt__(self, x: Attribute) -> bool: ...
+    def __ge__(self, x: Attribute) -> bool: ...
+
+# NOTE: We had several choices for the annotation to use for type arg:
+# 1) Type[_T]
+#   - Pros: works in PyCharm without plugin support
+#   - Cons: produces less informative error in the case of conflicting TypeVars
+#     e.g. `attr.ib(default='bad', type=int)`
+# 2) Callable[..., _T]
+#   - Pros: more informative errors than #1
+#   - Cons: validator tests results in confusing error.
+#     e.g. `attr.ib(type=int, validator=validate_str)`
+# 3) type (and do all of the work in the mypy plugin)
+#   - Pros: in mypy, the behavior of type argument is exactly the same as with
+#     annotations.
+#   - Cons: completely disables type inspections in PyCharm when using the
+#     type arg.
+# We chose option #1 until either PyCharm adds support for attrs, or python 2
+# reaches EOL.
+
+# `attr` lies about its return type to make the following possible:
+#     attr()    -> Any
+#     attr(8)   -> int
+#     attr(validator=<some callable>)  -> Whatever the callable expects.
+# This makes this type of assignments possible:
+#     x: int = attr(8)
+#
+# This form catches explicit None or no default but with no other arguments returns Any.
+@overload
+def attrib(
+    default: None = ...,
+    validator: None = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    convert: None = ...,
+    metadata: Optional[Mapping[Any, Any]] = ...,
+    type: None = ...,
+    converter: None = ...,
+    factory: None = ...,
+) -> Any: ...
+
+# This form catches an explicit None or no default and infers the type from the other arguments.
+@overload
+def attrib(
+    default: None = ...,
+    validator: Optional[_ValidatorArgType[_T]] = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    convert: Optional[_ConverterType[_T]] = ...,
+    metadata: Optional[Mapping[Any, Any]] = ...,
+    type: Optional[Type[_T]] = ...,
+    converter: Optional[_ConverterType[_T]] = ...,
+    factory: Optional[Callable[[], _T]] = ...,
+) -> _T: ...
+
+# This form catches an explicit default argument.
+@overload
+def attrib(
+    default: _T,
+    validator: Optional[_ValidatorArgType[_T]] = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    convert: Optional[_ConverterType[_T]] = ...,
+    metadata: Optional[Mapping[Any, Any]] = ...,
+    type: Optional[Type[_T]] = ...,
+    converter: Optional[_ConverterType[_T]] = ...,
+    factory: Optional[Callable[[], _T]] = ...,
+) -> _T: ...
+
+# This form covers type=non-Type: e.g. forward references (str), Any
+@overload
+def attrib(
+    default: Optional[_T] = ...,
+    validator: Optional[_ValidatorArgType[_T]] = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    convert: Optional[_ConverterType[_T]] = ...,
+    metadata: Optional[Mapping[Any, Any]] = ...,
+    type: object = ...,
+    converter: Optional[_ConverterType[_T]] = ...,
+    factory: Optional[Callable[[], _T]] = ...,
+) -> Any: ...
+@overload
+def attrs(
+    maybe_cls: _C,
+    these: Optional[Dict[str, Any]] = ...,
+    repr_ns: Optional[str] = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    slots: bool = ...,
+    frozen: bool = ...,
+    str: bool = ...,
+    auto_attribs: bool = ...,
+) -> _C: ...
+@overload
+def attrs(
+    maybe_cls: None = ...,
+    these: Optional[Dict[str, Any]] = ...,
+    repr_ns: Optional[str] = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    slots: bool = ...,
+    frozen: bool = ...,
+    str: bool = ...,
+    auto_attribs: bool = ...,
+) -> Callable[[_C], _C]: ...
+
+# TODO: add support for returning NamedTuple from the mypy plugin
+class _Fields(Tuple[Attribute, ...]):
+    def __getattr__(self, name: str) -> Attribute: ...
+
+def fields(cls: type) -> _Fields: ...
+def fields_dict(cls: type) -> Dict[str, Attribute]: ...
+def validate(inst: Any) -> None: ...
+
+# TODO: add support for returning a proper attrs class from the mypy plugin
+# we use Any instead of _CountingAttr so that e.g. `make_class('Foo', [attr.ib()])` is valid
+def make_class(
+    name: str,
+    attrs: Union[List[str], Tuple[str, ...], Dict[str, Any]],
+    bases: Tuple[type, ...] = ...,
+    repr_ns: Optional[str] = ...,
+    repr: bool = ...,
+    cmp: bool = ...,
+    hash: Optional[bool] = ...,
+    init: bool = ...,
+    slots: bool = ...,
+    frozen: bool = ...,
+    str: bool = ...,
+    auto_attribs: bool = ...,
+) -> type: ...
+
+# _funcs --
+
+# TODO: add support for returning TypedDict from the mypy plugin
+# FIXME: asdict/astuple do not honor their factory args.  waiting on one of these:
+# https://github.com/python/mypy/issues/4236
+# https://github.com/python/typing/issues/253
+def asdict(
+    inst: Any,
+    recurse: bool = ...,
+    filter: Optional[_FilterType] = ...,
+    dict_factory: Type[Mapping[Any, Any]] = ...,
+    retain_collection_types: bool = ...,
+) -> Dict[str, Any]: ...
+
+# TODO: add support for returning NamedTuple from the mypy plugin
+def astuple(
+    inst: Any,
+    recurse: bool = ...,
+    filter: Optional[_FilterType] = ...,
+    tuple_factory: Type[Sequence] = ...,
+    retain_collection_types: bool = ...,
+) -> Tuple[Any, ...]: ...
+def has(cls: type) -> bool: ...
+def assoc(inst: _T, **changes: Any) -> _T: ...
+def evolve(inst: _T, **changes: Any) -> _T: ...
+
+# _config --
+
+def set_run_validators(run: bool) -> None: ...
+def get_run_validators() -> bool: ...
+
+# aliases --
+
+s = attributes = attrs
+ib = attr = attrib
+dataclass = attrs  # Technically, partial(attrs, auto_attribs=True) ;)

src/attr/converters.pyi

@@ -0,0 +1,8 @@
+from typing import TypeVar, Optional
+from . import _ConverterType
+
+_T = TypeVar("_T")
+
+def optional(
+    converter: _ConverterType[_T]
+) -> _ConverterType[Optional[_T]]: ...

src/attr/exceptions.pyi

@@ -0,0 +1,7 @@
+class FrozenInstanceError(AttributeError):
+    msg: str = ...
+
+class AttrsAttributeNotFoundError(ValueError): ...
+class NotAnAttrsClassError(ValueError): ...
+class DefaultAlreadySetError(RuntimeError): ...
+class UnannotatedAttributeError(RuntimeError): ...

src/attr/filters.pyi

@@ -0,0 +1,5 @@
+from typing import Union
+from . import Attribute, _FilterType
+
+def include(*what: Union[type, Attribute]) -> _FilterType: ...
+def exclude(*what: Union[type, Attribute]) -> _FilterType: ...

src/attr/validators.pyi

@@ -0,0 +1,14 @@
+from typing import Container, List, Union, TypeVar, Type, Any, Optional, Tuple
+from . import _ValidatorType
+
+_T = TypeVar("_T")
+
+def instance_of(
+    type: Union[Tuple[Type[_T], ...], Type[_T]]
+) -> _ValidatorType[_T]: ...
+def provides(interface: Any) -> _ValidatorType[Any]: ...
+def optional(
+    validator: Union[_ValidatorType[_T], List[_ValidatorType[_T]]]
+) -> _ValidatorType[Optional[_T]]: ...
+def in_(options: Container[_T]) -> _ValidatorType[_T]: ...
+def and_(*validators: _ValidatorType[_T]) -> _ValidatorType[_T]: ...