Syntax for specifying plugins

[rmcgibbo]

Currently, to use a plugin, the ini file has to specify the relative or absolute filesystem path to the plugin.

To make distributing pip install-able plugins easier, it would be useful to have something like a setuptools entrypoint type syntax too.

[emmatyping]

Plugins are still rather experimental, so this would probably need to wait for a more stable plugin API to be implemented.

[rmcgibbo]

Making it easier for developers to distribute easy-to-use plugins could help uncover any issues by getting more usage.

[chadrik]

Hi all,
I'm afraid that this issue may become a blocker for getting the attrs project fully mypy-compliant.

Users of mypy will most often want to use the very latest version in order to get new bugfixes and features, but users of attrs will likely have more restrictions that force them to remain on older versions of the library. That means attrs users who upgrade to newer versions of mypy could (and will) inadvertently introduce a version of the attrs plugin which is not compatible with their version of attrs. Solutions have been suggested along the lines of creating a new github project that just holds the attrs stub, but the main benefit of that is that it frees up the attrs project to release on its own schedule rather than being pinned to mypy's. That's a good start, but it does not solve the primary problem of ensuring that attrs version X gets attrs-mypy-plugin version X.

I've invited @hynek to comment here, but I'm afraid that the level of inconvenience for both attrs developers and users may ultimately stall the feature entirely (already 8 months in the making).

I'd like to propose a compromise. It's already possible for third-parties to author plugins and load them via mypy.ini, via a path to a plugin file. If #4734 (or something like it) is merged but omitted from the docs or release notes, the current situation changes only slightly. Once merged, this feature will allow a few projects like attrs, which are at the bleeding edge of static type checking and well aware of the unsupported risks involved, to push the tooling forward, while shouldering the responsibility of making adjustments to our plugin if the API changes. On our end we can run our stub tests against a matrix of the recent mypy versions, and advertise compatible versions with each release of attrs.

So let's compare our options here:

• A. leave the plugin bundled here: users of attrs who upgrade mypy will eventually run into errors or bugs due to incompatibilities between the plugin version and their attrs version
• B. bundle the plugin with attrs: users of attrs who upgrade mypy will eventually run into errors due to changes in mypy's internal API

Both approaches will eventually lead to problems. Given all the baggage attached to option A, I definitely prefer B. And honestly, if the status quo remains, the best option that I can see is for us to bundle the plugin with attrs and encourage them to use the existing mypy.ini plugin registration feature....

I'd like to see an alternative to pkg_resources for something like this ideally (perhaps built on importlib.resources?

@ethanhs I've read up on importlib.resources, and will find some time to play around with it soon. It seems quite open-ended: do you have any ideas on how a third-party library would communicate the existence of a plugin via a ResourceLoader?

The main requirement is that we need a cheap way to register or broadcast the dotted path to a plugin class. We can do that simply enough by tweaking the existing mypy.ini approach to indicate the module to import. No need for pkg_resources or importlib.resources at all. The mypy.ini approach is less automatic, but that could be seen as a plus rather than a negative. What do you think?

[emmatyping]

@chadrik The comment about importlib.resources was just an idea. pkg_resources does a fair amount to have a nice API, and I don't think duplicating that would be easy. And I'm not really convinced it would be a good idea now. Fundamentally, you have to scan sys.path to have a similar API, which is going to be slow no matter what.

I do like the idea of using mypy.ini to point to a plugin, via perhaps an entry_point like string.