Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to freeze list of iotile plugins #472

Open
timburke opened this issue Jun 28, 2018 · 3 comments
Open

Add ability to freeze list of iotile plugins #472

timburke opened this issue Jun 28, 2018 · 3 comments
Labels
iotile-core Related to iotile-core package type:enhancement Marks a proposed enhancement request

Comments

@timburke
Copy link
Member

When running on an SBC with an SD card, enumerating the filesystem inside pkg_resources to find iotile plugins is very slow, adding 5-10 seconds to the start up time of any iotile command line script. It's not possible to actually speed up pkg_resources but in situations where we know that the user is not installing / removing plugins (i.e. when the file system is read only), we can prescan all installed plugins and produce an entry_points.txt file similar to the component registry that contains all of the entry points.

If this file exists it can be consulted before pkg_resources is imported to speed up application load time:

See pkg_resources slow behavior, known bug, here:
pypa/setuptools#926

@timburke timburke added iotile-core Related to iotile-core package type:enhancement Marks a proposed enhancement request labels Jun 28, 2018
@timburke
Copy link
Member Author

This will require changes across coretools:

  • A new class in iotile.core.utilities called EntryPointManager that checks for entry_points.txt and loads it otherwise it loads pkg_resources. It should have an iter_entry_points API as a drop in replacement for pkg_resources usage.
  • Port all Coretools packages to use EntryPointManager rather than directly using pkg_resources.

For this to be useful it has to be an all or nothing thing. If anybody still imports pkg_resources the slow performance will return.

@timburke
Copy link
Member Author

It seems like a lot of larger projects have the same issue with pkg_resources.iter_entry_points and are settling on the alternative entrypoints package: marcelotduarte/cx_Freeze#216 (comment).

@timburke
Copy link
Member Author

This basic ability was added in PR #645. You can now iotile registry freeze and iotile registry unfreeze, however there are a few remaining items:

  1. This only affects parts of CoreTools that have been ported over to use the centralized extension system Refactor the rest of CoreTools to use centralized extension system #624. Any part of CoreTools still using pkg_resources directly will not pick up the frozen plugins.

  2. There needs to be a clear logging message on program start saying that frozen plugins are being used so that users don't get confused and wonder why their newly pip installed packages aren't being detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
iotile-core Related to iotile-core package type:enhancement Marks a proposed enhancement request
Projects
None yet
Development

No branches or pull requests

1 participant