Improve error handling and give hints what to do

[WolfgangFahl]

What's the problem this feature will solve?

pip dumps a tracestack at me:

`Traceback (most recent call last):
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 581, in _build_master
ws.require(requires)
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 909, in require
needed = self.resolve(parse_requirements(requirements))
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 800, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (py-3rdparty-mediawiki 0.8.0 (/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages), Requirement.parse('py-3rdparty-mediawiki==0.9.0'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/wf/.local/bin/wikibackup", line 33, in
sys.exit(load_entry_point('py-3rdparty-mediawiki==0.9.0', 'console_scripts', 'wikibackup')())
File "/home/wf/.local/bin/wikibackup", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.10/importlib/metadata/init.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/wikibot3rd/wikipush.py", line 11, in
from wikibot3rd.smw import SMWClient
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/wikibot3rd/smw.py", line 10, in
import pywikibot
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pywikibot/init.py", line 21, in
from pywikibot import config as _config
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pywikibot/config.py", line 64, in
from pywikibot.logging import error, output, warning
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pywikibot/logging.py", line 36, in
from pywikibot.tools import deprecated_args, issue_deprecation_warning
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pywikibot/tools/init.py", line 23, in
import pkg_resources
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 3260, in
def _initialize_master_working_set():
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 3234, in _call_aside
f(*args, **kwargs)
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 3272, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 583, in _build_master
return cls._build_from_requirements(requires)
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 596, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 795, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'py-3rdparty-mediawiki==0.9.0' distribution was not found and is required by the application
`

Describe the solution you'd like

Instead of a tracestack I'd like to get a proper error message and hints what to do.

Alternative Solutions

Link to a Community Page / FAQ to show common solutions to the problem.

Additional context

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

pip -V
pip 23.0.1 from /home/wf/.local/lib/python3.10/site-packages/pip (python 3.10)

I assume the problem is from using pip install commands in my jenkins nightly builds that are not properly isolated. I was assuming a pip install in a venv will only install things locally but it looks like somehow i forgot to venv or did some other mishap that leads to jenkins nightly build installation being mixed up with my main Ubuntu OS installation.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[WolfgangFahl]

The real problem seems to be that #!/hd/luxio/var/lib/jenkins/jobs/scan2wiki/workspace/.venv/bin/python3 is used as an interpreter for some command line scripts and i have no clue what causes this - the error message is not helpful at all.

[pradyunsg]

Thanks for reporting this! I agree that improving error messages around package installation with pip is an important thing to do, and your proposal is an understandable one. :)

However (sadly) these issues typically aren't usually as simple as "do X" but instead usually serve as symptom of something else going wrong earlier in the management of the environment, and it's non-trivial to do a root-cause analysis of these in as a generic manner to place in an FAQ. Further, this specific error message is not coming out of pip, or scripts that pip controls generation of. This is an issue with pkg_resources (or py-3rdparty-mediawiki). I suggest filing an issue against pypa/setuptools (which owns pkg_resources) for this.

In this case, you might also benefit from installing wheel prior to installing this package and rely on pip-generated scripts instead that won't exhibit this issue.

Between the following issues, I don't think there's not much value in tracking improving this specific instance, since it's outside of pip's control and falls under the purvue of the following issues:

• Add additional diagnostics to our error messages #10421 tracks improving errors raised by pip itself, although this issue is unrelated to the errors raised within pip itself.
• Deprecate call to setup.py install when wheel is absent for source distributions without pyproject.toml #8559 tracks improving handling when wheel is missing (moving away from setup.py install, which is what generates these scripts).
• Deprecate --no-binary implying setup.py install #11451 tracks another case where moving away from setup.py install in a different circumstance.