test_epylint.py fails when invoked from a directory with a space in its name #6476

edreamleo · 2022-04-29T19:41:23Z

Bug description

test_epylint.py mysteriously fails in my environment, regardless of python version, and possibly regardless of pylint version.

Note: The name of my home directory is c:\Users\Edward Ream.

Blanks in the name of my home directory may be a factor.

Configuration

No response

Command used

`python -m pytest tests\test_epylint.py -vv`

Also fails with tox.

Pylint output

c:\leo.repo\ekr-pylint>python -m pytest tests\test_epylint.py -vv

c:\leo.repo\ekr-pylint>echo off
========================================================= test session starts =========================================================
platform win32 -- Python 3.9.6, pytest-7.1.1, pluggy-1.0.0 -- C:\Users\Edward Ream\Python\Python39\python.exe
cachedir: .pytest_cache
rootdir: c:\leo.repo\ekr-pylint, configfile: setup.cfg
collected 2 items

tests/test_epylint.py::test_epylint_good_command FAILED                                                                          [ 50%]
tests/test_epylint.py::test_epylint_strange_command FAILED                                                                       [100%]

============================================================== FAILURES ===============================================================
______________________________________________________ test_epylint_good_command ______________________________________________________

example_path = WindowsPath('C:/Users/Edward Ream/AppData/Local/Temp/pytest-of-Edward Ream/pytest-14/test_epylint_good_command0/my_app.py')

    def test_epylint_good_command(example_path: PosixPath) -> None:
        out, err = lint.py_run(
            # pylint: disable-next=consider-using-f-string
            "%s -E --disable=E1111 --msg-template '{category} {module} {obj} {line} {column} {msg}'"
            % example_path,
            return_std=True,
        )
        msg = out.read()
>       assert (
            msg
            == """\
    ************* Module my_app
     error my_app IvrAudioApp.run 4 8 Instance of 'IvrAudioApp' has no 'hassan' member
     """
        )
E       assert 'C:\\Users\\Edward does not exist\n' == ('************* Module my_app\n'\n " error my_app IvrAudioApp.run 4 8 Instance of 'IvrAudioApp' has no 'hassan' "\n 'member\n'\n ' ')
E         + C:\Users\Edward does not exist
E         - ************* Module my_app
E         -  error my_app IvrAudioApp.run 4 8 Instance of 'IvrAudioApp' has no 'hassan' member
E         -

tests\test_epylint.py:35: AssertionError
____________________________________________________ test_epylint_strange_command _____________________________________________________

example_path = WindowsPath('C:/Users/Edward Ream/AppData/Local/Temp/pytest-of-Edward Ream/pytest-14/test_epylint_strange_command0/my_app.py')

    def test_epylint_strange_command(example_path: PosixPath) -> None:
        out, err = lint.py_run(
            # pylint: disable-next=consider-using-f-string
            "%s -E --disable=E1111 --msg-template={category} {module} {obj} {line} {column} {msg}"
            % example_path,
            return_std=True,
        )
>       assert (
            out.read()
            == """\
    ************* Module {module}
     fatal
     ************* Module {obj}
     fatal
     ************* Module {line}
     fatal
     ************* Module {column}
     fatal
     ************* Module {msg}
     fatal
     ************* Module my_app
     error
     """
        )
E       AssertionError: assert 'C:\\Users\\Edward does not exist\n' == ('************* Module {module}\n'\n ' fatal\n'\n ' ************* Module {obj}\n'\n ' fatal\n'\n ' ************* Module {line}\n'\n ' fatal\n'\n ' ************* Module {column}\n'\n ' fatal\n'\n ' ************* Module {msg}\n'\n ' fatal\n'\n ' ************* Module my_app\n'\n ' error\n'\n ' ')
E         + C:\Users\Edward does not exist
E         - ************* Module {module}
E         -  fatal
E         -  ************* Module {obj}
E         -  fatal
E         -  ************* Module {line}
E         -  fatal
E         -  ************* Module {column}
E         -  fatal
E         -  ************* Module {msg}
E         -  fatal
E         -  ************* Module my_app
E         -  error
E         -

tests\test_epylint.py:52: AssertionError
======================================================= short test summary info =======================================================
FAILED tests/test_epylint.py::test_epylint_good_command - assert 'C:\\Users\\Edward does not exist\n' == ('************* Module my_ap...
FAILED tests/test_epylint.py::test_epylint_strange_command - AssertionError: assert 'C:\\Users\\Edward does not exist\n' == ('*******...
========================================================== 2 failed in 0.31s ==========================================================

c:\leo.repo\ekr-pylint>

Expected behavior

Test would pass.

The following line in the output above seems suspicious:

FAILED tests/test_epylint.py::test_epylint_good_command - assert 'C:\\Users\\Edward does not exist\n' == ('************* Module my_ap...

I would expect that the assert would involve C:\\Users\\Edward Ream

Pylint version

c:\leo.repo\leo-editor>python -m pylint-leo -v

pylint 2.13.5
astroid 2.11.2
Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)]

Note: Similar errors occur with python 3.6, 3.7, 3.8

OS / Environment

Leo 6.6.2-devel, devel branch, build b82ddf8693
2022-04-29 07:21:37 -0500
Python 3.9.6, PyQt version 6.2.4
Windows 10 AMD64 (build 10.0.22000) SP0

Additional dependencies

C:\leo.repo\leo-editor>python -m pip freeze

C:\leo.repo\leo-editor>echo off
alabaster==0.7.12
astroid==2.11.2
atomicwrites==1.4.0
attrs==21.4.0
Babel==2.10.1
beautifulsoup4==4.11.0
black==22.3.0
bleach==4.1.0
build==0.7.0
certifi==2021.10.8
charset-normalizer==2.0.12
click==8.1.2
colorama==0.4.4
csaps==1.1.0
cycler==0.11.0
defusedxml==0.7.1
dialite==0.5.3
dill==0.3.4
distlib==0.3.4
docutils==0.17.1
entrypoints==0.4
feedparser==6.0.8
filelock==3.6.0
flexx==0.8.3
fonttools==4.31.2
for==0.0.0.0.0.0a0.dev20210120
gitdb==4.0.9
GitPython==3.1.27
idna==3.3
ImageHash==4.2.1
imagesize==1.3.0
importlib-metadata==4.11.3
iniconfig==1.1.1
isort==5.10.1
jedi==0.18.1
Jinja2==3.1.1
jsonschema==4.4.0
jupyter-client==7.2.2
jupyter-core==4.9.2
jupyterlab-pygments==0.2.0
keyring==23.5.0
kiwisolver==1.4.2
lazy-object-proxy==1.7.1
lxml==4.8.0
Markdown==3.3.6
MarkupSafe==2.1.1
matplotlib==3.5.1
mccabe==0.7.0
mistune==0.8.4
mypy==0.942
mypy-extensions==0.4.3
nbclient==0.5.13
nbconvert==6.4.5
nbformat==5.2.0
nest-asyncio==1.5.5
numpy==1.22.3
packaging==21.3
pandocfilters==1.5.0
parso==0.8.3
pathspec==0.9.0
pep517==0.12.0
Pillow==9.0.1
pkginfo==1.8.2
platformdirs==2.5.1
pluggy==1.0.0
pscript==0.7.7
pwlf==2.1.0
py==1.11.0
pyDOE==0.3.8
pyenchant==3.2.2
Pygments==2.11.2
pylint==2.13.5
pyparsing==3.0.7
PyQt5==5.15.6
PyQt5-Qt5==5.15.2
PyQt5-sip==12.9.1
PyQt6==6.2.3
PyQt6-Qt6==6.2.4
PyQt6-sip==13.2.1
PyQt6-WebEngine==6.2.1
PyQt6-WebEngine-Qt6==6.2.4
PyQtWebEngine==5.15.5
PyQtWebEngine-Qt5==5.15.2
pyrsistent==0.18.1
pytest==7.1.1
python-dateutil==2.8.2
pytz==2022.1
PyWavelets==1.3.0
pywin32==303
pywin32-ctypes==0.2.0
PyYAML==6.0
pyzmq==22.3.0
readme-renderer==34.0
requests==2.27.1
requests-toolbelt==0.9.1
rfc3986==2.0.0
scipy==1.8.0
Send2Trash==1.8.0
sgmllib3k==1.0.0
six==1.16.0
smmap==5.0.0
snowballstemmer==2.2.0
soupsieve==2.3.2
Sphinx==4.5.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
testpath==0.6.0
toml==0.10.2
tomli==2.0.1
tomlkit==0.10.2
tornado==6.1
tox==3.25.0
tqdm==4.63.1
traitlets==5.1.1
twine==3.8.0
types-cryptography==3.3.20
types-docutils==0.18.0
types-Markdown==3.3.13
types-paramiko==2.8.19
types-PyYAML==6.0.7
types-requests==2.27.16
types-six==1.16.15
types-urllib3==1.26.11
typing_extensions==4.1.1
urllib3==1.26.9
virtualenv==20.14.1
webencodings==0.5.1
webruntime==0.5.8
websockets==10.2
Whoosh==2.7.4
windows-curses==2.3.0
wrapt==1.14.0
zipp==3.7.0

The text was updated successfully, but these errors were encountered:

jacobtylerwalls · 2022-04-30T01:51:27Z

Thanks for the report. Those two tests create a command line string to pass to py_run, but if the temp dir contains a space (yours did), they will fail, because py_run assumes a well-constructed string of command options. Any spaces in example_path need to be escaped before calling py_run. This could probably be done in def example_path before the return statement. Would you like to prepare a patch?

edreamleo · 2022-04-30T03:57:35Z

@jacobtylerwalls Yes, a patch would be a good first issue. I'll give it a go.

edreamleo · 2022-05-05T13:27:12Z

Update: I'm attempting to establish a baseline for running unit tests. It's turning into a project:

Create a new "Dev" user (on Windows) to avoid spaces in various paths.
Run all unit tests successfully on Windows/Python 3.10. At present, some (other) unit tests fail.
Complete reinstall of Python and pylint (after all tests pass).
- Does installing pylint create site-packages/easy-install.pth?
- Are additional packages required to run pytest? In particular, are the following needed?
  - pip install tomlkit
  - pip install gitpython (not python-git)
  - pip install pytest-benchmark
Run all unit tests successfully on Ubuntu/Python 3.10.

DanielNoord · 2022-05-05T14:47:50Z

Which tests are failing for you? Perhaps I can help resolve some issues!

edreamleo · 2022-05-05T15:01:08Z

@DanielNoord Thanks for the kind offer. I'll prepare a report for you soon, but first I'd like to do some homework :-)

edreamleo · 2022-05-05T16:00:38Z

@DanielNoord I've done a bit more reading about unit tests, and I nothing jumps out at me about what I might have done wrong, so here is a summary of what I have done.

Installation

Install Python3.10 into c:\Python\Python3.10.
Clone the pylint repo into c:\Repos\ekr-pylint.
Run pip install -e ekr-pylint

Here is the log:

c:\Repos>C:\Python\Python3.10\python.exe -m pip install -e ekr-pylint
Obtaining file:///C:/Repos/ekr-pylint
  Preparing metadata (setup.py) ... done
Requirement already satisfied: dill>=0.2 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (0.3.4)
Requirement already satisfied: platformdirs>=2.2.0 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (2.5.2)
Requirement already satisfied: astroid<=2.12.0-dev0,>=2.11.4 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (2.11.4)
Requirement already satisfied: isort<6,>=4.2.5 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (5.10.1)
Requirement already satisfied: mccabe<0.8,>=0.6 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (0.7.0)
Requirement already satisfied: tomlkit>=0.10.1 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (0.10.2)
Requirement already satisfied: tomli>=1.1.0 in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (2.0.1)
Requirement already satisfied: colorama in c:\python\python3.10\lib\site-packages (from pylint==2.14.0.dev0) (0.4.4)
Requirement already satisfied: lazy-object-proxy>=1.4.0 in c:\python\python3.10\lib\site-packages (from astroid<=2.12.0-dev0,>=2.11.4->pylint==2.14.0.dev0) (1.7.1)
Requirement already satisfied: wrapt<2,>=1.11 in c:\python\python3.10\lib\site-packages (from astroid<=2.12.0-dev0,>=2.11.4->pylint==2.14.0.dev0) (1.14.1)
Requirement already satisfied: setuptools>=20.0 in c:\python\python3.10\lib\site-packages (from astroid<=2.12.0-dev0,>=2.11.4->pylint==2.14.0.dev0) (58.1.0)
Installing collected packages: pylint
  Attempting uninstall: pylint
    Found existing installation: pylint 2.13.8
    Uninstalling pylint-2.13.8:
      Successfully uninstalled pylint-2.13.8
  Running setup.py develop for pylint
Successfully installed pylint-2.14.0.dev0

pip list reports:

C:\Repos\leo-editor>C:\Python\Python3.10\python.exe -m pip list
Package           Version     Editable project location
----------------- ----------- -------------------------
astroid           2.11.4
atomicwrites      1.4.0
attrs             21.4.0
colorama          0.4.4
dill              0.3.4
distlib           0.3.4
filelock          3.6.0
gitdb             4.0.9
GitPython         3.1.27
iniconfig         1.1.1
isort             5.10.1
lazy-object-proxy 1.7.1
mccabe            0.7.0
packaging         21.3
pip               22.0.4
platformdirs      2.5.2
pluggy            1.0.0
py                1.11.0
py-cpuinfo        8.0.0
pylint            2.14.0.dev0 c:\repos\ekr-pylint
pyparsing         3.0.8
PyQt6             6.3.0
PyQt6-Qt6         6.3.0
PyQt6-sip         13.3.1
pytest            7.1.2
pytest-benchmark  3.4.1
Send2Trash        1.8.0
setuptools        58.1.0
six               1.16.0
smmap             5.0.0
toml              0.10.2
tomli             2.0.1
tomlkit           0.10.2
tox               3.25.0
virtualenv        20.14.1
windows-curses    2.3.0
wrapt             1.14.1

where git reports C:\Apps\Git\cmd\git.exe

With this setup, unit tests fail in two different ways, depending on whether tox or pytest runs the test.

pytest -m pytest

c:\Repos\ekr-pylint>C:\Python\Python3.10\python.exe -m pytest
================================================= test session starts =================================================
platform win32 -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: c:\Repos\ekr-pylint, configfile: setup.cfg, testpaths: tests
plugins: benchmark-3.4.1
collected 1940 items

tests\test_check_parallel.py ..................................                                                  [  1%]
tests\test_deprecation.py ....                                                                                   [  1%]
tests\test_epylint.py ..                                                                                         [  2%]
tests\test_func.py ............                                                                                  [  2%]
tests\test_functional.py .....................................s...........s...................s................. [  7%]
...........................s.s.s................................................................................ [ 12%]
............FFF.........................sss.s................................................................... [ 18%]
................................................................................................................ [ 24%]
.....s..s...................ss..........................................................s....................... [ 30%]
................................s.........ss....F.s...s............................................s............ [ 36%]
....................s.....................................................................................       [ 41%]
tests\test_functional_directories.py .                                                                           [ 41%]
tests\test_import_graph.py ......                                                                                [ 41%]
tests\test_numversion.py ...........                                                                             [ 42%]
tests\test_pragma_parser.py ............                                                                         [ 43%]
tests\test_pylint_runners.py ........                                                                            [ 43%]
tests\test_regr.py .................x.                                                                           [ 44%]
tests\test_self.py ..........x................s................................................................. [ 49%]
...........................                                                                                      [ 50%]
tests\test_similar.py ...........                                                                                [ 51%]
tests\unittest_reporters_json.py ..                                                                              [ 51%]
tests\unittest_reporting.py .........                                                                            [ 51%]
tests\benchmark\test_baseline_benchmarks.py ...........                                                          [ 52%]
tests\checkers\unittest_base_checker.py ..                                                                       [ 52%]
tests\checkers\unittest_deprecated.py ........................                                                   [ 53%]
tests\checkers\unittest_design.py ...                                                                            [ 53%]
tests\checkers\unittest_format.py .........                                                                      [ 54%]
tests\checkers\unittest_imports.py ......                                                                        [ 54%]
tests\checkers\unittest_misc.py ...........                                                                      [ 55%]
tests\checkers\unittest_non_ascii_name.py ..........................x                                            [ 56%]
tests\checkers\unittest_refactoring.py ..                                                                        [ 56%]
tests\checkers\unittest_similar.py ..................                                                            [ 57%]
tests\checkers\unittest_spelling.py ssssssssssssssssssssssssssssssssss                                           [ 59%]
tests\checkers\unittest_stdlib.py .                                                                              [ 59%]
tests\checkers\unittest_strings.py .                                                                             [ 59%]
tests\checkers\unittest_typecheck.py ss.                                                                         [ 59%]
tests\checkers\unittest_utils.py .......................................                                         [ 61%]
tests\checkers\unittest_variables.py ..........                                                                  [ 62%]
tests\checkers\base\unittest_base.py s                                                                           [ 62%]
tests\checkers\base\unittest_multi_naming_style.py ....                                                          [ 62%]
tests\checkers\base\unittest_name_preset.py ....                                                                 [ 62%]
tests\checkers\unittest_unicode\unittest_bad_chars.py ....................................................       [ 65%]
tests\checkers\unittest_unicode\unittest_bidirectional_unicode.py .............................................. [ 67%]
.....                                                                                                            [ 67%]
tests\checkers\unittest_unicode\unittest_functions.py .......................................................... [ 70%]
..................................                                                                               [ 72%]
tests\checkers\unittest_unicode\unittest_invalid_encoding.py xx........x...........                              [ 73%]
tests\config\test_argparse_config.py ......                                                                      [ 74%]
tests\config\test_config.py ........                                                                             [ 74%]
tests\config\test_deprecations.py ............                                                                   [ 75%]
tests\config\test_find_default_config_files.py ............                                                      [ 75%]
tests\config\test_functional_config_loading.py ....................                                              [ 76%]
tests\config\unittest_config.py ........                                                                         [ 77%]
tests\extensions\test_check_docs_utils.py ..............                                                         [ 77%]
tests\extensions\test_private_import.py .....                                                                    [ 78%]
tests\functional\r\redundant_unittest_assert.py ss                                                               [ 78%]
tests\lint\test_namespace_packages.py .                                                                          [ 78%]
tests\lint\test_pylinter.py ..                                                                                   [ 78%]
tests\lint\test_utils.py F..                                                                                     [ 78%]
tests\lint\unittest_expand_modules.py .....                                                                      [ 78%]
tests\lint\unittest_lint.py ..............................................                                       [ 81%]
tests\message\test_no_removed_msgid_or_symbol_used.py .                                                          [ 81%]
tests\message\unittest_message.py .                                                                              [ 81%]
tests\message\unittest_message_definition.py ......                                                              [ 81%]
tests\message\unittest_message_definition_store.py .................                                             [ 82%]
tests\message\unittest_message_id_store.py ........                                                              [ 82%]
tests\primer\test_primer_external.py sssssssssss                                                                 [ 83%]
tests\primer\test_primer_stdlib.py sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 87%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 93%]
ssssssssss                                                                                                       [ 93%]
tests\profile\test_profile_against_externals.py s                                                                [ 93%]
tests\pyreverse\test_diadefs.py .........                                                                        [ 94%]
tests\pyreverse\test_diagrams.py .                                                                               [ 94%]
tests\pyreverse\test_inspector.py ........                                                                       [ 94%]
tests\pyreverse\test_main.py FF...................                                                               [ 95%]
tests\pyreverse\test_printer.py ..............                                                                   [ 96%]
tests\pyreverse\test_printer_factory.py .....                                                                    [ 96%]
tests\pyreverse\test_pyreverse_functional.py ...                                                                 [ 96%]
tests\pyreverse\test_utils.py ......................                                                             [ 98%]
tests\pyreverse\test_writer.py ...............                                                                   [ 98%]
tests\testutils\test_configuration_test.py .                                                                     [ 98%]
tests\testutils\test_functional_testutils.py ....                                                                [ 99%]
tests\testutils\test_lint_module_output_update.py s...                                                           [ 99%]
tests\testutils\test_output_line.py ........                                                                     [ 99%]
tests\testutils\test_package_to_lint.py ..                                                                       [ 99%]
tests\utils\unittest_ast_walker.py ..                                                                            [ 99%]
tests\utils\unittest_utils.py ..                                                                                 [100%]

====================================================== FAILURES =======================================================
_______________________________________ test_functional[typing_broken_noreturn] _______________________________________

self = <pylint.testutils.lint_module_test.LintModuleTest object at 0x000001EC2D17B010>

    def runTest(self) -> None:
>       self._runTest()
E       AssertionError: Wrong results for file "typing_broken_noreturn":
E
E       Expected in testdata:
E         26: broken-noreturn
E
E       Unexpected in testdata:
E         11: import-error

pylint\testutils\lint_module_test.py:111: AssertionError
________________________________ test_functional[typing_broken_noreturn_future_import] ________________________________

self = <pylint.testutils.lint_module_test.LintModuleTest object at 0x000001EC2D2C96F0>

    def runTest(self) -> None:
>       self._runTest()
E       AssertionError: Wrong results for file "typing_broken_noreturn_future_import":
E
E       Unexpected in testdata:
E         16: import-error

pylint\testutils\lint_module_test.py:111: AssertionError
------------------------------------------------ Captured stdout call -------------------------------------------------
************* Module C:\Repos\ekr-pylint\tests\functional\ext\typing\typing_broken_noreturn_future_import.rc
C:\Repos\ekr-pylint\tests\functional\ext\typing\typing_broken_noreturn_future_import.rc:1:0: E0015: Unrecognized option found: min_pyver (unrecognized-option)
____________________________________ test_functional[typing_broken_noreturn_py372] ____________________________________

self = <pylint.testutils.lint_module_test.LintModuleTest object at 0x000001EC2D3156C0>

    def runTest(self) -> None:
>       self._runTest()
E       AssertionError: Wrong results for file "typing_broken_noreturn_py372":
E
E       Unexpected in testdata:
E         13: import-error

pylint\testutils\lint_module_test.py:111: AssertionError
__________________________________________ test_functional[symlink_module1] ___________________________________________

self = <pylint.testutils.lint_module_test.LintModuleTest object at 0x000001EC3B01B7F0>

    def runTest(self) -> None:
>       self._runTest()
E       AssertionError: Wrong results for file "symlink_module":
E
E       Unexpected in testdata:
E          1: syntax-error

pylint\testutils\lint_module_test.py:111: AssertionError
______________________________________________ test_prepare_crash_report ______________________________________________

tmp_path = WindowsPath('C:/Users/Dev/AppData/Local/Temp/pytest-of-Dev/pytest-2/test_prepare_crash_report0')

>   ???
E   assert 'raise Exception(exception_content)' in 'First, please verify that the bug is not already filled:\nhttps://github.com/PyCQA/pylint/issues/\n\nThen create a ne...\leo.repo\\ekr-pylint\\tests\\lint\\test_utils.py", line 21, in test_prepare_crash_report\nException: Exmessage\n```\n'

c:\leo.repo\ekr-pylint\tests\lint\test_utils.py:32: AssertionError
_____________________________ test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests] _____________________________

>   ???
E   AssertionError: assert ['C:\\Repos\\...ylint\\tests'] == ['c:\\Repos\\...ylint\\tests']
E     At index 0 diff: 'C:\\Repos\\ekr-pylint\\tests' != 'c:\\Repos\\ekr-pylint\\tests'
E     Use -v to get more diff

c:\leo.repo\ekr-pylint\tests\pyreverse\test_main.py:63: AssertionError
__________________________ test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests\\data] __________________________

>   ???
E   AssertionError: assert ['C:\\Repos\\...ylint\\tests'] == ['c:\\Repos\\...ylint\\tests']
E     At index 0 diff: 'C:\\Repos\\ekr-pylint\\tests' != 'c:\\Repos\\ekr-pylint\\tests'
E     Use -v to get more diff

c:\leo.repo\ekr-pylint\tests\pyreverse\test_main.py:63: AssertionError

--------------------------------------------------------------------------------------------------- benchmark 'baseline': 11 tests ---------------------------------------------------------------------------------------------------
Name (time in ms)                                              Min                   Max                  Mean             StdDev                Median                IQR            Outliers       OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_baseline_benchmark_j1                                  8.5843 (1.0)          9.7963 (1.0)          8.8998 (1.0)       0.2921 (1.0)          8.7791 (1.0)       0.3288 (1.0)          19;8  112.3623 (1.0)         104           1
test_baseline_benchmark_j1_all_checks_single_file          90.9335 (10.59)       97.6698 (9.97)        92.7827 (10.43)     2.2545 (7.72)        91.8639 (10.46)     2.4965 (7.59)          2;1   10.7779 (0.10)         11           1
test_baseline_benchmark_check_parallel_j2                 746.4829 (86.96)      768.5774 (78.46)      758.3118 (85.21)     8.6398 (29.58)      761.1352 (86.70)    12.6003 (38.32)         2;0    1.3187 (0.01)          5           1
test_baseline_benchmark_j2                                760.5586 (88.60)      789.3623 (80.58)      777.1954 (87.33)    11.3103 (38.72)      777.0990 (88.52)    16.4745 (50.10)         2;0    1.2867 (0.01)          5           1
test_baseline_benchmark_j1_single_working_checker       1,018.9174 (118.70)   1,020.6394 (104.19)   1,019.8716 (114.60)    0.7522 (2.58)     1,020.1620 (116.20)    1.3008 (3.96)          2;0    0.9805 (0.01)          5           1
test_baseline_benchmark_j2_single_working_checker       1,325.5788 (154.42)   1,340.9242 (136.88)   1,332.8069 (149.76)    7.3177 (25.05)    1,330.5660 (151.56)   13.9056 (42.29)         2;0    0.7503 (0.01)          5           1
test_baseline_benchmark_j1_all_checks_lots_of_files     3,861.7138 (449.86)   3,945.7098 (402.78)   3,904.3358 (438.70)   32.1351 (110.02)   3,913.4528 (445.77)   44.4331 (135.14)        2;0    0.2561 (0.00)          5           1
test_baseline_lots_of_files_j1                          4,263.6537 (496.68)   4,313.5293 (440.32)   4,289.3875 (481.97)   17.6785 (60.53)    4,290.3040 (488.70)   14.5886 (44.37)         2;0    0.2331 (0.00)          5           1
test_baseline_lots_of_files_j1_empty_checker            4,276.1691 (498.14)   4,333.4108 (442.35)   4,299.4789 (483.10)   22.3253 (76.44)    4,294.7829 (489.21)   31.3913 (95.47)         2;0    0.2326 (0.00)          5           1
test_baseline_lots_of_files_j2_empty_checker            4,680.3263 (545.22)   4,741.2680 (483.99)   4,709.1548 (529.13)   27.6932 (94.81)    4,706.8280 (536.14)   51.9694 (158.06)        2;0    0.2124 (0.00)          5           1
test_baseline_lots_of_files_j2                          4,685.9644 (545.88)   4,746.0064 (484.47)   4,713.0657 (529.57)   24.7115 (84.61)    4,714.4347 (537.01)   40.8873 (124.35)        2;0    0.2122 (0.00)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
=============================================== short test summary info ===============================================
FAILED tests/test_functional.py::test_functional[typing_broken_noreturn] - AssertionError: Wrong results for file "ty...
FAILED tests/test_functional.py::test_functional[typing_broken_noreturn_future_import] - AssertionError: Wrong result...
FAILED tests/test_functional.py::test_functional[typing_broken_noreturn_py372] - AssertionError: Wrong results for fi...
FAILED tests/test_functional.py::test_functional[symlink_module1] - AssertionError: Wrong results for file "symlink_m...
FAILED tests/lint/test_utils.py::test_prepare_crash_report - assert 'raise Exception(exception_content)' in 'First, p...
FAILED tests/pyreverse/test_main.py::test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests] - AssertionError: as...
FAILED tests/pyreverse/test_main.py::test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests\\data] - AssertionErr...
========================= 7 failed, 1653 passed, 274 skipped, 6 xfailed in 342.19s (0:05:42) ==========================

python -m tox --recreate -e py310

c:\Repos\ekr-pylint>C:\Python\Python3.10\python.exe -m tox --recreate -e py310
GLOB sdist-make: c:\Repos\ekr-pylint\setup.py
py310 recreate: C:\Repos\ekr-pylint\.tox\py310
py310 installdeps: -rc:\Repos\ekr-pylint/requirements_test.txt
py310 inst: C:\Repos\ekr-pylint\.tox\.tmp\package\4\pylint-2.14.0.dev0.zip
py310 installed: astroid==2.11.4,atomicwrites==1.4.0,attrs==21.4.0,black==22.3.0,certifi==2021.10.8,cfgv==3.3.1,charset-normalizer==2.0.12,cli-ui==0.16.1,click==8.1.3,colorama==0.4.4,contextlib2==21.6.0,contributors-txt==0.7.5,coverage==6.3.2,coveralls==3.3.1,dill==0.3.4,distlib==0.3.4,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,flake8==4.0.1,flake8-typing-imports==1.12.0,gprof2dot==2021.2.21,identify==2.5.0,idna==3.3,iniconfig==1.1.1,isort==5.10.1,lazy-object-proxy==1.7.1,mccabe==0.6.1,mypy==0.950,mypy-extensions==0.4.3,nodeenv==1.6.0,packaging==21.3,pathspec==0.9.0,platformdirs==2.5.2,pluggy==1.0.0,pre-commit==2.19.0,py==1.11.0,py-cpuinfo==8.0.0,pycodestyle==2.8.0,pyenchant==3.2.2,pyflakes==2.4.0,pylint @ file:///C:/Repos/ekr-pylint/.tox/.tmp/package/4/pylint-2.14.0.dev0.zip,pyparsing==3.0.8,pytest==7.1.2,pytest-benchmark==3.4.1,pytest-cov==3.0.0,pytest-forked==1.4.0,pytest-profiling==1.7.0,pytest-timeout==2.1.0,pytest-xdist==2.5.0,PyYAML==6.0,requests==2.27.1,schema==0.7.5,six==1.16.0,tabulate==0.8.9,tbump==6.8.0,toml==0.10.2,tomli==2.0.1,tomlkit==0.10.2,types-pkg-resources==0.1.3,typing_extensions==4.2.0,Unidecode==1.3.4,urllib3==1.26.9,virtualenv==20.14.1,wrapt==1.14.1
py310 run-test-pre: PYTHONHASHSEED='748'
py310 run-test: commands[0] | pytest --benchmark-disable 'c:\Repos\ekr-pylint/tests/'
================================================= test session starts =================================================
platform win32 -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
cachedir: .tox\py310\.pytest_cache
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: c:\Repos\ekr-pylint, configfile: setup.cfg, testpaths: tests
plugins: benchmark-3.4.1, cov-3.0.0, forked-1.4.0, profiling-1.7.0, timeout-2.1.0, xdist-2.5.0
collected 1927 items / 2 errors

======================================================= ERRORS ========================================================
________________________________ ERROR collecting tests/primer/test_primer_external.py ________________________________
ImportError while importing test module 'c:\Repos\ekr-pylint\tests\primer\test_primer_external.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Python\Python3.10\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
c:\leo.repo\ekr-pylint\tests\primer\test_primer_external.py:15: in <module>
    ???
.tox\py310\lib\site-packages\pylint\testutils\primer.py:10: in <module>
    import git
E   ModuleNotFoundError: No module named 'git'
______________________________ ERROR collecting tests/testutils/test_package_to_lint.py _______________________________
ImportError while importing test module 'c:\Repos\ekr-pylint\tests\testutils\test_package_to_lint.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Python\Python3.10\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
c:\leo.repo\ekr-pylint\tests\testutils\test_package_to_lint.py:5: in <module>
    ???
.tox\py310\lib\site-packages\pylint\testutils\primer.py:10: in <module>
    import git
E   ModuleNotFoundError: No module named 'git'
=============================================== short test summary info ===============================================
ERROR tests/primer/test_primer_external.py
ERROR tests/testutils/test_package_to_lint.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================== 2 errors in 1.49s ==================================================
ERROR: InvocationError for command 'C:\Repos\ekr-pylint\.tox\py310\Scripts\pytest.EXE' --benchmark-disable tests (exited with code 2)
_______________________________________________________ summary _______________________________________________________
ERROR:   py310: commands failed

As you can see, the details elements don't format the test logs very well.

Any help would be appreciated.

DanielNoord · 2022-05-05T16:44:46Z

There are number of issues here. Some of which we should fix, some of which you need to fix yourself.

Apparently tox does not install gitpython. @Pierre-Sassoulas Could you investigate this? I have no experience with tox but I guess we should install -e .[testutils] somewhere?
@edreamleo You will need to install typing-extensions. This is not a runtime dependency on 3.10, but it is a test dependency on 3.10. We should probably fix that. I will look into this myself.
@edreamleo Then there is the symlink issue. Could you check if you have git symlinks enabled? Some of the discussions here should help: Check sets for duplicate entries #5880
As for the issues with the crash reporter: I don't know yet. Perhaps they resolve themselves after the others have been fixed 😄

edreamleo · 2022-05-05T17:25:23Z

@DanielNoord Thanks for your quick response.

gitpython: As shown above, "pip list" reports GitPython 3.1.27. Is there something more I should do?
git symlinks: I'm not sure I know what you mean. Do you have a URL for git symlinks?

DanielNoord · 2022-05-05T18:01:25Z

@DanielNoord Thanks for your quick response.

gitpython: As shown above, "pip list" reports GitPython 3.1.27. Is there something more I should do?

No apparently we have broken the tox command and nobody had noticed until now. But running pytest on its own should still work.

git symlinks: I'm not sure I know what you mean. Do you have a URL for git symlinks?

I think these two links should help:
https://www.quora.com/How-does-Git-handle-symbolic-links-on-Windows
https://github-wiki-see.page/m/git-for-windows/git/wiki/Symbolic-Links

Bascially, git can store and record symbolic links on all UNIX systems but struggles to do so on Windows (or at least, most previous versions). There are workarounds to fix this, so you'll need to explore what works best for you. Because we want to test pylint and its behaviour with symbolic links we have a test that utilises them.

edreamleo · 2022-05-06T12:11:23Z

@DanielNoord Progress, but still failures.

What I did

git config --global core.symlinks true
pip install typing-extensions

Note: According to my notes , I had to install the following (yesterday) before pytest would run any of the tests. I'll re-check with a completely clean installation after all tests pass.

- pip install tomlkit
- pip install gitpython  (not python-git)
- pip install pytest-benchmark

So maybe there are more missing dependencies.

Results (pytest)

c:\Repos\ekr-pylint>C:\Python\Python3.10\python.exe -m pytest
================================================= test session starts =================================================
platform win32 -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: c:\Repos\ekr-pylint, configfile: setup.cfg, testpaths: tests
plugins: benchmark-3.4.1
collected 1940 items

tests\test_check_parallel.py ..................................                                                  [  1%]
tests\test_deprecation.py ....                                                                                   [  1%]
tests\test_epylint.py ..                                                                                         [  2%]
tests\test_func.py ............                                                                                  [  2%]
tests\test_functional.py .....................................s...........s...................s................. [  7%]
...........................s.s.s................................................................................ [ 12%]
........................................sss.s................................................................... [ 18%]
................................................................................................................ [ 24%]
.....s..s...................ss..........................................................s....................... [ 30%]
................................s.........ss....F.s...s............................................s............ [ 36%]
....................s.....................................................................................       [ 41%]
tests\test_functional_directories.py .                                                                           [ 41%]
tests\test_import_graph.py ......                                                                                [ 41%]
tests\test_numversion.py ...........                                                                             [ 42%]
tests\test_pragma_parser.py ............                                                                         [ 43%]
tests\test_pylint_runners.py ........                                                                            [ 43%]
tests\test_regr.py .................x.                                                                           [ 44%]
tests\test_self.py ..........x................s................................................................. [ 49%]
...........................                                                                                      [ 50%]
tests\test_similar.py ...........                                                                                [ 51%]
tests\unittest_reporters_json.py ..                                                                              [ 51%]
tests\unittest_reporting.py .........                                                                            [ 51%]
tests\benchmark\test_baseline_benchmarks.py ...........                                                          [ 52%]
tests\checkers\unittest_base_checker.py ..                                                                       [ 52%]
tests\checkers\unittest_deprecated.py ........................                                                   [ 53%]
tests\checkers\unittest_design.py ...                                                                            [ 53%]
tests\checkers\unittest_format.py .........                                                                      [ 54%]
tests\checkers\unittest_imports.py ......                                                                        [ 54%]
tests\checkers\unittest_misc.py ...........                                                                      [ 55%]
tests\checkers\unittest_non_ascii_name.py ..........................x                                            [ 56%]
tests\checkers\unittest_refactoring.py ..                                                                        [ 56%]
tests\checkers\unittest_similar.py ..................                                                            [ 57%]
tests\checkers\unittest_spelling.py ssssssssssssssssssssssssssssssssss                                           [ 59%]
tests\checkers\unittest_stdlib.py .                                                                              [ 59%]
tests\checkers\unittest_strings.py .                                                                             [ 59%]
tests\checkers\unittest_typecheck.py ss.                                                                         [ 59%]
tests\checkers\unittest_utils.py .......................................                                         [ 61%]
tests\checkers\unittest_variables.py ..........                                                                  [ 62%]
tests\checkers\base\unittest_base.py s                                                                           [ 62%]
tests\checkers\base\unittest_multi_naming_style.py ....                                                          [ 62%]
tests\checkers\base\unittest_name_preset.py ....                                                                 [ 62%]
tests\checkers\unittest_unicode\unittest_bad_chars.py ....................................................       [ 65%]
tests\checkers\unittest_unicode\unittest_bidirectional_unicode.py .............................................. [ 67%]
.....                                                                                                            [ 67%]
tests\checkers\unittest_unicode\unittest_functions.py .......................................................... [ 70%]
..................................                                                                               [ 72%]
tests\checkers\unittest_unicode\unittest_invalid_encoding.py xx........x...........                              [ 73%]
tests\config\test_argparse_config.py ......                                                                      [ 74%]
tests\config\test_config.py ........                                                                             [ 74%]
tests\config\test_deprecations.py ............                                                                   [ 75%]
tests\config\test_find_default_config_files.py ............                                                      [ 75%]
tests\config\test_functional_config_loading.py ....................                                              [ 76%]
tests\config\unittest_config.py ........                                                                         [ 77%]
tests\extensions\test_check_docs_utils.py ..............                                                         [ 77%]
tests\extensions\test_private_import.py .....                                                                    [ 78%]
tests\functional\r\redundant_unittest_assert.py ss                                                               [ 78%]
tests\lint\test_namespace_packages.py .                                                                          [ 78%]
tests\lint\test_pylinter.py ..                                                                                   [ 78%]
tests\lint\test_utils.py F..                                                                                     [ 78%]
tests\lint\unittest_expand_modules.py .....                                                                      [ 78%]
tests\lint\unittest_lint.py ..............................................                                       [ 81%]
tests\message\test_no_removed_msgid_or_symbol_used.py .                                                          [ 81%]
tests\message\unittest_message.py .                                                                              [ 81%]
tests\message\unittest_message_definition.py ......                                                              [ 81%]
tests\message\unittest_message_definition_store.py .................                                             [ 82%]
tests\message\unittest_message_id_store.py ........                                                              [ 82%]
tests\primer\test_primer_external.py sssssssssss                                                                 [ 83%]
tests\primer\test_primer_stdlib.py sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 87%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 93%]
ssssssssss                                                                                                       [ 93%]
tests\profile\test_profile_against_externals.py s                                                                [ 93%]
tests\pyreverse\test_diadefs.py .........                                                                        [ 94%]
tests\pyreverse\test_diagrams.py .                                                                               [ 94%]
tests\pyreverse\test_inspector.py ........                                                                       [ 94%]
tests\pyreverse\test_main.py FF...................                                                               [ 95%]
tests\pyreverse\test_printer.py ..............                                                                   [ 96%]
tests\pyreverse\test_printer_factory.py .....                                                                    [ 96%]
tests\pyreverse\test_pyreverse_functional.py ...                                                                 [ 96%]
tests\pyreverse\test_utils.py ......................                                                             [ 98%]
tests\pyreverse\test_writer.py ...............                                                                   [ 98%]
tests\testutils\test_configuration_test.py .                                                                     [ 98%]
tests\testutils\test_functional_testutils.py ....                                                                [ 99%]
tests\testutils\test_lint_module_output_update.py s...                                                           [ 99%]
tests\testutils\test_output_line.py ........                                                                     [ 99%]
tests\testutils\test_package_to_lint.py ..                                                                       [ 99%]
tests\utils\unittest_ast_walker.py ..                                                                            [ 99%]
tests\utils\unittest_utils.py ..                                                                                 [100%]

====================================================== FAILURES =======================================================
__________________________________________ test_functional[symlink_module1] ___________________________________________

self = <pylint.testutils.lint_module_test.LintModuleTest object at 0x0000020A971F9390>

    def runTest(self) -> None:
>       self._runTest()
E       AssertionError: Wrong results for file "symlink_module":
E
E       Unexpected in testdata:
E          1: syntax-error

pylint\testutils\lint_module_test.py:111: AssertionError
______________________________________________ test_prepare_crash_report ______________________________________________

tmp_path = WindowsPath('C:/Users/Dev/AppData/Local/Temp/pytest-of-Dev/pytest-3/test_prepare_crash_report0')

>   ???
E   assert 'raise Exception(exception_content)' in 'First, please verify that the bug is not already filled:\nhttps://github.com/PyCQA/pylint/issues/\n\nThen create a ne...\leo.repo\\ekr-pylint\\tests\\lint\\test_utils.py", line 21, in test_prepare_crash_report\nException: Exmessage\n```\n'

c:\leo.repo\ekr-pylint\tests\lint\test_utils.py:32: AssertionError
_____________________________ test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests] _____________________________

>   ???
E   AssertionError: assert ['C:\\Repos\\...ylint\\tests'] == ['c:\\Repos\\...ylint\\tests']
E     At index 0 diff: 'C:\\Repos\\ekr-pylint\\tests' != 'c:\\Repos\\ekr-pylint\\tests'
E     Use -v to get more diff

c:\leo.repo\ekr-pylint\tests\pyreverse\test_main.py:63: AssertionError
__________________________ test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests\\data] __________________________

>   ???
E   AssertionError: assert ['C:\\Repos\\...ylint\\tests'] == ['c:\\Repos\\...ylint\\tests']
E     At index 0 diff: 'C:\\Repos\\ekr-pylint\\tests' != 'c:\\Repos\\ekr-pylint\\tests'
E     Use -v to get more diff

c:\leo.repo\ekr-pylint\tests\pyreverse\test_main.py:63: AssertionError

--------------------------------------------------------------------------------------------------- benchmark 'baseline': 11 tests ---------------------------------------------------------------------------------------------------
Name (time in ms)                                              Min                   Max                  Mean             StdDev                Median                IQR            Outliers       OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_baseline_benchmark_j1                                  8.5270 (1.0)          9.6231 (1.0)          8.7187 (1.0)       0.2019 (1.0)          8.6430 (1.0)       0.1201 (1.0)         14;14  114.6958 (1.0)         103           1
test_baseline_benchmark_j1_all_checks_single_file          88.0588 (10.33)       95.4196 (9.92)        90.9770 (10.43)     2.1681 (10.74)       90.8423 (10.51)     3.2948 (27.43)         4;0   10.9918 (0.10)         11           1
test_baseline_benchmark_check_parallel_j2                 739.1700 (86.69)      765.5585 (79.55)      749.7216 (85.99)    10.4671 (51.85)      745.1962 (86.22)    14.4866 (120.62)        2;0    1.3338 (0.01)          5           1
test_baseline_benchmark_j2                                754.9964 (88.54)      767.6548 (79.77)      760.9060 (87.27)     5.6017 (27.75)      762.8509 (88.26)     9.6501 (80.35)         3;0    1.3142 (0.01)          5           1
test_baseline_benchmark_j1_single_working_checker       1,019.5026 (119.56)   1,021.3025 (106.13)   1,020.4154 (117.04)    0.6954 (3.44)     1,020.3942 (118.06)    1.0432 (8.69)          2;0    0.9800 (0.01)          5           1
test_baseline_benchmark_j2_single_working_checker       1,308.1302 (153.41)   1,324.0677 (137.59)   1,317.0144 (151.06)    7.2413 (35.87)    1,318.6101 (152.56)   13.3816 (111.42)        1;0    0.7593 (0.01)          5           1
test_baseline_benchmark_j1_all_checks_lots_of_files     3,820.9051 (448.09)   3,846.7152 (399.74)   3,833.3034 (439.66)    9.4133 (46.63)    3,831.1316 (443.26)   10.6226 (88.45)         2;0    0.2609 (0.00)          5           1
test_baseline_lots_of_files_j1                          4,219.0481 (494.79)   4,264.3918 (443.14)   4,244.1330 (486.78)   21.5858 (106.93)   4,253.3812 (492.12)   39.9699 (332.80)        1;0    0.2356 (0.00)          5           1
test_baseline_lots_of_files_j1_empty_checker            4,227.6355 (495.79)   4,262.4767 (442.94)   4,245.6330 (486.96)   13.0950 (64.87)    4,244.7830 (491.12)   17.9307 (149.30)        2;0    0.2355 (0.00)          5           1
test_baseline_lots_of_files_j2                          4,503.3397 (528.13)   4,565.2388 (474.40)   4,539.3786 (520.65)   25.1924 (124.79)   4,549.1627 (526.34)   38.5980 (321.38)        2;0    0.2203 (0.00)          5           1
test_baseline_lots_of_files_j2_empty_checker            4,584.8548 (537.69)   4,626.2589 (480.75)   4,598.1414 (527.39)   16.9580 (84.00)    4,590.5582 (531.13)   20.9260 (174.24)        1;0    0.2175 (0.00)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
=============================================== short test summary info ===============================================
FAILED tests/test_functional.py::test_functional[symlink_module1] - AssertionError: Wrong results for file "symlink_m...
FAILED tests/lint/test_utils.py::test_prepare_crash_report - assert 'raise Exception(exception_content)' in 'First, p...
FAILED tests/pyreverse/test_main.py::test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests] - AssertionError: as...
FAILED tests/pyreverse/test_main.py::test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests\\data] - AssertionErr...
========================= 4 failed, 1656 passed, 274 skipped, 6 xfailed in 338.25s (0:05:38) ==========================

Summary

One test fails apparently because of symlinks.
Three tests fail due to the case of the C drive: C:\ vs c:\.

DanielNoord · 2022-05-06T12:20:09Z

Those failures due to C: and c: are interesting.

Apparently sys.path uses an uppercase whereas according to pytest your C drive is lowercase (see names before each test). I found python/cpython#84548, which is probably related to this issue. They consider this an edge-case, but I feel like this might be related to the issue you're seeing here...
Could you check if indeed realpath does indeed mangle your drive letter?

edreamleo · 2022-05-06T12:29:31Z

@DanielNoord Here's what I see:

C:\Repos\leo-editor>C:\Python\Python3.10\python.exe
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;os.path.realpath('c:/')
'C:\\'
>>> os.path.realpath('C:/')
'C:\\'
>>> os.path.realpath('c:\\')
'C:\\'
>>> os.path.realpath('C:\\')
'C:\\'

edreamleo · 2022-05-06T12:36:59Z

@DanielNoord Hah!

>>> sys.path
['',
'C:\\Python\\Python3.10\\python310.zip',
'C:\\Python\\Python3.10\\DLLs',
'C:\\Python\\Python3.10\\lib',
'C:\\Python\\Python3.10',
'C:\\Python\\Python3.10\\lib\\site-packages',
'c:\\repos\\ekr-pylint'    <====
]

I'm checking to see where the last line comes from.

edreamleo · 2022-05-06T13:16:19Z

@DanielNoord

The culprit is C:\Python\Python3.10\Lib\site-packages\easy-install.pth. This file contains one line:
c:\repos\ekr-pylint
Iirc, pylint installs easy-install.pth.

No other possibilities seem likely:

A search in c:\Users did not find anything suspicious.
There are no sitecustomize.py files in C:\Python (For Dev user).
The Dev user has not changed the Windows systems settings (including path) in any way.

Update: I checked sys.path (as shown in the previous comment) immediately after a restart of Windows. The "Edward Ream" user was never started.

DanielNoord · 2022-05-06T14:43:17Z

Hm, I'm not sure what easy-install does. A short google brought me to:
https://setuptools.pypa.io/en/latest/deprecated/easy_install.html.

Still, I think this is likely caused by the issue mentioned above and either pylint or easy-install using realpath where it preferably shouldn't. Do the tests run fine after a restart of Windows?

edreamleo · 2022-05-06T15:43:08Z

@DanielNoord

Tests: They fail as before. I would not have expected all tests to pass merely because of a restart.

easy-install.pth: Hmm. I expect the .pth file was installed as a consequence of installing pylint. I'll put it on the list to check when I do a fresh install. Could it somehow be a holdover? I don't think so, given that C:\Python\Python3.10 is all new. But we shall see.

realpath: Whatever realpath does, imo it would be reasonable (for unit tests only) to lowercase file names for the purposes of comparison (Windows only).

DanielNoord · 2022-05-06T17:27:37Z

easy-install.pth: Hmm. I expect the .pth file was installed as a consequence of installing pylint. I'll put it on the list to check when I do a fresh install. Could it somehow be a holdover? I don't think so, given that C:\Python\Python3.10 is all new. But we shall see.

.pth files come from setuptools if I am not mistaken. They could come from setuptools installing pylint but personally I have never seen one in my own environments.

realpath: Whatever realpath does, imo it would be reasonable (for unit tests only) to lowercase file names for the purposes of comparison (Windows only).

Well, I haven't had time to investigate this properly yet but I can imagine that these differences actually cause us problems as well. There is reason why we unitest these. So just normalising in the test doesn't seem like the immediate best approach imo.

edreamleo · 2022-05-06T17:39:50Z

@DanielNoord Update re easy-install.pth:

Summary

easy-install.pth must exist. Otherwise python won't be able to import pylint.
The case of easy-install.pth determines the case of the entry for pylint in sys.path.
Alas, unit tests fail as before even after "fixing" the file name's case in easy-install.pth.
This suggests that realpath may be part of the problem, as you, Daniel, have said.

Details

easy-install.pth must exist, regardless of the case of the filename. Otherwise python -m pylint will fail.
"Fixing" the case of the .pth file changes the case of the file in sys.path. After changing the .pth file to:

# c:\repos\ekr-pylint
C:\Repos\ekr-pylint

python 3.10 reports:

Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for z in sys.path: print(z)
...

C:\Python\Python3.10\python310.zip
C:\Python\Python3.10\DLLs
C:\Python\Python3.10\lib
C:\Python\Python3.10
C:\Python\Python3.10\lib\site-packages
C:\Repos\ekr-pylint  <=== Now the case is as expected for c:\ and Repos

Alas, four unit tests still fail as before:

====================================================== FAILURES =======================================================
__________________________________________ test_functional[symlink_module1] ___________________________________________

self = <pylint.testutils.lint_module_test.LintModuleTest object at 0x00000262390F31C0>

    def runTest(self) -> None:
>       self._runTest()
E       AssertionError: Wrong results for file "symlink_module":
E
E       Unexpected in testdata:
E          1: syntax-error

pylint\testutils\lint_module_test.py:111: AssertionError
______________________________________________ test_prepare_crash_report ______________________________________________

tmp_path = WindowsPath('C:/Users/Dev/AppData/Local/Temp/pytest-of-Dev/pytest-6/test_prepare_crash_report0')

>   ???
E   assert 'raise Exception(exception_content)' in 'First, please verify that the bug is not already filled:\nhttps://github.com/PyCQA/pylint/issues/\n\nThen create a ne...\leo.repo\\ekr-pylint\\tests\\lint\\test_utils.py", line 21, in test_prepare_crash_report\nException: Exmessage\n```\n'

c:\leo.repo\ekr-pylint\tests\lint\test_utils.py:32: AssertionError
_____________________________ test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests] _____________________________

>   ???
E   AssertionError: assert ['C:\\Repos\\...ylint\\tests'] == ['c:\\Repos\\...ylint\\tests']
E     At index 0 diff: 'C:\\Repos\\ekr-pylint\\tests' != 'c:\\Repos\\ekr-pylint\\tests'
E     Use -v to get more diff

c:\leo.repo\ekr-pylint\tests\pyreverse\test_main.py:63: AssertionError
__________________________ test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests\\data] __________________________

>   ???
E   AssertionError: assert ['C:\\Repos\\...ylint\\tests'] == ['c:\\Repos\\...ylint\\tests']
E     At index 0 diff: 'C:\\Repos\\ekr-pylint\\tests' != 'c:\\Repos\\ekr-pylint\\tests'
E     Use -v to get more diff

c:\leo.repo\ekr-pylint\tests\pyreverse\test_main.py:63: AssertionError

[Benchmark data omitted]

=============================================== short test summary info ===============================================
FAILED tests/test_functional.py::test_functional[symlink_module1] - AssertionError: Wrong results for file "symlink_m...
FAILED tests/lint/test_utils.py::test_prepare_crash_report - assert 'raise Exception(exception_content)' in 'First, p...
FAILED tests/pyreverse/test_main.py::test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests] - AssertionError: as...
FAILED tests/pyreverse/test_main.py::test_project_root_in_sys_path[c:\\Repos\\ekr-pylint\\tests\\data] - AssertionErr...
========================= 4 failed, 1656 passed, 274 skipped, 6 xfailed in 332.77s (0:05:32) ==========================

DanielNoord · 2022-05-06T17:44:07Z

The first failure is still related to symlinks. You might want to re-clone pylint. When you clone a project without symlinks enabled I think the symlinks break and will need to be recloned in order to work.

For the others, I would need more time. I'd say for now you can just let those fail and see if the CI on GitHub does pass. The drive-case stuff is strange but I don't think we have ever seen any issues with it being reported so it would not be very high on my todo-list for now.

edreamleo · 2022-05-06T21:12:44Z

@DanielNoord Yes, fudging the unit tests relating to realpath might be bad idea. And fudging the tests might be an even worse idea for this issue (#6476).

I am not sure that this issue is worth fixing at all. A (general?) solution like hacking the tmp_path fixture is way beyond me :-)

I'll do as you suggest and re-clone pylint and let the other tests fail for now.

Many thanks for all your help. This has been a pleasant experience! Next up: installing Python 3.10.4 on Ubuntu.

DanielNoord · 2022-05-06T21:14:12Z

Glad to help! You also helped put this issue on my radar, if we ever run into this in the wild I will now know where to look for potential sources 👍

clavedeluna · 2022-12-02T12:47:09Z

@edreamleo Is there an action item to do for this issue, or can we close it?

edreamleo · 2022-12-02T20:14:46Z

@clavedeluna I had forgotten all about this issue :-) Feel free to close it.

jacobtylerwalls · 2023-06-07T13:12:41Z

I think we can refocus this on the issue and suggested patch identified here. Should be a bite-sized contribution for someone.

edreamleo · 2023-06-07T13:57:38Z

@jacobtylerwalls I'll take a nibble sometime soon.

jacobtylerwalls · 2023-06-07T14:57:18Z

Oops! epylint was removed in #7737. Sorry. Save your appetite for nibble on something else!

edreamleo · 2023-06-07T18:39:49Z

@jacobtylerwalls :-)

edreamleo added the Needs triage 📥 Just created, needs acknowledgment, triage, and proper labelling label Apr 29, 2022

jacobtylerwalls changed the title ~~test_epylint.py fails~~ test_epylint.py fails when the temp dir used has a space Apr 30, 2022

jacobtylerwalls added Bug 🪲 Minor 💅 Polishing pylint is always nice tests Good first issue Friendly and approachable by new contributors and removed Needs triage 📥 Just created, needs acknowledgment, triage, and proper labelling labels Apr 30, 2022

Pierre-Sassoulas added this to the 2.15.0 milestone May 13, 2022

edreamleo mentioned this issue May 28, 2022

Issue with lower case drive names when running tests on Windows python/mypy#12880

Open

DanielNoord modified the milestones: 2.15.0, 2.16.0 Aug 23, 2022

Pierre-Sassoulas added the Hacktoberfest label Sep 15, 2022

clavedeluna added Waiting on author Indicate that maintainers are waiting for a message of the author and removed Good first issue Friendly and approachable by new contributors Hacktoberfest labels Dec 2, 2022

clavedeluna closed this as completed Dec 2, 2022

jacobtylerwalls reopened this Jun 7, 2023

jacobtylerwalls added Good first issue Friendly and approachable by new contributors and removed Waiting on author Indicate that maintainers are waiting for a message of the author labels Jun 7, 2023

jacobtylerwalls removed this from the 2.16.0 milestone Jun 7, 2023

jacobtylerwalls changed the title ~~test_epylint.py fails when the temp dir used has a space~~ test_epylint.py fails when invoked from a directory with a space in its name Jun 7, 2023

jacobtylerwalls closed this as not planned Won't fix, can't repro, duplicate, stale Jun 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_epylint.py fails when invoked from a directory with a space in its name #6476

test_epylint.py fails when invoked from a directory with a space in its name #6476

edreamleo commented Apr 29, 2022 •

edited

Loading

jacobtylerwalls commented Apr 30, 2022

edreamleo commented Apr 30, 2022

edreamleo commented May 5, 2022 •

edited

Loading

DanielNoord commented May 5, 2022

edreamleo commented May 5, 2022

edreamleo commented May 5, 2022 •

edited by DanielNoord

Loading

DanielNoord commented May 5, 2022

edreamleo commented May 5, 2022

DanielNoord commented May 5, 2022

edreamleo commented May 6, 2022 •

edited

Loading

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022 •

edited

Loading

edreamleo commented May 6, 2022

edreamleo commented May 6, 2022 •

edited

Loading

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022

DanielNoord commented May 6, 2022

clavedeluna commented Dec 2, 2022

edreamleo commented Dec 2, 2022

jacobtylerwalls commented Jun 7, 2023

edreamleo commented Jun 7, 2023

jacobtylerwalls commented Jun 7, 2023

edreamleo commented Jun 7, 2023

test_epylint.py fails when invoked from a directory with a space in its name #6476

test_epylint.py fails when invoked from a directory with a space in its name #6476

Comments

edreamleo commented Apr 29, 2022 • edited Loading

Bug description

Configuration

Command used

Pylint output

Expected behavior

Pylint version

OS / Environment

Additional dependencies

jacobtylerwalls commented Apr 30, 2022

edreamleo commented Apr 30, 2022

edreamleo commented May 5, 2022 • edited Loading

DanielNoord commented May 5, 2022

edreamleo commented May 5, 2022

edreamleo commented May 5, 2022 • edited by DanielNoord Loading

DanielNoord commented May 5, 2022

edreamleo commented May 5, 2022

DanielNoord commented May 5, 2022

edreamleo commented May 6, 2022 • edited Loading

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022 • edited Loading

edreamleo commented May 6, 2022

edreamleo commented May 6, 2022 • edited Loading

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022

DanielNoord commented May 6, 2022

edreamleo commented May 6, 2022

DanielNoord commented May 6, 2022

clavedeluna commented Dec 2, 2022

edreamleo commented Dec 2, 2022

jacobtylerwalls commented Jun 7, 2023

edreamleo commented Jun 7, 2023

jacobtylerwalls commented Jun 7, 2023

edreamleo commented Jun 7, 2023

edreamleo commented Apr 29, 2022 •

edited

Loading

edreamleo commented May 5, 2022 •

edited

Loading

edreamleo commented May 5, 2022 •

edited by DanielNoord

Loading

edreamleo commented May 6, 2022 •

edited

Loading

edreamleo commented May 6, 2022 •

edited

Loading

edreamleo commented May 6, 2022 •

edited

Loading