Coverage 6.2, coverage.exceptions.DataError no such table: file

[HansBug]

Describe the bug

As the title said, this bug appears in my project's action.

To Reproduce

This problem has been seen in python 3.8 once (in the previous attempt of the link above), and in python 3.9 in the latest attempt.
There are many manual forks and pipes in my code, so I guess it may have something to do with these (in fact, I haven't seen such errors in other projects, and I use version 6.2 coverage). It can not be stably reproduced, and only seen in the github action. I have never seen this error in my local environment. The error message is like this

test/utils/test_units.py::TestUtilsUnits::test_time_to_delta_str PASSED
test/utils/test_value.py::TestUtilsValue::test_value_proxy_init PASSED
test/utils/test_value.py::TestUtilsValue::test_value_proxy_set PASSED
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/sqldata.py", line 1100, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.OperationalError: no such table: file
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/sqldata.py", line 1105, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.OperationalError: no such table: file
INTERNALERROR> 
INTERNALERROR> The above exception was the direct cause of the following exception:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_cov/plugin.py", line 294, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_cov/engine.py", line 230, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/control.py", line 722, in combine
INTERNALERROR>     combine_parallel_data(
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/data.py", line 131, in combine_parallel_data
INTERNALERROR>     data.update(new_data, aliases=aliases)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/sqldata.py", line 659, in update
INTERNALERROR>     this_tracers = {path: "" for path, in conn.execute("select path from file")}
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/sqldata.py", line 1122, in execute
INTERNALERROR>     raise DataError(f"Couldn't use data file {self.filename!r}: {msg}") from exc
INTERNALERROR> coverage.exceptions.DataError: Couldn't use data file '/__w/pji/pji/.coverage': no such table: file

======================= 319 passed, 1 warning in 45.06s ========================
make: *** [Makefile:17: unittest] Error 3
Error: Process completed with exit code 2.

Expected behavior
Well I think it should be ended up properly like the other jobs.

Additional context
How to run my project.

After clone it, you may run

make run_dev

to enter a docker-based system (This step is necessary because this project need root permission). And then install the python dependencies

pip install -r requirements.txt
pip install -r requirements-test.txt

Finally, perform unit tests

make unittest

[nedbat]

If it only happens in GitHub Actions, I'm not sure how to debug it... Any ideas?

[HansBug]

Well, I have just found this error in my local environment (but not stably reproduce), it is exactly the same as the traceback above.

It has now been confirmed that this problem can be reproduced in the local environment, with the docker environment opened by make run_dev.

[HansBug]

After repeated testing in the docker environment just now, the above error occurs about once every 20 runs.

My local OS is ubuntu 20.04, docker version is

Client:
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.8
 Git commit:        20.10.7-0ubuntu5~20.04.2
 Built:             Mon Nov  1 00:34:17 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.8
  Git commit:       20.10.7-0ubuntu5~20.04.2
  Built:            Fri Oct 22 00:45:53 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.5-0ubuntu3~20.04.1
  GitCommit:        
 runc:
  Version:          1.0.1-0ubuntu2~20.04.1
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:        

The python in my docker environment is python3.6.15, and all of the pip installations are

attrs==21.4.0
bitmath==1.3.3.1
chardet==4.0.0
click==8.0.3
colorama==0.4.4
coverage==6.2
execnet==1.9.0
flake8==3.9.2
hbutils==0.0.1
importlib-metadata==4.8.3
iniconfig==1.1.1
itermate==1.0.2
mccabe==0.6.1
mock==4.0.3
packaging==21.3
pluggy==1.0.0
py==1.11.0
pycodestyle==2.7.0
pyflakes==2.3.1
pyparsing==3.0.6
pysyslimit==0.5.0
pytest==6.2.5
pytest-cov==3.0.0
pytest-forked==1.4.0
pytest-mock==3.6.1
pytest-rerunfailures==10.2
pytest-timeout==2.0.2
pytest-xdist==2.5.0
pytimeparse==1.1.8
PyYAML==6.0
toml==0.10.2
tomli==1.2.3
typing_extensions==4.0.1
where==1.0.2
zipp==3.6.0

And I found that this error has never been seen in many other of my projects (some of them have been maintained for long time), these projects have no manual forks and pipes, even have no subprocesses called. Therefore, I think the error here is most likely related to the child process and other process related operations.

[nedbat]

Thanks for the Docker reproduction. It seems to fail for me at least half the time, so it's a good demonstration of the problem. I think this is a straightforward race condition in the creation of the database. Let me see what I can do.

[nedbat]

It turns out this is not a race condition in the creation of the databases. It's a race condition caused by pytest-cov, which is both erasing the data and starting tests at the same time.

I have a fix for coverage so that it doesn't fail running SQL statements: it's on the nedbat/fix-1303 branch. You can install it with:

pip install git+https://github.com/nedbat/coveragepy.git@nedbat/fix-1303

When I run your tests with that branch of coverage, there are no failures. You are using Python 3.6, so this branch is based on the 6.2 release. But master of coverage.py has dropped 3.6 support, so when I release this for real, it won't run on 3.6. I don't know how much of a problem that will be for you.

One thing I noticed was that the final coverage stats can vary from test run to test run, and this happens both with the fix, and with 6.2. The number of statements is always 3179 but the number of missing statements can be 70 or 107. The statements that come and go in the coverage are those run by test_apply_1 and test_apply_2, though I'm not sure the particular tests are significant. I think variance is due to the race condition in pytest-cov.

[nedbat]

@ionelmc /cc about the race condition in pytest-cov. I can elaborate.

[HansBug]

OK, I have the situation.
In view of the actual situation of the code here, I'm also going to try whether removing dependencies such as pytest-xdist will be effective to further determine the problem.

[nedbat]

One option is to not use pytest-cov, though it means you have to manage the subprocess measurement yourself: https://coverage.readthedocs.io/en/6.2/subprocess.html

[nedbat]

This is now fixed in commit b41be3f

[nedbat]

This is now released as part of coverage 6.3.

[HansBug]

Sorry for bothering you again, but the bug has appeared again in coverage version 6.3, this time with Python 3.10.

This is my CI log, which is exactly the same as the abovementioned error: https://github.com/HansBug/pji/runs/4996123439?check_suite_focus=true

-------------------- supplement ----------------------------

Again, only in Python 3.10: https://github.com/HansBug/pji/runs/4996182427?check_suite_focus=true

[pietrodn]

This is happening to me on Python 3.9, on an Intel Mac, with coverage==6.3.1.

Full tracebacks here.

[zpz]

I had a very similar issue and solved it following this pytest page https://pytest-cov.readthedocs.io/en/latest/subprocess-support.html

[jvavrek]

I also see this on Python 3.9 with coverage==6.3.2

[timozerrer]

This issue still persists with coverage==6.4.3 on Python 3.8

[hmcezar]

I'm also having this issue with Python 3.8. The log is available here:

https://gist.github.com/hmcezar/96674132c64a20304070dbcb5a4c7d36

It happened when running tests for this PR:
Cascella-Group-UiO/HyMD#181

[mgoldbas]

I am facing this issue as well using a local version of make triggered by pre-commit

[jonathankehler]

Seeing this in python 3.8 with coverage==6.4.4

[nedbat]

If you are seeing this error, please provide a way for me to reproduce it. What repo should I clone, and what commands should I run?

[hmcezar]

If you are seeing this error, please provide a way for me to reproduce it. What repo should I clone, and what commands should I run?

@nedbat from my understanding, this is not something you can consistently reproduce. If you want to see one of the cases in which it happened, please check the log of the following CI action:
https://github.com/Cascella-Group-UiO/HyMD/actions/runs/2913278694/jobs/4640884182

This happened in one of the commits (5890a80) of the following PR:
Cascella-Group-UiO/HyMD#181

[hmcezar]

Just an update, the bug happened again in my GitHub actions (see repository https://github.com/Cascella-Group-UiO/HyMD) and the build is "failing" due to the bug.
I checked out the code that did not pass in the main branch and tried running the tests locally, but even after ~20 runs, I didn't manage to reproduce the bug.
I really think that it happens more often (not to say, only) in GitHub actions than it happens locally.

[Pablongo24]

This is also happening to me.

Python==3.12
coverage==7.4.4

Any solutions?

[nedbat]

@Pablongo24 if you can provide a way to reproduce the problem, we can dig into finding a solution.

[germa89]

@nedbat

I'm getting this issue in here:

• run: https://github.com/ansys/pymapdl/actions/runs/9972934231/job/27557400671?pr=3239#step:17:258
• PR: feat: refactoring create_temp_dir ansys/pymapdl#3239

(I hope you can see that)

In my case, it was tempfile.NamedTemporaryFile object which generates the issues. Not sure how those three are related (pytest, coverage and tempfile.NamedTemporaryFile). But I'm dropping this here, in case someone finds it useful.