Bug in RustNotify causing RuntimeError: Already borrowed on exit/signal

[07pepa]

Description

i see this tracebacks/exception when terminating my aplication in docker or windows/ python 3.10 and 3.11

(can be trigered from cli (ctrl+c) or pycharm (stoping)

i tried to set my stopevent in signal hander but without any progress

traceback in output section

Example Code

import asyncio
import signal
import requests
from watchfiles import awatch


class SettingMonitor:
    def __init__(self):
        self.stop = asyncio.Event()

    def _hookup_signal_handler(self):
        sig_to_str = {
            signal.SIGINT: "Interrupt",
            signal.SIGTERM: "Terminate",
            signal.SIGABRT: "Abort",
            signal.SIGILL: "Illegal instruction",
            signal.SIGSEGV: "Segmentation fault"
        }

        for sig in sig_to_str.keys():
            def mk_handler():
                current = signal.getsignal(sig)

                def handler(sig_num, frame):
                    self.stop.set()
                    print(f"handling signal "
                          f"{sig_to_str[sig_num] if sig_num in sig_to_str else 'unknown' + sig_num} ")

                    current(sig_num, frame)

                return handler

            signal.signal(sig, mk_handler())

    async def start(self):
        self._hookup_signal_handler()# comenting this line makes no diference
        async for _ in awatch("/", stop_event=self.stop):#"/" path is just example
            pass


async def main():
    await SettingMonitor().start()


if __name__ == "__main__":
    asyncio.run(main())

Watchfiles Output

direct output from console:


handling signal Interrupt
Traceback (most recent call last):
  File "<REDACTED>", in <module>
    asyncio.run(main())
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 650, in run_until_complete
handling signal Interrupt 
    return future.result()
           ^^^^^^^^^^^^^^^
  File "<REDACTED>", in main
    await SettingMonitor().start()
  File "<REDACTED>, in start
    async for _ in awatch("/", stop_event=self.stop):
  File "C:\Program Files\Python311\Lib\site-packages\watchfiles\main.py", line 238, in awatch
    with RustNotify([str(p) for p in paths], debug, force_polling, poll_delay_ms, recursive) as watcher:
RuntimeError: Already borrowed

Operating System & Architecture

Windows-10-10.0.19045-SP0
10.0.19045

and docker FROM python:3.10-slim

Environment

windows (native and docker) , an docker on raspberi pi 4 (usual install via pip)

Python & Watchfiles Version

python: 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)], watchfiles: 0.18.0

Rust & Cargo Version

cargo 1.64.0 (387270bc7 2022-09-16); rustc 1.64.0 (a55dd71d5 2022-09-19)

[samuelcolvin]

Looks like a duplicate of PyO3/pyo3#2525, best to post there and they might be able to help.

I haven't been able to reproduce this error.

[samuelcolvin]

Which version of 3.10 were you using?

[07pepa]

[Python 3.10.8] probably but definitely 3.11.0

[justvanrossum]

FWIW, I can reproduce this with Python 3.10.9 on macOS 10.15, with watchfiles 0.18.1, with the following script:

import asyncio
import watchfiles


async def watcher():
    async for changes in watchfiles.awatch("."):
        print(changes)
        break


async def cancel(task):
    await asyncio.sleep(0.5)
    print("cancelling")
    task.cancel()


async def main():
    watcherTask = asyncio.create_task(watcher())
    cancelTask = asyncio.create_task(cancel(watcherTask))
    await watcherTask
    await cancelTask


asyncio.run(main())

The output is this:

cancelling
Traceback (most recent call last):
  File "/Users/just/code/git/BlackFoundry/fontra/alread_borrowed.py", line 24, in <module>
    asyncio.run(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/Users/just/code/git/BlackFoundry/fontra/alread_borrowed.py", line 20, in main
    await watcherTask
  File "/Users/just/code/git/BlackFoundry/fontra/alread_borrowed.py", line 6, in watcher
    async for changes in watchfiles.awatch("."):
  File "/Users/just/code/git/BlackFoundry/fontra/venv/lib/python3.10/site-packages/watchfiles/main.py", line 238, in awatch
    with RustNotify([str(p) for p in paths], debug, force_polling, poll_delay_ms, recursive) as watcher:
RuntimeError: Already borrowed

[samuelcolvin]

Yup, this has been demonstrated, see PyO3/pyo3#2525 (comment).

Fix welcome.