FileNotFoundError on files on reload-exclude

[Kludex]

When I use --reload-exclude, or another directory is supposed to be watched, I still have the following:

Running development server on 0.0.0.0:8000
INFO:     Will watch for changes in these directories: ['/app/src']
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [7] using WatchFiles
Traceback (most recent call last):
  File "/opt/app-env/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/app-env/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/main.py", line 408, in main
    run(
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/main.py", line 571, in run
    ChangeReload(config, target=server.run, sockets=[sock]).run()
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/basereload.py", line 45, in run
    for changes in self:
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/basereload.py", line 64, in __next__
    return self.should_restart()
  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/watchfilesreload.py", line 85, in should_restart
    changes = next(self.watcher)
  File "/opt/app-env/lib/python3.8/site-packages/watchfiles/main.py", line 119, in watch
    with RustNotify([str(p) for p in paths], debug, force_polling, poll_delay_ms, recursive) as watcher:
FileNotFoundError: Permission denied (os error 13) about ["/app/data"]

The command used was:

uvicorn --reload --reload-dir /app/src main:app

I guess it's because we don't really exclude, we just check the path to see if we should reload.

cc @samuelcolvin

Important

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[samuelcolvin]

yes, notify, and therefore watchfiles doesn't allow you to exclude a directory, just select multiple directories to watch.

[Kludex]

If I wrap the following line on except FileNotFoundError:

  File "/opt/app-env/lib/python3.8/site-packages/uvicorn/supervisors/watchfilesreload.py", line 85, in should_restart
    changes = next(self.watcher)

Would it be a bad idea? How do you think we should solve this? 🤔

[samuelcolvin]

I thought about that, but I think the rust code will exit early if it finds an error, so you might miss changes. I haven't checked, that's just my guess.

[samuelcolvin]

you might also ignore other errors, not just permission denied.

[Kludex]

Can I interpret the above as "on uvicorn, we should instead wait for samuelcolvin/watchfiles#147, and then change the behavior to 'ignore these errors'"?

[samuelcolvin]

yes.

[Kludex]

Thanks 🙏

[AdamMusa]

I resolved this problem by typing this command in my terminal
sudo sysctl -w fs.inotify.max_user_watches=457856

[aminalaee]

@Kludex I have merged samuelcolvin/watchfiles#147
Should we just add a new flag ignore-permission-denied here? Or do we want to implicitly enable that when we have reload_excludes? If we can decide it I will continue the work.

[samuelcolvin]

You can just set the env variable. I think that's enough for now.

[Yasamato]

@samuelcolvin could you refer me to which env variable you are talking?

I am currently experiencing the same problem of a sub directory having restricted permissions and crashing the --reload functionality.

For replication:

pip install uvicorn[standard]

docker-compose.yml

version: '3'
services:
  db:
    image: mariadb
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: mydatabase
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    volumes:
      - ./db:/var/lib/mysql

and start with docker compose up -d creating the folder ./db. Now we have some python web app in ./api and run it via uvicorn:

uvicorn api.main:app --reload  --port 3000 --reload-dir api --reload-exclude 'db/**/*'

But this results in original posted permissions denied error and the functionality of hot reloading not working.

[tsdaemon]

A similar problem happen when watchfile fails to find a file: _rust_notify.WatchfilesRustInternalError: error in underlying watcher: IO error for operation on %CWD%/.web/nocompile: No such file or directory (os error 2) about ["%CWD%/.web/nocompile"]. .web is a folder which is not in reload-dir and is explicitly excluded. This is for a reflex app which uses uvicorn under the hood.

I am not entirely sure whether watching ALL files and then ignore is a reasonable solution. Took me quite some time wondering how is it even possible that error comes from a folder which definitely should be ignored until I looked into the uvicorn code. I am also not sure how good is it for performance: a project may have a lot of extra py files in CWD, for example tests.