-
-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Read/Write File Lock #380
base: main
Are you sure you want to change the base?
Read/Write File Lock #380
Conversation
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
@gaborbernat This is ready for review now. Happy to iterate on the API if needed (especially the Wrapper part, I think it's a bit clunky, though perhaps all it needs is just a rename). |
The CI is failing. |
Thanks, will fix the type issues. Aside from that, I see the Windows tests are failing with
Considering there is no Windows implementation here, can you advise on how to deal with this? |
I see a lot of markers to ignore typing errors. This is not the right approach. We should make the typing correct, not ignore the errors generated. This is the same for documentation, links that you nitpick ignored. |
Thanks - I'm still iterating on this. Some of the ignores are copied/follow patterns as in existing code. Will mark as draft until I am done. |
In general the current way the async APIs are implemented necessitates ignores. I would like to avoid breaking changes, but it seems quite challenging to fix that otherwise (and I wouldn't want to do this in this PR anyway). |
I think I managed to get rid of most of the added type ignores (and the sphinx nitpick ignores). The ones that remain + linter noqas should be consistent with the rest of the codebase. |
Thank you, heads up that I will be unavailable until the new year, so I will not be able to accept this change request until then. |
This PR adds a
ReadWriteFileLock
API and sync & async implementation (currently only for Unix. I don't know if this is possible on Windows).ReadWriteFileLock
allows the caller to operate in either READ or WRITE mode. The lock can be held by multiple READers at the same time, but a WRITEr is guaranteed to have exclusive access to the lock (across both READers and WRITErs). The lock is writer-preferred by employing an outer and inner lock (with the outer serving as staging for acquiring the inner - this means the outer lock is much easier to acquire).In reader case:
In writer case:
While this seems to be working well in practice, there are no actual guarantees of ordering and priority.
I am not firmly settled on the API. Other idea would be to roll wrapper functionality into the base class and provide
write_acquire
andread_acquire
functions (though the context managers would get a bit trickier - one would need to dowith self.read_write_lock.read():
instead of simplyself.read_write_lock:
- as this doesn't allow specifying the mode)Closes #307