-
-
Notifications
You must be signed in to change notification settings - Fork 951
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generic ExceptionHandler type #2403
base: master
Are you sure you want to change the base?
Conversation
starlette/applications.py
Outdated
@@ -139,8 +140,8 @@ def add_middleware( | |||
|
|||
def add_exception_handler( | |||
self, | |||
exc_class_or_status_code: int | typing.Type[Exception], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NB: this has now been tightened so that the Exception for exc_class_or_status_code
must match the Exception in the handler Callable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Kludex isn't this a deprecated function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. We never deprecated add_exception_handler
... 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, to be more precise, the Exception in the handler Callable is still contravariant, so this passes as one would hope:
def fallback_exception_handler(request: Request, exc: Exception) -> JSONResponse:
...
app.add_exception_handler(Exception, fallback_exception_handler)
app.add_exception_handler(MyException, fallback_exception_handler)
ie: the ExceptionType
TypeVar changes the handler Callable Exception
to a type parameter. The value of this type parameter is provided at the call site, eg: in the example here its Exception
in the first call, and MyException
in the second. But the contravariance of Callable remains.
And also the Callable is still not covariant, so this fails as one might expect:
def my_exception_handler(request: Request, exc: MyException) -> JSONResponse:
...
# this fails, because subtypes of the exc_class_or_status_code Exception in the Callable
# aren't permitted ie: Callable is not covariant in the function's input parameter types
app.add_exception_handler(Exception, my_exception_handler)
Can you remove the trailing commas, please? |
Hi @Kludex these were added by |
This reverts commit 7cecce7.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is nice, btw 👍
Thanks.
starlette/types.py
Outdated
@@ -21,10 +21,10 @@ | |||
] | |||
Lifespan = typing.Union[StatelessLifespan[AppType], StatefulLifespan[AppType]] | |||
|
|||
E = typing.TypeVar("E", bound=Exception, contravariant=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we call it ExceptionType
?
…xception-handler-typing
Does this catch the error cases? from starlette.applications import Request, Starlette
from starlette.responses import JSONResponse
class MyException(Exception):
def __init__(self, message: str) -> None:
self.message = message
self.extra = "extra"
def my_exception_handler(request: Request, exc: MyException) -> JSONResponse:
return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)
app = Starlette(debug=True, routes=[])
app.add_exception_handler(Exception, my_exception_handler) # should error
handlers = {Exception: my_exception_handler}
app = Starlette(debug=True, routes=[], exception_handlers=handlers) # should error |
Yes your example errors because of the tighter bound mentioned above, eg:
|
For the record, this has been attempted before and the conclusion at the time was that typing this correctly is not possible: #2048 I didn't look in detail but I would suggest we understand why it is possible now when it wasn't a couple months ago, or what this PR does differently. |
What about if you have multiple types? |
Yes that will pass type checks. I haven't changed the dictionary key which is still Thank you btw for pointing me at the previous attempt, I hadn't seen that. This PR differs by not introducing a new class for the handler function. Instead it makes Exception a type parameter so it can vary. This allows handlers to provide a more specific type, so long as it's an Exception subtype. This overcomes the inherent contravariance in Callable type parameters, which currently prevents subtypes of Exception being accepted by the type checker. |
Adding a generic exception handler should be relatively easy, or am I missing something? import typing
from starlette.requests import Request
from starlette.responses import Response
from starlette.websockets import WebSocket
TException = typing.TypeVar("TException")
HTTPExceptionHandler = typing.Callable[
["Request", TException], typing.Union["Response", typing.Awaitable["Response"]]
]
WebSocketExceptionHandler = typing.Callable[
["WebSocket", TException], typing.Awaitable[None]
]
ExceptionHandler = typing.Union[HTTPExceptionHandler[TException], WebSocketExceptionHandler[TException]]
def add_exception_handler(
exc_class_or_status_code: int | typing.Type[TException],
handler: ExceptionHandler[TException],
) -> None:
pass
def value_error_handler(request: Request, exc: ValueError) -> Response:
return Response()
def exc_handler(request: Request, exc: Exception) -> Response:
return Response()
add_exception_handler(Exception, value_error_handler) # Error, value_error_handler should accept `Exception`
add_exception_handler(ValueError, value_error_handler) # Ok
add_exception_handler(Exception, exc_handler) # Ok
add_exception_handler(TypeError, exc_handler) # Ok |
Hi @ThirVondukr your snippet is essentially what the PR implements. The one difference is the PR's |
There's an even bigger discussion on #1456. |
So... Does this test summarize what we need here? def test_types() -> None:
class MyException(Exception):
def __init__(self, message: str) -> None:
self.message = message
self.extra = "extra"
def my_exc_handler(request: Request, exc: MyException) -> JSONResponse:
return JSONResponse(
{"detail": exc.message, "extra": exc.extra}, status_code=400
)
app = Starlette(debug=True, routes=[])
# should error (type ignore is needed)
app.add_exception_handler(Exception, my_exc_handler) # type: ignore[arg-type]
# doesn't error
app.add_exception_handler(MyException, my_exc_handler)
# should error (type ignore is needed)
handlers = {Exception: my_exc_handler}
Starlette(debug=True, routes=[], exception_handlers=handlers) # type: ignore[arg-type]
# doesn't error
my_exception_handlers = {MyException: my_exc_handler}
Starlette(debug=True, routes=[], exception_handlers=my_exception_handlers)
# should error (type ignore is needed)
multiple_handlers = {MyException: my_exc_handler, Exception: my_exc_handler}
Starlette(debug=True, routes=[], exception_handlers=multiple_handlers) # type: ignore[arg-type]
# doesn't error
Starlette(exception_handlers={MyException: my_exc_handler})
# should error (type ignore is needed)
Starlette(exception_handlers={Exception: my_exc_handler}) # type: ignore[arg-type] |
Can we just have the |
Hi @Kludex, Thanks for taking a look again!
By this do you mean don't worry about the cases in 1. or update the PR to no longer address the case in 2. ? Whilst it would be possible to update the PR to preserve the current behaviour of |
…xception-handler-typing
Is there any traction on this fix? This is embarassing for Starlette/FastAPI users using mypy |
After merging in master again, the tests are failing on this PR (and other PRs too) because of a change unrelated to this PR:
|
I am also having this issue. Is there any more work to be done? I´d be happy to help |
I need to read the reply and review again. |
exception_handlers: typing.Mapping[typing.Any, ExceptionHandler[ExceptionType]] | None = None, I think the fundamental problem identified in the previous attemps is that for the e.g. for these handlers exception_handlers = {
MyException: my_exception_handler,
404: my_404_handler,
} the value of each mapping key defines the allowed type of each handler. You can't define a (You can define a |
One thing is to fully validate with the type system, one other thing is to make it go through type validation. The minimum is to allow valid exception handlers to not fail mypy checks and this should be a top-priority for Starlette maintainers for me. The quality of type annotations can be improved as much as possible over time but this is less urgent for me. |
On the Does someone want to propose an alternative API for exception handlers? |
Just to make sure I understand correctly, could you share the code for the failing example you mention here? |
…xception-handler-typing
from starlette.applications import Request, Starlette
from starlette.responses import JSONResponse
def test_types() -> None:
class MyException(Exception):
def __init__(self, message: str) -> None:
self.message = message
self.extra = "extra"
class PotatoException(Exception):
def __init__(self, message: str) -> None:
self.message = message
def my_exc_handler(request: Request, exc: MyException) -> JSONResponse:
return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)
def my_exc_handler2(request: Request, exc: PotatoException) -> JSONResponse:
return JSONResponse({"detail": exc.message}, status_code=400)
handlers = {Exception: my_exc_handler, PotatoException: my_exc_handler2}
Starlette(debug=True, routes=[], exception_handlers=handlers)
multiple_handlers = {MyException: my_exc_handler, Exception: my_exc_handler, PotatoException: my_exc_handler2}
Starlette(debug=True, routes=[], exception_handlers=multiple_handlers)
Starlette(exception_handlers={Exception: my_exc_handler, PotatoException: my_exc_handler2}) Extending the code above. |
I've proposed https://mypy-play.net/?mypy=latest&python=3.11&gist=ace307dc705717b1ef1fedc21708b8f3 before. I think it's the only way to make it actually work for all cases, and it's backwards compatible. IMO we should do something like that and simplify the existing type hints to be very loose and accept all cases. |
Gotcha thanks.. so this in-lined dictionary case: Starlette(exception_handlers={Exception: my_exc_handler, PotatoException: my_exc_handler2}) Fails with:
This can be supported with overloads on |
This makes sense to me. Its more practical and simpler than overloads. See how it looks for the |
Summary
Allows callables that handle Exception subtypes to pass the type checker (mypy/pyright), eg: this will now correctly type check:
I think the cause of the problem is Callable is contravariant (for good reason) and so whilst MyException is a sub-type of Exception, the Callable is not. This PR uses a TypeVar bound to Exception to address this.
See this discussion comment.
Checklist