Support Request Cancellation, Timeouts via RequestController

[SamJakob]

Closes #204, closes #424, closes #567
Supersedes and closes #521, supersedes and closes #602, supersedes and closes #819

This PR implements an opt-in RequestController API to manage requests. The API is opt-in, future-proof and backwards-compatible both for implementations of Client and for end users. If a controller is not specified by the user, the existing functionality of the package is maintained. Likewise, clients can opt-in to the new API and show they are doing so by overriding supportsController to be true and implementing the functionality. Later functionality of a similar nature can also be added in an opt-in manner by simply adding it to the RequestController class.

See the full examples below, but a brief tour of the RequestController's functionality is as follows:

// Create a request controller, can be passed into multiple requests to
// cancel them all at once or to set timeouts on all of them at once.
// Or, one can be created per request.
final controller = RequestController(
  // Optional timeout for the overall request (entire round trip).
  timeout: Duration(seconds: 5),
  // Optional timeouts for each state of the request.
  // If a state is not specified, it will not timeout.
  partialTimeouts: PartialTimeouts(
    // Timeout for connecting to the server.
    connectTimeout: Duration(seconds: 5),
    // Timeout for the request body to be sent and a response to become
    // available from the server.
    sendTimeout: Duration(seconds: 4),
    // Timeout for processing the response body client-side.
    receiveTimeout: Duration(seconds: 6),
  ),
);

It can then be used as follows:

final response = await client.get(
  // URL that hangs for 5 seconds before responding.
  Uri.parse('http://localhost:3456/longrequest'),
  controller: controller,
);
// Unhandled exception:
// TimeoutException after 0:00:04.000000: Future not completed

// or,
controller.cancel(); // Causes request future to throw CancelledException

I note, from above, that there was existing work in both request cancellation and timeouts however I felt that this different approach could cleanly solve both issues with a single, more ergonomic, user-friendly and intuitive API. This work started originally as a private fork of the package to implement some functionality required for another package, however I felt that the results might be viable for inclusion in the upstream package.

---

Additionally, with respect to #819, I happened to implement this similarly to make it possible to abort a response of an IOClient (useful for applications that need to interrupt a response with a large payload). For this, I created a ClosableStreamedResponse that extends StreamedResponse with an additional close future.

This should only be necessary for the IOStreamedResponse (for which it is implemented with detachSocket and destroy which has been tested and works). The browser client appears to obtain the entire buffered response from the browser (via the onLoad event) and return that, so there's no sense or utility in closing that response - it can be aborted with xhr.abort.

For ClosableStreamedResponses to be cancellable with the controller, Response.fromStream grabs the ActiveRequestTracker for the response's request (if there is one) and, if that controller has cancel called on it, closes the response stream if and only if the response's stream is a ClosableStreamedResponse. (See lines 70-78 of response.dart).

---

Examples

Partial Timeouts (for each step of the request-response cycle)

import 'package:http/http.dart';

Future<void> main() async {
  Client client = Client();

  // Create a request controller, can be passed into multiple requests to
  // cancel them all at once or to set timeouts on all of them at once.
  // Or, one can be created per request.
  final controller = RequestController(
    // Optional timeout for the overall request (entire round trip).
    timeout: Duration(seconds: 5),
    // Optional timeouts for each state of the request.
    // If a state is not specified, it will not timeout.
    partialTimeouts: PartialTimeouts(
      // Timeout for connecting to the server.
      connectTimeout: Duration(seconds: 5),
      // Timeout for the request body to be sent and a response to become
      // available from the server.
      sendTimeout: Duration(seconds: 4),
      // Timeout for processing the response body client-side.
      receiveTimeout: Duration(seconds: 6),
    ),
  );

  final response = await client.get(
    // URL that hangs for 5 seconds before responding.
    Uri.parse('http://localhost:3456/longrequest'),
    controller: controller,
  );

  // Unhandled exception:
  // TimeoutException after 0:00:04.000000: Future not completed
  print(response.body);
}

In this case, the send timeout triggers a TimeoutException because no response is received from the server.

Server-side logs:

04:29:58.608 PM [Request (/longrequest)]        resume
04:29:58.610 PM [Request (/longrequest)]        readable
04:29:58.610 PM [Request (/longrequest)]        end
04:29:58.610 PM [Request (/longrequest)]        close
04:30:02.618 PM [Response (/longrequest)]       close

Partial Timeouts (another example)

import 'package:http/http.dart';

Future<void> main() async {
  Client client = Client();

  // Create a request controller, can be passed into multiple requests to
  // cancel them all at once or to set timeouts on all of them at once.
  // Or, one can be created per request.
  final controller = RequestController(
    // Optional timeout for the overall request (entire round trip).
    timeout: Duration(seconds: 10),
    // Optional timeouts for each state of the request.
    // If a state is not specified, it will not timeout.
    partialTimeouts: PartialTimeouts(
      // Timeout for connecting to the server.
      connectTimeout: Duration(seconds: 5),
      // Timeout for the request body to be sent and a response to become
      // available from the server.
      sendTimeout: Duration(seconds: 4),
      // Timeout for processing the response body client-side.
      receiveTimeout: Duration(seconds: 6),
    ),
  );

  final response = await client.get(
    // URL that immediately writes headers and a partial body, then hangs for
    // 5 seconds before responding with the remainder.
    Uri.parse('http://localhost:3456/longpoll'),
    controller: controller,
  );

  // Hello, world! (the payload sent immediately)
  // Hello, world! (the payload sent after 5 seconds)
  print(response.body);
}

In this case, none of the timeouts trigger an exception because the response body becomes available immediately, even if it is not complete and the server hangs for only 5 seconds before completing the response, meaning the receive timeout of 6 seconds is not triggered either.

Server-side logs:

04:37:36.225 PM [Request (/longpoll)]   resume
04:37:36.226 PM [Request (/longpoll)]   readable
04:37:36.226 PM [Request (/longpoll)]   end
04:37:36.226 PM [Request (/longpoll)]   close
04:37:41.230 PM [Response (/longpoll)]  finish
04:37:41.230 PM [Request (/longpoll)]   resume
04:37:41.231 PM [Response (/longpoll)]  close

Cancelation Example (cancel during response)

import 'package:http/http.dart';

Future<void> main() async {
  Client client = Client();

  // Create a request controller, can be passed into multiple requests to
  // cancel them all at once or to set timeouts on all of them at once.
  // Or, one can be created per request.
  final controller = RequestController();

  final responseFuture = client.get(
    // URL that hangs for 5 seconds before responding.
    Uri.parse('http://localhost:3456/longrequest'),
    controller: controller,
  );

  await Future.delayed(Duration(milliseconds: 500));
  controller.cancel();

  final response = await responseFuture;
  print(response.body);

  // Unhandled exception:
  // CancelledException: Request cancelled
  print(response.body);
}

Cancels after 500ms which is whilst the response is being written. The connection is destroyed so both the client and server clean up immediately after the request is cancelled.

Server-side logs:

04:41:11.651 PM [Request (/longrequest)]        resume
04:41:11.651 PM [Request (/longrequest)]        readable
04:41:11.652 PM [Request (/longrequest)]        end
04:41:11.652 PM [Request (/longrequest)]        close
04:41:12.137 PM [Response (/longrequest)]       close

Cancellation Example (cancelled immediately)

import 'package:http/http.dart';

Future<void> main() async {
  Client client = Client();

  // Create a request controller, can be passed into multiple requests to
  // cancel them all at once or to set timeouts on all of them at once.
  // Or, one can be created per request.
  final controller = RequestController();

  final responseFuture = client.get(
    // URL that hangs for 5 seconds before responding.
    Uri.parse('http://localhost:3456/longrequest'),
    controller: controller,
  );

  controller.cancel();

  final response = await responseFuture;
  print(response.body);

  // Unhandled exception:
  // CancelledException: Request cancelled
  print(response.body);
}

The request is cancelled immediately and isn't even logged on the server.

Server-side logs:

(none)

---

Server implementation and Dart client example used to test:
https://github.com/SamJakob/cancellable_http_dart_testing

---

• I’ve reviewed the contributor guide and applied the relevant portions to this PR.

Contribution guidelines:

• See our contributor guide for general expectations for PRs.
• Larger or significant changes should be discussed in an issue before creating a PR.
• Contributions to our repos should follow the Dart style guide and use dart format.
• Most changes should add an entry to the changelog and may need to rev the pubspec package version.
• Changes to packages require corresponding tests.

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

[natebosch]

This is an interesting API.

What are your use cases for being able to independently control parts of the timeout?

I don't think we'll be able to move forward with this PR in the short term. It will take a deeper review to see if this API is in the direction we'd want to go with cancellation.

cc @brianquinlan

[natebosch]

This is a breaking change for implements Client uses. For example https://github.com/google/googleapis.dart/blob/ef216bc000480b20d8a08776343bd2f8eacd423d/googleapis_auth/lib/src/auth_client.dart#L10

[SamJakob]

Oops - as was already the case with BaseClient that was supposed to return false by default to allow clients to opt-in.

Edit: should be resolved with latest commit?

[natebosch]

Adding the API at all is breaking. Any class which implement Client without extends BaseClient will be missing the override.

[SamJakob]

Oh yes, of course. I'll do some testing against something like the linked client (AuthClient) for that then - I didn't initially find any clients that implemented Client so I presumed they would use BaseClient like ReplyClient does.

That might need further review then, but off the top of my head a backwards-compatible solution could be to introduce some stub that could be mixed in or included and tested for.

For example: https://dartpad.dev/bc9a693d707f95a0356291817f38db1e

[SamJakob]

I think generally the ability to control timeouts over individual parts of the lifecycle would be beneficial for specific use cases - particularly those dealing with large payloads - off the top of my head, some examples could be:

• to abort a connection if connecting to the server takes too long, but not once the initial connection has been made.
• to trigger a timeout only once the response is being delivered (for example, uploading a large payload that takes - say 30 seconds - to upload and subsequently alotting a time frame in which to start receiving a response).

These could be used similarly to the headerTimeout and contentTimeout discussed in #521.

Also, sendTimeout refers to the point in the lifecycle where the response headers are delivered because that I considered to be the end of the sending phase, but maybe it's more intuitive to call that headersTimeout and have responseTimeout be contentTimeout.

I would imagine it is most useful for things like StreamedResponse where you could set a timeout for the initial connection and response headers (connectTimeout and sendTimeout) but not for the response or the request as a whole.

Also, the lifecycle is tracked implicitly for the purposes of properly supporting cancellation in any client implementing the RequestController API, so I felt it useful to expose the timeouts for each of those stages too.

[brianquinlan]

Does HttpRequest.OPENED really imply that a connection has been established? I thought that it just meant that .open() has been called.

[SamJakob]

No, it doesn't (as suggested here on StackOverflow) so that probably shouldn't be implemented in the BrowserClient. (I wasn't sure if preflight requests might happen at the open stage, but since they depend on the headers and origin - having thought on it - I guess not and checking the Chromium seems to confirm that.)

That said, whilst looking into the browser side of things, I saw that the new fetch API (caniuse: fetch API) supports streams (caniuse: ReadableStream) and could probably be used to implement this functionality as it is on the Dart side.

What kind of support is being aimed for on the browser side? Is there a reason XMLHTTPRequest is still being used or might it be worth looking at moving to the fetch API? - See #595

[brianquinlan]

I think that partialTimeouts is a pretty complex API to implement and I'm not sure that it can be implemented completely on any Client other than IOClient.

Would it make sense to ask people who need these more advanced features to use package:dio? Though I'm not sure if their timeout, cancellation and progress support works if adapting from a package:http Client.

[SamJakob]

I agree that it is complex and maybe there is a point to be made that its complexity falls outside the scope of the package. I had some reservations about partialTimeouts with respect to testing too. That said, I replied to your review comment to address implementing it in BrowserClient with a streaming implementation of the fetch API that could allow for implementation of both platforms in a fairly similar way.

Also, as mentioned before, tracking the lifecycle in a similar manner would still be essential (at least on dart:io) to ensure the request is cancelled properly on certain platforms. It would just be a matter of determining the costs/benefits of exposing timeouts over it as an API.

I think the major benefit of this package is that it encapsulates all of the platform-specific implementations and exposes a single consistent API so as long as it is done consistently I would advocate including the feature in this package because it would save anyone needing a cross-platform HTTP solution that also requires control over partialTimeouts from having to re-implement each platform implementation.

[JaffaKetchup]

Just seen that this has been re-opened! 🎉

I've been wondering whether an approach without a dedicated controller could work and the timeouts part could work to get the basic functionality implemented faster.

Regardless, hoping this can be merged as relying on Dio for this is a bit annoying!

(Separately, I was wondering whether aborting HTTP is possible in WASM?)