Activator should wait for active requests to drain before terminating

[mattmoor]

In what area(s)?

/area autoscale
/area networking
/kind good-first-issue

Describe the feature

The activator receives the stop signal here, we should hook in a new activation handler here that manages a sync.WaitGroup{} that increments/decrements on new requests, and then does a wg.Wait() after the stopCh signal is received.

[vagababov]

From the Server.Shutdown docs:

Shutdown gracefully shuts down the server without interrupting any active connections. Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return to idle and then shut down. 

So, the way I read it, once we call shutdown

1. we won't get any new requests
2. all the current ones will be processed

Thus, I think this issue is unnecessary?

[markusthoemmes]

Agree with @vagababov. Have we seen this not working as expected?

[mattmoor]

You may be right, but two concerns come to mind:

1. What happens with streaming connections (e.g. websocket or GRPC)?
2. Even once the code handles graceful termination, I'm concerned that terminationGracePeriodSeconds is too low for our activator pods (read: it is the default of 30s), which is much shorter than even our default request timeout.

What I describe above is likely handled by the Shutdown internally (great!), but we likely still need some config for 2.

[tcnghia]

/assign @vagababov

looks like this will be closed by #4671