MSC2746: Improved VoIP Signalling

proposals/2746-reliable-voip.md

@@ -0,0 +1,194 @@
+# MSC2746: Improved Signalling for 1:1 VoIP
+
+Matrix has basic support for signalling 1:1 WebRTC calls, but has a number of shortcomings:
+
+ * If several devices try to answer the same call, there is no way for them to determine clearly
+   that the caller has set up the call with a different device, and no way for the caller to
+   determine candidate events map to which answer.
+ * Hangup reasons are often incorrect.
+ * There is confusion and no clear guidance on how clients should determine whether an incoming
+   invite is stale or not.
+ * There is no support for renegotiation of SDP, for changing ICE candidates / hold/resume
+   functionality, etc.
+ * There is no distinction between rejecting a call and ending it, which means that in trying
+   to reject a call, a client can inadvertantly cause a call that has been sucessfully set up
+   on a different device to be hung up.
+
+## Proposal
+### Change the `version` field in all VoIP events to `1`
+This will be used to determine whether determine whether devices support this new version of the protocol.
+If clients see events with `version` other than `0` or `1`, they should treat these the same as if they had
+`version` == `1`.
+
+### Add `answer_id` to `m.call.answer`
+This is a random string generated by the sending device in the same way as `call_id` (ie. in this case,
+it should be sufficiently unique in the context of this VoIP call). This identifies each answer sent to
+a given `m.call.invite`.
+
+*Also considered: use the `event_id` of the answer event: this is rejected for similarity with `call_id`
+where it is desirable to know the ID of a call before receving the remote echo of the invite event (this
+will be useful in future for call transfers where the transferor can assign a `call_id` for the transferee
+to use).*
+
+### Add `answer_id` to `m.call.candidates`
+This allows the caller to determine which candidate events correspond to which answer (for the callee,
+all candidates with matching `call_id` not from its own user are from the caller party, of which there
+is only one.)
+
+### Introduce `m.call.ack`
+This event is sent by the caller once it has chosen an answer. Its `answer_id`
+field indicates the answer it's chosen (and has `call_id` too). If the callee
+sees an ack for an answer ID other than the one it sent, it ends the call and
+informs the user the call was answered elsewhere. It does not send any events.
+Media can start flowing before this event is seen or even sent. Clients that
+implement previous versions of this specification will ignore this event and
+behave as they did before.
+
+Example:
+```
+{
+    "type": "m.call.ack",
+    "content": {
+        "call_id": "12345",
+        "answer_id": "67890",
+    },
+}
+```
+
+### Introduce `m.call.reject`
+
+ * If the `m.call.invite` event has `version` `1`, a client wishing to reject a call instead
+   sends an `m.call.reject` event. This rejects the call on all devices, but if another device
+   has already sent an accept, it disregards the reject and carries on. The reject has an
+   `answer_id` just like an answer, and the caller acks it just like an answer. If the other
+   client that had already sent an answer sees the caller ack the reject instead of its answer,
+   it ends the call.
+ * If the `m.call.invite` event has `version` `0`, the callee send an `m.call.hangup` event before.
+
+Example:
+```
+{
+    "type": "m.call.reject",
+    "content" : {
+        "version": 1,
+        "call_id": "12345",
+        "answer_id": "67890",
+    }
+}
+```
+
+If the calling user chooses to end the call before setup is complete, the client sends `m.call.hangup`
+as previously.
+
+### Clarify what actions a client may take in response to an invite
+The client may:
+ * Attempt to accept the call by sending an answer
+ * Actively reject the call everywhere: reject the call as per above, which will stop the call from
+   ringing on all the user's devices and the caller's client will inform them that the user has
+   rejected their call.
+ * Ignore the call: send no events, but stop alerting the user about the call. The user's other
+   devices will continue to ring, and the caller's device will continue to indicate that the call
+   is ringing, and will time the call out in the normal way if no other device responds.
+
+### Introduce more reason codes to `m.call.hangup`
+ * `ice_timeout`: The connection failed after some media was exchanged (as opposed to current
+   `ice_failed` which means no media connection could be established). Note that, in the case of
+   an ICE renegotiation, a client should be sure to send `ice_timeout` rather than `ice_failed` if
+   media had previously been received successfully, even if the ICE renegotiation itself failed.
+ * `user_hangup`: Clients must now send this code when the user chooses to end the call, although
+   for backwards compatability, a clients should treat an absence of the `reason` field as
+   `user_hangup`.
+ * `user_media_failed`: The client was unable to start capturing media in such a way as it is unable
+   to continue the call.
+ * `unknown_error`: Some other failure occurred that meant the client was unable to continue the call
+   rather than the user choosing to end it.
+
+### Introduce `m.call.negotiate`
+This introduces SDP negotiation semantics for media pause, hold/resume, ICE restarts and voice/video
+call up/downgrading. Clients should implement & honour hold functionality as per WebRTC's
+recommendation: https://www.w3.org/TR/webrtc/#hold-functionality
+
+If both the invite event and the accepted answer event have `v` equal to `1`, either party may
+send `m.call.negotiate` with an `sdp` field to offer new SDP to the other party. This event has
+`call_id` with the ID of the call and those sent by the callee have `answer_id` equal to the ID
+of the client's answer. The caller ignores any negotiate events with `answer_id` not equal to the
+answer it accepted. Clients may either use the same mechanism used for remote echo of messages
+to recognise and ignore their own negotiate messages (ie. txn id) or they may ignore messages
+from their own user, or they may use the presence or absence of an `answer_id` field.
+
+This has a `lifetime` field as in `m.call.invite`, after which the sender of the negotiate event
+should consider the negotiation failed (timed out) and the recipient should ignore it.
+
+Example:
+```
+{
+    "type": "m.call.negotiate",
+    "content": {
+        "call_id": "12345",
+        "answer_id": "67890",
+        "sdp": "[some sdp]",
+        "lifetime": 10000,
+    }
+}
+```
+
+### Designate one party as 'polite'
+In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example)
+we introduce rules to establish which party is polite. By default, the callee is the polite party.
+In a glare situation, if the client receives an invite whilst preparing to send, it becomes the callee
+and therefore becomes the polite party. If an invite is received after the client has sent one, the
+party whose invite had the lexicographically greater call ID becomes the polite party.
+
+### Add explicit recommendations for call event liveness.
+`m.call.invite` contains a `lifetime` field that indicates how long the offer is valid for. When
+a client receives an invite, it should use the `age` field of the event plus the time since it
+received the event from the homeserver to determine whether the invite is still valid. If the
+invite is still valid *and will remain valid for long enough for the user to accept the call*,
+it should signal an incoming call. The amount of time allowed for the user to accept the call may
+vary between clients, for example, it may be longer on a locked mobile device than on an unlocked
+desktop device.
+
+The client should only signal an incoming call once it has completed processing the entire sync
+response. This ensures that if the sync response contains subsequent events that indicate the call
+has been hung up, rejected, or answered elsewhere, the client does not signal it.
+
+If on startup, after processing locally stored events, the client determines that there is an invite
+that is still valid, it should still signal it but only after it has completed a sync from the homeserver.
+
+### Introduce recommendations for batching of ICE candidates
+Clients should aim to send a small number of candidate events, with guidelines:
+ * ICE candidates which can be discovered immediately or almost immediately in the invite/answer
+   event itself (eg. host candidates). If server reflexive or relay candiates can be gathered in
+   a sufficiently short period of time, these should be sent here too. A delay of around 200ms is
+   suggested as a starting point.
+ * The client should then allow some time for further candidates to be gathered in order to batch them,
+   rather than sending each candidate as it arrives. A starting point of 2 seconds after sending the
+   invite or 500ms after sending the answer is suggested as starting point (since a delay is natural
+   anyway after the invite whilst the client waits for the user to accept it).
+
+### Add DTMF
+Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standard as of August
+2020 does not support receiving DTMF but a Matrix client can receive and interpret the DTMF sent
+in the RTP payload.
+
+### Deprecate `type` in `m.call.invite` and `m.call.answer`
+These are redundant: clients should continue to send them but must not require
+them to be present on events they receive.
+
+## Potential issues
+ * It remains explicity impossible to place a call to yourself. Matrix uses a shared medium for
+   signalling so a client will always see invites from other devices. We would need to introduce
+   a way for a client to signal to other devices that they should treat the invite as an incoming
+   call and mechanisms to clarify what events were from which party in a call. This would mean a
+   significant amount of protocol dedcated to just this feature, so this MSC omits it.
+
+## Alternatives
+ * We could use event IDs for `call_id` and `answer_id` as discussed above.
+ * The event type of `m.call.ack` mirrors that of SIP, although gives few other clues on its purpose.
+   `m.call.choose_answer` was considered but is quite verbose.
+## Security considerations
+ * IP addresses remain in the room in candidates, as they did in the previous version of the spec.
+   This is not ideal, but alternatives were either sending candidates over to-device messages
+   (would slow down call setup because a target device would have to be established before sending
+   candidates) or redacting them afterwards (the volume of events sent during calls can already
+   cause rate limiting issues and this would exacerbate this).