RTCRtpSender / Receiver objects need to be added to the specification

[dontcallmedom]

Originally reported on W3C Bugzilla ISSUE-25497 Tue, 29 Apr 2014 09:43:56 GMT
Reported by Harald Alvestrand
Assigned to

The proposal for RTCRtpSender /Receiver sent to the mailing list on April 28 is reproduced below. The April 28 meeting seemed to agree that this is a sensible direction, but details may need adjusting before integrating.

I suggest we call the SendDoohickeys RTCRtpSenders and the corresponding receive-side objects RTCRtpReceivers. These objects allow control of how a MediaStreamTrack is encoded and sent on the wire, including "hold" state, prioritization, and multiple encodings (e.g. simulcast).

You get a RTCRtpSender as a return value from AddTrack (which replaces AddStream). You get a RTCRtpReceiver as an argument to the new onaddtrack callback (which replaces onaddstream). The actual track object can be obtained as a property from the RTCRtpReceiver (see below).

For getting access to ICE/DTLS info, both RTCRtpSenders and RTCRtpReceivers can also have a reference to a RTCDtlsTransport object, which will have its own state variables, including the RTCIceConnectionState of that particular transport, and the .remoteCertificates for the DTLS connection. This allows applications to monitor the state of individual transports, as well as inspect the certificates for individual transports.

The actual interface is as follows:

// used with RTCRtpSender

interface RTCDtlsTransport {

attribute RTCIceConnectionState state;

sequence getRemoteCertificates();

//... more stuff as needed

};

// used with RTCRtpSender

interface RTCRtpEncodingParams {

double priority = 1.0; // relative priority of this encoding

unsigned int maxBitrate = null; // maximum bits to use for this encoding

boolean active; // sending or "paused/onhold"

};

// the "send" doohickey

interface RTCRtpSender {

readonly attribute MediaStreamTrack track;

readonly attribute RTCDtlsTransport transport;

// 1-N encodings; in the future, N can be > 1, for simulcast or layered coding

// Each EncodingParams specifies the details of what to send (e.g. bitrate)

sequence getEncodings();

};

// the "receive" doohickey

interface RTCRtpReceiver {

readonly attribute RTCDtlsTransport transport;

readonly attribute MediaStreamTrack track;

// more stuff as needed

};

// parameter to the onaddtrack event

interface RemoteTrackEvent : Event {

readonly attribute RtpReceiver receiver;

readonly attribute MediaStreamTrack track;

readonly attribute MediaStream stream;

};

partial interface RTCPeerConnection {

// because a track can be part of multiple streams, the id parameter

// indicates which particular stream should be referenced in signaling

RtpSender addTrack(track, streamId); // replaces addStream

void removeTrack(RtpSender); // replaces removeStream

sequence getSenders();

sequence getReceivers();

EventHandler onaddtrack; // replaces onaddstream; event object is RemoteTrackEvent.

                                    // note that onremovestream is not needed, since tracks are 'removed'

                                    // simply by progressing to the ENDED state

};

For backcompat, addStream, removeStream, getLocalStreams, getRemoteStreams, and onaddstream can be trivially polyfilled in JS, so there is minimal impact on current applications.

All together, the pipeline looks like this:

Source ---> MST ---> RtpSender ---> DtlsTransport ---> (The Internet) ---> DtlsTransport ---> RtpReceiver ---> MST --->

Each RtpSender/Receiver references a single MST, although a single RtpSender/Receiver can send/receive multiple encodings (e.g. simulcast).

There are N RtpSenders/Receivers per DtlsTransport; N is controlled by the policy specified for BUNDLE.

[dontcallmedom]

Original comment by Harald Alvestrand on W3C Bugzilla. Thu, 08 May 2014 06:06:33 GMT

Summary from Justin Uberti, May 8:

Trying to bring this discussion to a conclusion... I sense consensus around the following:

• General concept of doohickeys, i.e. RTCRtpSender, RTPRtpReceiver.
• addTrack(track, streamId)/removeTrack and onaddtrack.
• Implicit cloning never occurs. This means that if you want to add a track twice to a PC, you need to first clone it.

More discussion is still needed on:

• RTCRtpEncodingParams
• RTCDtlsTransport

Therefore I would like to advance the initial, uncontroversial parts of this proposal, i.e. what I describe below. We can then discuss the exact nature of the encodings and transport objects separately on the list, and at the interim meeting.

// the "send" doohickey

interface RTCRtpSender {

readonly attribute MediaStreamTrack track;

};

// the "receive" doohickey

interface RTCRtpReceiver {

readonly attribute MediaStreamTrack track;

};

// parameter to the onaddtrack event

interface AddTrackEvent : Event {

readonly attribute RtpReceiver receiver;

readonly attribute MediaStreamTrack track;

readonly attribute MediaStream stream;

};

partial interface RTCPeerConnection {

// because a track can be part of multiple streams, the id parameter

// indicates which particular stream should be referenced in signaling

RTCRtpSender addTrack(track, streamId); // replaces addStream; fails if |track| has already been added

void removeTrack(RTCRtpSender); // replaces removeStream

sequence getSenders();

sequence getReceivers();

EventHandler onaddtrack; // replaces onaddstream; event object is RemoteTrackEvent.

                                    // note that onremovestream is not needed, since tracks are 'removed'

                                    // simply by progressing to the ENDED state

};

[dontcallmedom]

Original comment by Harald Alvestrand on W3C Bugzilla. Mon, 03 Nov 2014 20:30:59 GMT

Reference to basic solution as of Oct 2014:

https://www.w3.org/2011/04/webrtc/wiki/images/6/6c/WebRTC_RTCSender-Receiver%2C_TPAC_2014.pdf

Slide 5 gives the basic structure. This has consensus, and should be integrated.

[juberti]

Closing per PR #175

[elagerway]

Guys, when ORTC objects, like these, land in the WebRTC 1.0 spec, please provide proper attribution back to the ORTC CG. Preferably with links pointing directly to the objects in the ORTC API. Everyone should understand where these objects originated and where they are being maintained, as an example:

The following objects originated in the ORTC API and are maintained by the ORTC CG:

RTCRtpSender Object

RTCRtpReceiver Object

[juberti]

Regarding "maintained by", that seems problematic for a spec that is actually defining said objects.

I will leave attribution as a question for the chairs.

[elagerway]

Valid point re: maintained, I modified the attribution accordingly...

The following objects originated in the W3C ORTC CG and have been adapted for use in this specification.

[dontcallmedom]

Re attribution, I agree it makes sense, and would suggest putting it in the Acknowledgements appendix.

[juberti]

OK. I will make a PR for this.

[fluffy]

This has been added so I am closing this bug