Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update media time usage for remote playback protocol. #328

Merged
merged 3 commits into from
May 31, 2024
Merged

Update media time usage for remote playback protocol. #328

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4f8215a
Update usage of time
markafoltz Feb 28, 2024
ffea304
Clarify the units for epoch.
markafoltz Feb 28, 2024
0cf12c7
Respond to baylesj comments
markafoltz May 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 37 additions & 18 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ urlPrefix: https://html.spec.whatwg.org/multipage/media.html#; type: dfn; spec:
text: poster frame
text: timeline offset
text: media resource
text: media timeline
urlPrefix: https://www.w3.org/TR/presentation-api/#dfn-; type: dfn; spec: PRESENTATION-API
text: available presentation display
text: controlling user agent
Expand Down Expand Up @@ -732,7 +733,7 @@ The messages used in this authentication method are: [=auth-spake2-handshake=],
how [=auth-spake2-handshake=] and [=auth-spake2-confirmation=] are computed.

The values `A` and `B` used in SPAKE2 are the [=agent fingerprints=] of the
client and server, respectively. `pw` is the PSK presented to the user.
client and server, respectively. `pw` is the PSK presented to the user.

The PSK presenter or the PSK consumer may initiate authentication (assuming the
role of Alice in SPAKE2).
Expand Down Expand Up @@ -1092,8 +1093,6 @@ defined in this section.
For all messages defined in this section, see Appendix A for the full
CDDL definitions.

Issue(159): Refinements to Remote Playback protocol.

To learn which receivers are [=compatible remote playback devices=] for a
particular URL or set of URLs, the controller may send a
[=remote-playback-availability-request=] message with the following values:
Expand Down Expand Up @@ -1486,17 +1485,18 @@ changed.
for the initial state in the [=remote-playback-start-response=] message.

: epoch
:: The "zero time" of the media timeline. See
[=timeline offset=] and
:: The "zero time" of the [=media timeline=], in milliseconds relative to the
epoch. See [=timeline offset=] and
{{HTMLMediaElement/getStartDate()|HTMLMediaElement.getStartDate()}}.
The default is an unknown epoch
for the initial state in the [=remote-playback-start-response=] message.
The default is an unknown epoch for the initial state in the
[=remote-playback-start-response=] message, which is represented by null.

: duration
:: The duration of the media timeline. See
:: The duration of the [=media timeline=], in seconds. See
{{HTMLMediaElement/duration|HTMLMediaElement.duration}}.
The default is an unknown duration
for the initial state in the [=remote-playback-start-response=] message.
for the initial state in the [=remote-playback-start-response=] message,
which is represented by null.

: buffered-time-ranges
:: The time ranges for which media has been buffered. See
Expand Down Expand Up @@ -1572,8 +1572,9 @@ changed.
{{HTMLVideoElement/videoWidth|HTMLVideoElement.videoWidth}}
and
{{HTMLVideoElement/videoHeight|HTMLVideoElement.videoHeight}}.
The default is unknown
for the initial state in the [=remote-playback-start-response=] message.
The default is an unknown resolution
for the initial state in the [=remote-playback-start-response=] message,
which is represented by null.

: audio-tracks
:: The available audio tracks, which can individually enabled or disabled. See
Expand All @@ -1594,9 +1595,12 @@ changed.
The default is an empty array
for the initial state in the [=remote-playback-start-response=] message.

All times, time ranges, and durations (such as position, duration, and
seekable-time-ranges) used above use a common [=media-time=] value (see Appendix A)
which includes a [=time scale=].
Media positions, durations, and time ranges are defined in terms of the [=media
timeline=] specified in HTML, which are fractional seconds between zero and the
media duration.

NOTE: An Open Screen agent can convert between values on the media timeline and
the media sync time sent with individual media frames. See Appendix D.

<div class="note">
<table>
Expand All @@ -1606,20 +1610,20 @@ which includes a [=time scale=].
<tr><td>loading</td> <td>`empty`</td></tr>
<tr><td>loaded</td> <td>`nothing`</td></tr>
<tr><td>error</td> <td>No error</td></tr>
<tr><td>epoch</td> <td>`unknown`</td></tr>
<tr><td>duration</td> <td>`unknown`</td></tr>
<tr><td>epoch</td> <td>`null`</td></tr>
<tr><td>duration</td> <td>`null`</td></tr>
<tr><td>buffered-time-ranges</td><td>Empty array</td></tr>
<tr><td>played-time-ranges</td> <td>Empty array</td></tr>
<tr><td>seekable-time-ranges</td><td>Empty array</td></tr>
<tr><td>position</td> <td>0</td></tr>
<tr><td>position</td> <td>0.0</td></tr>
<tr><td>playbackRate</td> <td>1.0</td></tr>
<tr><td>paused</td> <td>False</td></tr>
<tr><td>seeking</td> <td>False</td></tr>
<tr><td>stalled</td> <td>False</td></tr>
<tr><td>ended</td> <td>False</td></tr>
<tr><td>volume</td> <td>1.0</td></tr>
<tr><td>muted</td> <td>False</td></tr>
<tr><td>resolution</td> <td>`unknown`</td></tr>
<tr><td>resolution</td> <td>`null`</td></tr>
<tr><td>audio-tracks</td> <td>Empty array</td></tr>
<tr><td>video-tracks</td> <td>Empty array</td></tr>
<tr><td>text-tracks</td> <td>Empty array</td></tr>
Expand Down Expand Up @@ -2898,3 +2902,18 @@ To decode a PSK `P` given a QR code, follow these steps:

1. Obtain the string `N` by decoding the QR code.
2. Parse `N` as a hexadecimal number to obtain `P`.

Appendix D: Media Time Conversions {#appendix-d}
==================================

To convert between a media synchronization timestamp for a given audio or video
frame and a media timeline value, the following formula can be used:

```
media-timeline-value = media-zero-time + (value / scale)
```

Where:
- `media-zero-time` is the origin of the [=media timeline=] as defined in HTML.
- `value` and `scale` are the values passed in the `sync-time` field of the
markafoltz marked this conversation as resolved.
Show resolved Hide resolved
corresponding [=audio-frame=] or [=video-frame=].
46 changes: 25 additions & 21 deletions messages_appendix.cddl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ request-id = uint

microseconds = uint

epoch-time = int

media-timeline = float64

; type key 1001
auth-capabilities = {
0: uint ; psk-ease-of-input
Expand Down Expand Up @@ -343,10 +347,10 @@ remote-playback-controls = {
? 3: bool ; loop
? 4: bool ; paused
? 5: bool ; muted
? 6: float ; volume
? 7: media-time ; seek
? 8: media-time ; fast-seek
? 9: float ; playback-rate
? 6: float64 ; volume
? 7: media-timeline ; seek
? 8: media-timeline ; fast-seek
? 9: float64 ; playback-rate
? 10: text ; poster
? 11: [* text] ; enabled-audio-track-ids
? 12: text ; selected-video-track-id
Expand Down Expand Up @@ -376,20 +380,20 @@ remote-playback-state = {
enough: 4
) ; loaded
? 5: media-error : error
? 6: media-time / unknown; epoch
? 7: media-time / unknown / forever ; duration
? 8: [* media-time-range] ; buffered-time-ranges
? 9: [* media-time-range] ; seekable-time-ranges
? 10: [* media-time-range] ; played-time-ranges
? 11: media-time ; position
? 12: float : playbackRate
? 6: epoch-time / null; epoch
? 7: media-timeline / null ; duration
? 8: [* media-timeline-range] ; buffered-time-ranges
? 9: [* media-timeline-range] ; seekable-time-ranges
? 10: [* media-timeline-range] ; played-time-ranges
? 11: media-timeline ; position
? 12: float64 : playbackRate
? 13: bool ; paused
? 14: bool ; seeking
? 15: bool ; stalled
? 16: bool ; ended
? 17: float ; volume
? 17: float64 ; volume
? 18: bool ; muted
? 19: video-resolution / unknown ; resolution
? 19: video-resolution / null ; resolution
? 20: [* audio-track-state] audio-tracks
? 21: [* video-track-state] video-tracks
? 22: [* text-track-state] text-tracks
Expand Down Expand Up @@ -422,16 +426,16 @@ text-track-mode = &(

text-track-cue = {
1: text ; id
2: media-time-range ; range
2: media-timeline-range ; range
3: text ; text
}

media-time-range = [
start: media-time,
end: media-time
media-timeline-range = [
markafoltz marked this conversation as resolved.
Show resolved Hide resolved
start: media-timeline,
end: media-timeline
]

media-time = [
media-sync-time = [
value: uint
scale: uint
]
Expand Down Expand Up @@ -478,7 +482,7 @@ audio-frame = [
payload: bytes
? optional: {
? 0: uint ; duration
? 1: media-time ; sync-time
? 1: media-sync-time ; sync-time
}
]

Expand All @@ -491,7 +495,7 @@ video-frame = {
? 4: uint ; duration
5: bytes ; payload
? 6: uint ; video-rotation
? 7: media-time ; sync-time
? 7: media-sync-time ; sync-time
}

; type key 24
Expand All @@ -501,7 +505,7 @@ data-frame = {
? 2: uint ; start-time
? 3: uint ; duration
4: any; payload
? 5: media-time ; sync-time
? 5: media-sync-time ; sync-time
}

ratio = [
Expand Down
Loading