Skip to content

Commit

Permalink
Video: Add avc3 and hev1 muxer support for MediaRecorder
Browse files Browse the repository at this point in the history 
Background: Safari initially supported `avc1` mp4 recording. Then,
Chrome added avc1 support since version M126 this year. The benefit
of `avc1` is that it is the only H.264 tag natively supported by
Apple devices. The same situation applies to `hvc1`.

However, there are some limitations for `avc1` and `hvc1`. That is,
they only allow "out of band" parameter sets and cannot change
parameter sets during encoding. So, if the capture device can
produce a video stream with a dynamically changing resolution,
then only `avc3` and `hev1` can satisfy the needs as they support
"in band" parameter sets so muxer can insert parameter sets into
the bitstream if they are changed.

This CL extends the existing `H264AnnexBToAvcBitstreamConverter`
and `H265AnnexBToHevcBitstreamConverter` by adding a property to
allow control the insertion of parameter sets into the bitstream.
Also wire the `Mp4MuxerDelegate`to write correct tag based on
the settings.

Bug: 381179823
Bug: 40276507
Change-Id: I6d104546e9cf46b18fae4dc1fd8af1124b918e7c
  • Loading branch information
@StaZhu @chromium-wpt-export-bot
StaZhu authored and chromium-wpt-export-bot committed Dec 13, 2024
1 parent e45da16 commit 9e79981
Show file tree
Hide file tree
Showing 6 changed files with 46 additions and 18 deletions.
10 changes: 7 additions & 3 deletions mediacapture-record/MediaRecorder-canvas-media-source.https.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,17 @@
<meta name=variant content="?mimeType=video/webm;codecs=vp8,opus">
<meta name=variant content="?mimeType=video/webm;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/webm;codecs=av1,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc3.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4">
<link rel="help"
href="https://w3c.github.io/mediacapture-record/MediaRecorder.html#dom-mediarecorder-mimeType">
Expand Down
10 changes: 7 additions & 3 deletions mediacapture-record/MediaRecorder-events-and-exceptions.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,17 @@
<meta name=variant content="?mimeType=video/webm;codecs=vp8,opus">
<meta name=variant content="?mimeType=video/webm;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/webm;codecs=av1,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc3.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4">
<link rel="help" href="https://w3c.github.io/mediacapture-record/MediaRecorder.html#mediarecorder">
<script src="/resources/testharness.js"></script>
Expand Down
11 changes: 9 additions & 2 deletions mediacapture-record/MediaRecorder-mimetype.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,13 @@
'video/webm; codecs="vp8"',
'video/webm; codecs="vp9"',
'video/webm; codecs="av1"',
'video/mp4; codecs="avc1"',
'video/mp4; codecs="avc1.64003E"',
'video/mp4; codecs="avc3.64003E"',
'video/mp4; codecs="vp9"',
'video/mp4; codecs="hvc1.1.6.L186.B0"',
'video/mp4; codecs="hev1.1.6.L186.B0"',
'video/x-matroska; codecs="hvc1.1.6.L186.B0"',
'video/x-matroska; codecs="hev1.1.6.L186.B0"',
];

const AUDIO_VIDEO_MIME_TYPES = [
Expand All @@ -46,11 +49,15 @@
'video/webm; codecs="vp9, vorbis"',
'video/webm; codecs="vp9, opus"',
'video/webm; codecs="av1, opus"',
'video/mp4; codecs="avc1, mp4a.40.2"',
'video/mp4; codecs="avc1.64003E, mp4a.40.2"',
'video/mp4; codecs="avc3.64003E, mp4a.40.2"',
'video/mp4; codecs="vp9, opus"',
'video/mp4; codecs="hvc1.1.6.L186.B0, mp4a.40.2"',
'video/mp4; codecs="hev1.1.6.L186.B0, mp4a.40.2"',
'video/mp4; codecs="hvc1.1.6.L186.B0, opus"',
'video/mp4; codecs="hev1.1.6.L186.B0, opus"',
'video/x-matroska; codecs="hvc1.1.6.L186.B0, opus"',
'video/x-matroska; codecs="hev1.1.6.L186.B0, opus"',
];

const AUDIO_MIME_TYPES = [
Expand Down
13 changes: 9 additions & 4 deletions mediacapture-record/MediaRecorder-pause-resume.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,20 @@
<meta name=variant content="?mimeType=video/webm;codecs=vp8,opus">
<meta name=variant content="?mimeType=video/webm;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/webm;codecs=av1,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc3.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1.64003E,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=avc3.64003E,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=vp9,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4">
<link rel="help" href="https://w3c.github.io/mediacapture-record/MediaRecorder.html#mediarecorder">
<script src="/resources/testharness.js"></script>
Expand Down
10 changes: 7 additions & 3 deletions mediacapture-record/MediaRecorder-peerconnection.https.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,19 @@
<meta name=variant content="?kinds=video,audio&mimeType=video/webm;codecs=vp8,opus">
<meta name=variant content="?kinds=video&mimeType=video/webm;codecs=vp9">
<meta name=variant content="?kinds=video,audio&mimeType=video/webm;codecs=vp9,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=avc1,mp4a.40.2">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=avc1.64003E,mp4a.40.2">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=avc3.64003E,mp4a.40.2">
<meta name=variant content="?kinds=video&mimeType=video/mp4;codecs=vp9">
<meta name=variant content="?kinds=audio&mimeType=audio/mp4;codecs=opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=vp9,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=av01,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=av01,mp4a.40.2">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=av01.1.19M.08,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=av01.1.19M.08,mp4a.40.2">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4;codecs=hev1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?kinds=video,audio&mimeType=video/x-matroska;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/x-matroska;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?kinds=video,audio&mimeType=video/mp4">

<link rel="help"
Expand Down
10 changes: 7 additions & 3 deletions mediacapture-record/MediaRecorder-stop.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,17 @@
<meta name=variant content="?mimeType=video/webm;codecs=vp8,opus">
<meta name=variant content="?mimeType=video/webm;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/webm;codecs=av1,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc1.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=avc3.64003E,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=vp9,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=av01.1.19M.08,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4;codecs=hvc1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/mp4;codecs=hev1.1.6.L186.B0,mp4a.40.2">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hvc1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/x-matroska;codecs=hev1.1.6.L186.B0,opus">
<meta name=variant content="?mimeType=video/mp4">
<link rel="help" href="https://w3c.github.io/mediacapture-record/MediaRecorder.html#mediarecorder">
<script src="/resources/testharness.js"></script>
Expand Down

0 comments on commit 9e79981

Please sign in to comment.