-
Notifications
You must be signed in to change notification settings - Fork 1
/
index.html
197 lines (189 loc) · 8.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Playout Statistics API for WebAudio</title>
<script src="https://www.w3.org/Tools/respec/respec-w3c" class="remove"></script>
<script src="https://w3c.github.io/hr-time/#dom-domhighrestimestamp"></script>
<script class='remove'>
"use strict";
// See https://github.com/w3c/respec/wiki/ for how to configure ReSpec
var respecConfig = {
"githubAPI": "WICG/audio_context_playout_stats",
"editors": [{
name: "Fredrik Hernqvist",
email: "[email protected]",
company: "Google",
companyURL: "google.com",
},{
name: "Palak Agarwal",
email: "[email protected]",
company: "Google",
companyURL: "google.com",
},
// Add additional editors here.
// https://github.com/w3c/respec/wiki/editors
],
"group": "media",
"latestVersion": "https://wicg.github.io/web_audio_playout/",
"shortName": "audio_context_playout_stats",
"specStatus": "unofficial",
"wg": "WICG",
};
</script>
</head>
<body>
<section id="abstract">
<p>
When playing audio through WebAudio, we want to be able to measure the delay of that audio and the glitchiness of
the audio. This document contains a proposal of an API that would allow WebAudio users to do this.
</p>
</section>
<section id="sotd">
<p>
This is an unofficial proposal.
</p>
</section>
<section id="introduction">
<h2>Introduction</h2>
<p>
There is currently no way to detect whether WebAudio playout has glitches (gaps in the played audio, which
typically happens due to underperformance in the audio pipeline). There is an existing way to measure the
instantaneous playout latency using <a
href="https://webaudio.github.io/web-audio-api/#dom-audiocontext-outputlatency">AudioContext.outputLatency</a>,
but no simple way to measure
average/minimum/maximum latency over time.
</p>
<p>
Glitches and high latency are bad for the user experience, so if any of these occur it can be useful for the
application to be able to detect this and possibly take some action to improve the playout.
</p>
</section>
<section data-dfn-for="AudioContext">
<h2>Extension of the <a href="https://webaudio.github.io/web-audio-api/#AudioContext"><dfn>AudioContext</dfn></a>
interface
</h2>
<pre class="idl">
partial interface AudioContext {
[SameObject] readonly attribute AudioPlayoutStats playoutStats;
};
</pre>
<p>
<section>
<h2>Attributes</h2>
<section>
<h2><dfn>playoutStats</dfn> attribute</h2>
<p>
The {{AudioContext/AudioPlayoutStats}} under AudioContext is a dedicated object for statistics reporting;
Similar to <a href="https://w3c.github.io/webrtc-stats/#dom-rtcaudioplayoutstats">RTCAudioPlayoutStats</a>,
but it is for the playout path via AudioDestinationNode and the associated output device.
This will allow us to measure glitches occurring due to underperforming AudioWorklets as well as glitches and
delay occurring in the playout path between the AudioContext and the output device.</p>
</section>
</section>
</p>
</section>
<section data-dfn-for="AudioPlayoutStats">
<h2><dfn>AudioPlayoutStats</dfn> interface</h2>
<pre class="idl">
[Exposed=Window, SecureContext]
interface AudioPlayoutStats {
readonly attribute DOMHighResTimeStamp fallbackFramesDuration;
readonly attribute unsigned long fallbackFramesEvents;
readonly attribute DOMHighResTimeStamp totalFramesDuration;
readonly attribute DOMHighResTimeStamp averageLatency;
readonly attribute DOMHighResTimeStamp minimumLatency;
readonly attribute DOMHighResTimeStamp maximumLatency;
undefined resetLatency();
[Default] object toJSON();
};
</pre>
<section>
<h2>Attributes</h2>
<section>
<h2><dfn>fallbackFramesDuration</dfn> attribute</h2>
<p>This value is measured in milliseconds and is incremented
each time a fallback frame is played by the output device at the end of the playout path. This metric can be
used together with {{AudioPlayoutStats/totalFramesDuration}} to calculate the percentage of played out media
that was not provided by the AudioContext.
</p>
</section>
<section>
<h2><dfn>fallbackFramesEvents</dfn> attribute</h2>
<p>This measures the number of synthesized fallback frames events.
This counter increases every time a fallback frame is played after a non-fallback frame. That is, multiple
consecutive fallback frames will increase {{AudioPlayoutStats/fallbackFramesDuration}} multiple times but is a
single fallback
frames event.
</p>
</section>
<section>
<h2><dfn>totalFramesDuration</dfn> attribute</h2>
<p>This attribute is the total duration, in milliseconds, of all audio
frames that have been played by the audio device. Includes both fallback and non-fallback frames.</p>
</section>
<section>
<h2><dfn>averageLatency</dfn> attribute</h2>
<p>This is the average latency for the frames played since the
last call to {{AudioPlayoutStats/resetLatency()}}, or since the creation of the AudioContext if
{{AudioPlayoutStats/resetLatency()}} has
not been called.</p>
</section>
<section>
<h2><dfn>minimumLatency</dfn> attribute</h2>
<p>This measures the minimum latency for the frames played since the
last call to {{AudioPlayoutStats/resetLatency()}}, or since the creation of the AudioContext if
{{AudioPlayoutStats/resetLatency()}} has not been called. </p>
</section>
<section>
<h2><dfn>maximumLatency</dfn> attribute</h2>
<p>This measures the maximum latency for the frames played since the
last call to {{AudioPlayoutStats/resetLatency()}}, or since the creation of the AudioContext if
{{AudioPlayoutStats/resetLatency()}} has not been called.
</p>
</section>
</section>
<section>
<h2>Methods</h2>
<section>
<h2><dfn>resetLatency</dfn> method</h2>
<p>This method resets the latency counters. Note that it does not remove
latency information that has accrued but not yet been exposed through the API.</p>
</section>
</section>
</section>
<section id="usage">
<h2>Usage Example</h2>
This is an example of how the API can be used to calculate the following stats over a time interval:
<ul>
<li>
Fraction of fallback frames
</li>
<li>
Frequency of fallback frames events (glitches)
</li>
<li>
Average playout delay
</li>
</ul>
<pre class="js">
var oldTotalFramesDuration = audioContext.playoutStats.totalFramesDuration;
var oldFallbackFramesDuration = audioContext.playoutStats.fallbackFramesDuration;
var oldFallbackFramesEvents = audioContext.playoutStats.fallbackFramesEvents;
audioContext.playoutStats.resetLatency();
// Wait while playing audio
...
// the number of seconds that were covered by the frames played by the output device between the two executions.
let deltaTotalFramesDuration = (audioContext.playoutStats.totalFramesDuration - oldTotalFramesDuration) / 1000;
let deltaFallbackFramesDuration = (audioContext.playoutStats.fallbackFramesDuration - oldFallbackFramesDuration) / 1000;
let deltaFallbackFramesEvents = audioContext.playoutStats.fallbackFramesEvents - oldFallbackFramesEvents;
// fallback frames fraction stat over the last deltaTotalFramesDuration seconds
let fallbackFramesFraction = deltaFallbackFramesDuration / deltaTotalFramesDuration;
// fallback event frequency stat over the last deltaTotalFramesDuration seconds
let fallbackEventFrequency = deltaFallbackFramesEvents / deltaTotalFramesDuration;
// Average playout delay stat during the last deltaTotalFramesDuration seconds
let playoutDelay = audioContext.playoutStats.averageLatency / 1000;
</pre>
</section>
</body>
</html>