WebAudio Experiences

Webaudio Programming

The WebAudio Api consists of a number of Javascript classes for programming audio applications in the browser. The Api is really easy to use, the classes represent synthesizer hardware like oscillators, filters, delays, gain, etc. An example:

var context = new (window.AudioContext || window.webkitAudioContext)(); var oscillator = new OscillatorNode(context); oscillator.frequency.value = 440; oscillator.connect(context.destination); oscillator.start(); setTimeout(()=>{ oscillator.disconnect(); oscillator.stop(); }, 1000);

Thats how easy it is to sound a tone of 440Hz! First we get an audio context, you need only one of these troughout your project, the context must be the same for all audionodes you want to inter-connect. Then an OscillatorNode object is constructed in the context, the oscillator is connected to the context's destination, then oscillator.start() is called and it starts to produce sound. After a timeout of 1 second the oscillator is stopped and disconnected. Try it out for yourself.

Now when you want to make a musical instrument, the tone should only sound when a key is pressed, and stop sounding when the key is released. This is simply accomplished by a handler on the button for mouseup that switches off the oscillator like the timeout handler did in the previous example. I lowered the frequency 1 octave. Press and release the button quickly a number of times and listen carefully to the sound. Do you hear sounds you would'nt expect, a clicking sound when you press en release the button? Well I do, and in my opinion it is a shortcoming of the webaudio API. The clicking sounds, the glitches, are the result of connecting sound buffers without smoothening out the connection point. When the phase of the 220 Hz wave is at its maximum point, and then connected to a silent output, the output changes in 1 render quantum from 0 to max. That change causes a sharp glitch. To prevent such a thing one should use a technique called zero-point switching.

The next example has a few keys to play notes. When you want to play tones, you need to know the frequency of each tone. There is a simple calculation for this, if the first key of your keyboard represents a frequency of f, then each higher key represents a frequency of f multiplied by 2 to the power of keynumber / 12, in Javascript:

var fBase = 220; var keynumber = 8; var keyFrequency = fBase * Math.pow(2, keynumber / 12);

what is about 349 Hz. Try the keyboard, and again notice the disturbing glitches.

Every musical instrument needs some sort of an envelope for its volume, a way to let the volume rise and decay faster or slower. We can use a GainNode for volume control. Most audionodes have AudioParam interfaces for control. The GainNode has 1, its called gain. An audioparam has 1 property; value, and methods for changing the value over time. Just what we need for our envelope aka ADSR. Now, setting the value property of an AudioParam or changing the value over time are operations in different universes, the one does not take notice of the other. If you want to change a parameter gradually over time, you have to use 'automation events' all the way. Lets give our oscillator a GainNode for the envelope:

var envelope = new GainNode(context); oscillator.connect(envelope); envelope.connect(context.destination); envelope.gain.setValueAtTime(0, 0); // set zero gain envelope.linearRampToValueAtTime(1,context.currentTime + 0.5) //when the key is released envelope.gain.setValueAtTime(1, 0); // set gain of 1 envelope.linearRampToValueAtTime(0, context.currentTime + 0.5)

When a key is pressed the first automation event sets the gain immediately to zero. Then a linear ramp changes the gain from 0 to 1 in .5 seconds. Automation events use the context clock for time-scheduling of events. In the specs it says that a scheduled event with a time before the currentTime is clamped to currentTime, so you may use 0 for eventtime if the event has to happen immidiately. Future eventtimes must be added to the currentTime. But be aware that when you set a series of immediate events with time 0, they can become skewed when system interrupts happen inbetween. Try the keyboard and notice the sound. When a key is pressed and released slowly, no disturbing noises are heard, but when keys are pressed in a rythm faster then halve a second its just as worse as without envelope. The reason is that the attack ramp is interrupted before it reaches the value of 1. When the next key is pressed while the attack of the former is halfway its ramp, then the realtime gain will be 0.5 when it is reset to 1 for the decay ramp. The volume jump causes a glitch of halve the gain.

The solution to this problem seems easy, just read the actual realtime gain at the moment the ramp is interrupted, and use this value to init the release. Easy except for the value can not be read back!. This, in my opinion, is the biggest flaw of the audio API. The value property of the AudioParam allways returns the value assigned to it, but not the real-time value of automation events.

Wat does the webaudio specification have to say about the value of an AudioParam? This:

An intrinsic parameter value will be calculated at each time, which is either the value set directly to the value attribute, or, if there are any automation events with times before or at this time, the value as calculated from these events. When read, the value attribute always returns the intrinsic value for the current time. If automation events are removed from a given time range, then the intrinsic value will remain unchanged and stay at its previous value until either the value attribute is directly set, or automation events are added for the time range.

Seems clear enough, the value parameter should return the real time value set directly or by automation events. But it does not, at least not in Firefox. When you are interested then read this post wich seems to indicate that the webaudio developers themselves are not too sure and disagree about the interpretation of the specs on this point. The second part, stating the value does remain unchanged when automation events are removed (by cancelScheduledValues) is also not the case, at least not with Firefox where it falls back to the last set immediate value. However this problem is recognized by the developers, and a 'patch' called cancelAndHoldAtTime on its way.

So, on the one hand, the API is quite low-level, no smoothening of buffer connections, on the other hand, the programmer does not have much grip on this either, because he can not obtain the parameter value during an automation event. I will now take a look at the automation events, and see if there is a solution for our problem.

There are 3 methods on the AudioParam for gradually changing of value:

  1. linearRampToValueAtTime and exponentialRampToValueAtTime. These work the same way, except for the ramp curve.
  2. setTargetAtTime. This wurks differently, wich is demonstrated by the tests to the right. It ramps the frequency of an oscillator node from 100 to 100 Hz and back in 3 seconds by pressing the +/- buttons.

When a button is pressed during the frequency change, you will notice that with linearRamp the ramp is not interrupted, and the next ramp does not start before the former has finished. With setTarget the ramp stops immediately, and the intrinsic real-time value is automatically used as the starting pont of the next one. Therefore, if you want a continious ramp algorithm, you have to use setTargetAtTime. The disadvantage is that ramping takes much longer then the specified 3 seconds. You can hear the frequency still changing after 10 seconds! This is because the setTargetAtTime function is modelled after the time constant of a RC filter. As the diagram 'capacitor step response' on the Wiki page shows, the first 63% of the ramp is fairly steep, allmost linear, but flattens at the top. The time it takes for the ramp to rise to 63% is the timeconstant passed to the setTargetAtTime function.

When the flat top of the curve is not desired, then only the first 63% of the curve can be used for the ramp:

oscillator.frequency.setTargetAtTime(100 + (900 * 1.5823), 0, 3); oscillator.frequency.setValueAtTime(1000, context.currentTime + 3);

The first line sets the target value computed from the wanted change in frequence multiplied by 100/63, the target value will become 1582 Hz eventually, but after 3 seconds it is exactly 1000Hz. The next line schedules a setValueAtTime event after 3 seconds, this cancels the setTarget event, setting the frequency to 1000Hz. Works fine as you can test for yourself, except... when the initial ramp is interrupted by a new ramp, because setValueAtTime remains scheduled, and after 3 seconds the frequency is still set to a 1000Hz. When the ramp is interrupted inbetween, then one should use cancelScheduledValues to disable the projected setValueAtTime, but that won't work either, because it will also cancel the initial setValueAtTime, letting the frequency fall back to the initial immediate setting of the value attribute.

The only solution to the ramping problem is to do your own interpolation of the realtime value during ramping. I made an object that does just that.

class LinearRamp{ constructor(context, audioParam, value){ if(value == undefined) value = audioParam.value; this.context = context; this.param = audioParam; this.param.setValueAtTime(value, 0); this.V0 = value; this.V1 = value; this.T0 = 0; this.T1 = 0; } to(v, t){ var rv = this.V1, ct = this.context.currentTime, end = ct + t; if(this.T1 > ct){ rv = this.V0 + (this.V1 - this.V0) * (ct - this.T0) / (this.T1 - this.T0); this.param.cancelScheduledValues(0); } this.param.setValueAtTime(rv, 0); this.param.linearRampToValueAtTime(v, end); this.V0 = rv; this.V1 = v; this.T0 = ct; this.T1 = end; } }

The AudioParam is passed to the constructor, with the context and an optional initial value. The 'to' method tests if currentTime is before the endtime of the previous ramp, if so it interpolates the value and cancels the previous event. Then the value is set immediately using setValueAtTime. The same principle can be used for exponentialRampToValueAtTime and setTargetAtTime, only the formula differs. Try the result in the test to the right.

The final test is the keyboard again, but now with an envelope that is controlled by the LinearRamp object.

I had a nice time fooling around with the webaudio Api, but to answer the question; is it good enough for professional use? Not by far in my opinion. Synthesized sounds just produce too much unwanted noises. I performed tests on a slow system, a 4 year old Thinkpad 3051, and glitches become noticeable during play when the mousecursor is moved at the same time. The fact that it is not easy to continue the gradual change of a parameter value after interruption seems to indicate that the designers of the API did'nt think too much about user interaction. I also tried playing sounds using samples and that sounds ok. So if you want to play sounds on your website my advice would be to use prerecorded samples and not synthesized sounds. But who knows, in a few years time maybe, the webaudio programmers will succeed in making the webaudio API sound like music.

Another shortcoming of the API in my opinion is the inability for connection with additional user input devices, eg gamepads or a keyboard. Imagine that someone would want to make an application for piano study. Midi keyboards that connect to the USB port of a computer are widely available. But there is no API that makes it possible to connect your app with a Midi keyboard, there is no support for Midi at all.

440 Hz Oscillator
Keyed Oscillator
Keyboard
Keyboard with Envelope
linearRampToValueInTime
setTargetAtTime
setTargetAtTime 63%
Interpolated Linear Ramp
Keyboard with Envelope + interpolation
©2018 vdVeen.net