JavaScript Spectrogram

JavaScript Spectrogram Library

This article describes spectrogram displays, both waterfall and rasterscan styles and represents the user guide for the Spectrogram JavaScript library. This library facilitates the building spectrogram displays for web pages. All the displays on this page were created using the Spectrogram-2v00.js library.

Spectrogram basics

The Spectrogram shows the behavior of a signal's spectrum as a function of time. They are commonly used in the fields of music, sonar, radar, speech processing and the like. A spectrogram is formed by plotting each successive spectrum as a line of pixels, the pixel colors representing signal amplitude in each frequency bin. The pixel data from each newly arrived spectrum is plotted adjacent to the preceding line so that a rectangular image of the signal's behavior builds up over time.

Spectrograms are instances of the more general 3D displays Rasterscan or Waterfall. A spectrogram has X,Y dimensions of frequency and time but waterfall and raster scans can display quite arbitrary units on each axis. This article provides JavaScript software for generating both Waterfall and Rasterscan displays.

An example of a spectrogram can be seen on the right, just press the "START DEMO" button. The spectrogram is in Waterfall format displaying the spectra of a audio track. The spectra are generated using the Web Audio API. The AudioContext instance takes the left channel of the audio track and routes it to a Web Audio Analyser node which generates the frequency spectrum of the latest audio data at each call to the node's getByteFrequencyData method. The JavaScript array holding the returned spectrum becomes the input buffer for a Waterfall display.

The code snippet below demonstrates the use of the Spectrogram module. The code shows the constructor of an object that creates a Waterfall object. It has 'begin' and 'halt' methods to start and stop the regular waterfall display updates.

function AudioSpectrogram(analyserNode, cvsID)
{
  const frqBuf = new Uint8Array(analyserNode.frequencyBinCount); // 1024
  const wfNumPts = 50*analyserNode.frequencyBinCount/128; // 400 +ve freq bins
  const wfBufAry = {buffer: frqBuf};
  const wf = new Waterfall(wfBufAry, wfNumPts, wfNumPts, "right", {});    
  const canvas = document.getElementById(cvsID);
  const ctx = canvas.getContext("2d");

  this.playing = false;
  this.begin = ()=>{
    wf.start();
    this.playing = true;
    this.drawOnScreen();
  };

  this.halt = ()=>{
    wf.stop();
    this.playing = false;
  };

  this.drawOnScreen = ()=>{
    analyserNode.getByteFrequencyData(frqBuf, 0);
    ctx.drawImage(wf.offScreenCvs, 0, 0);
    if (this.playing) requestAnimationFrame(this.drawOnScreen);
  };
}

Waterfall displays

The basic waterfall display is aptly named as the data flows down the screen as shown in Fig 1. The effect comes about by shifting the block of previously stacked spectra down the screen by one line and the new line of data is added at the top, the oldest line of data disappearing off the bottom of the display.

Building the waterfall display is a two step process. The raw waterfall is built off-screen by directly writing each pixels color into the memory buffer of an off-screen canvas. Each data point is represented as a single pixel and successive lines are writing as adjacent rows of pixels. This method necessarily fixes the waterfall dimension to the length of the data vectors to be plotted and to the number of data lines in the display. The second step is to use this off-screen canvas as a source for drawing an image of the Waterfall in the on-screen canvas using the canvas.writeDataImage method thus allowing on-the-fly scaling of the data for presentation in any required size.

Test Signal Specs
Type:         Linear frequency sweep
Minimum freq: 500Hz
Maximum freq: 1000Hz
Sweep time:   2sec
Background:   White noise
Signal Processing
Sample frequency:   8192 smp/sec
FFT length:         512 samples
Spectral line rate: 16/sec (real time)
Spectrum period:    62.5 msec

The waterfall display in Fig. 1 shows the first 200 frequency points plotted as a function of time. The current time and newest data is always at the top of the screen. The line rate is set at 40 line/sec and the raw waterfall is set to holds 180 lines, the oldest line, written 4.5 sec ago, is at the bottom. The signal is a simulated FFT of a frequency sweep signal in a background of white noise (set inset panel for details). To allow comparisons among the different style displays, all the following spectrograms use these same input data.

The other possible styles of waterfall display are shown in Fig 2. Fig 2a shows data scrolling upward ie. the oldest data is at the top of the screen and the newest data at the bottom. Spectrogram are often shown scrolling the data horizontally. Fig 2b shows a horizontal waterfall scrolling to the right and Fig 2c shows one scrolling to the left.

Other waterfall display styles

Rasterscan displays

Rasterscan displays plot each new line of data adjacent to the previous line progressing across or down (or up) the screen. The existing data is not moved. When the input point reaches the far end of the display area it jumps back to the starting edge and proceeds down or across the display again, each new line overwriting the oldest line on the screen. The rasterscan shows a fixed number of lines representing a fixed period of the most recent signal data.

Fig 3 shows several rasterscan displays all using the same data as Fig 1. Note: To make clear where the current time is on the display, a white line is drawn in front of the most recent line added.

RasterImage

The RasterImage object can be used to create 2D displays that don't have time as an axis. The code will accept any data generated or stored as an array of arrays of data points. Such data typically come from beamforming instruments: radars, sonars and radio telescopes. The rasterscan generator can display these signals as a color coded, rectangular image of signal intensity representing the 2D area scanned by the instrument.

To create this type of display first create the data as an array of lines of data points where each data point represents the index into a color map. Then create the RasterImage object passing a reference to the data array along with the number of lines and the length of each line of data. The color map can be loaded as as one of the optional parameters. A default color map is provided, it has 256 color entries so data indices should be in the range 0..255 when using it. The RasterImage created is an HTML canvas. This can be imported directly into a web page DOM or used as input to a graphics library drawing method.

The code snippet to create Fig 4 is shown below.

// make a 2D data array with some Gaussian blobs
for (let r=0; r<rows; r++)
{
  sigData[r] = new Float32Array(buf, r*cols*4, cols);
}
// now add a few Gaussian blobs
addGaussian(sigData, rows, cols, 20, 30, 10);
addGaussian(sigData, rows, cols, 60, 70, 1);
addGaussian(sigData, rows, cols, 50, 55, 4);
addGaussian(sigData, rows, cols, 83, 12, 5);

// make a canvas object holding the color display of the data
const rasterImg = new RasterImage(sigData, cols, rows, {dir:"up"}); 

...  // set up Cango to display the RasterImage on screen

const dataPlot = new Img(rasterImg, {imgWidth:xmax-xmin, imgHeight:ymax-ymin, lorg:7});
// now draw the image
cgo9.render(dataPlot);
  

JavaScript Spectrogram code

The Spectrogram-2v00.js library supplies JavaScript constructors for Waterfall, Rasterscan and RasterImage objects to create spectrogram displays for display in web pages.

Both Waterfall and Rasterscan objects have an 'offscreenCvs' property which is a reference to the raw spectrogram they maintain as an offscreen canvas. Application code can use spectrogram.offscreenCvs as the source for the canvas.drawImageData method to render the spectrogram to an on-screen canvas element with 'on-the-fly' rescaling to an arbitrary size and aspect ratio.

The default color map used to convert the input signal is a 256 level map with colors similar to the popular Matlab 'Jet' colormap[2], but a user defined colormap can be supplied with as few as 2 or as many as 16 million colors using the 'setColorMap' method or passing the colorMap as the 'colorMap' property of the constructor's 'options' parameter.

The raw spectrogram may be built up at a fixed line rate along using the 'setLineRate', 'start' and 'stop' methods. Alternatively, the 'newline' method is provided so the spectrogram can be built up synchronously with the arrival of a new array of signal data.

Waterfall constructor

Syntax:

var wfall = new Waterfall(ipObj, ptsPerLine, lines, direction, options);

Description:

Creates a new Waterfall object which comprises an offscreen canvas property 'offscreenCvs' into which the contents of ipObj.buffer are written at a fixed rate. This offscreen canvas is available as a source for drawing an image of the waterfall into an on-screen canvas element.

Lines of data are added at a uniform rate set by the property 'lineRate'. Each new line is always written into the same line at the top, side or bottom of the display all the previous lines are copied as a block and moved one line in the direction set by the 'direction' property creating a waterfall effect. The number of data points plotted for each line is set by the 'pxPerLine' parameter and the total number of lines in the canvas is set by the 'lines' parameter. The line rate and other user parameters may be set as key-value pairs in the 'options' object. If not been set as an optional parameter or a call to setLineRate the default lineRate of 30 line/sec is used.

Parameters:

Returns:

Waterfall Object:

This object's 'offScreenCvs' property holds the output canvas display (see Waterfall and Rasterscan common property).

This object has the methods described below (see Waterfall and Rasterscan common methods).

Example:

Here are some code snippets showing the basic use with the input buffer array used as a double buffer for efficiency.

function getDynamicDataBuffer(dataGen)   // returns a pointer to the dynamic data buffer
{
  // create double buffer for the data can be any type but values will be assigned to Uint8
  const bufferAry = [];
  ...
  function genDynamicData()
  {
    dataGen.getLine(bufferAry[1]);  // dataGen gathers the data
    // swap the buffers
    let tmpBuf = bufferAry[0];
    bufferAry[0] = bufferAry[1];
    bufferAry[1] = tmpBuf;
    // repeat forever
    if (playing) 
    {
      setTimeout(genDynamicData, linePeriod);
    } 
  }

  bufferAry[0] = new Float32Array(dataGen.fftLen);  // values should be 0..255
  bufferAry[1] = new Float32Array(dataGen.fftLen);  // ensure length is set before returning pointer
  // start the repeating loop
  genDynamicData();

  return {buffer: bufferAry[0]};  // value of the pointer will change
}

const dataObj = getDynamicDataBuffer(); 
const wf = new Waterfall(dataObj, wfLines, wfPts, "DOWN", {startbin: fftLen/2, lineRate: 30});

Rasterscan constructor

Syntax:

var raster = new Rasterscan(ipObj, width, height, direction, options);

Description:

Creates a new Rasterscan object which comprises an offscreen canvas with input signal data from the ipObj.buffer array drawn as a line of colored pixels. This offscreen canvas is available as a source for drawing an image of the waterfall into an on-screen canvas element.

Lines of data are added at a uniform rate set by the property 'lineRate'. Each line of data is drawn adjacent to the previous line so the new data progresses across the canvas in the direction specified by the 'direction' parameter. When the input reaches the last line of pixels set by the 'lines' parameter the input point jumps back the the first line. The input point is marked by a line of white pixels being drawn in front of the new data (erasing the oldest data line).

Parameters:

Returns:

Rasterscan Object:

This object's 'offScreenCvs' property holds the output canvas display (see Waterfall and Rasterscan common property).

This object has the methods described below (see Waterfall and Rasterscan common methods).

Waterfall and Rasterscan constructor 'options' properties

Waterfall and Rasterscan objects' common property

Waterfall and Rasterscan objects' common methods

Canvas direct pixel writing

The HTML canvas element has an ImageData property which allows direct read and write access to the pixel data.

ImageData object

Syntax:

cvs = document.getElementByID(canvasID);

imgData = cvs.getImageData();

Properties:

Using a Color Map

To set the color of a pixel to indicate a signal level we must map the signal value to a color, this color will be represented by 4 color component vales red, green, blue and alpha (transparency) values. If the signal can be well described by say 20 levels then then each level can be assigned its own color, by an r,g,b and an a value. These RGBA values can be stored in a 4 element array and then 20 of these 4 element arrays makes up a colormap. The example below assumes that each color is completely opaque so the alpha value for each color is set to 255.

  var colorMap = [[level0.r, level0.g, level0.b, 255],   // color for level 0
                  [level1.r, level1.g, level1.b, 255],   // color for level 1
                  ... 
                  [level19.r, level19.g, level19.b, 255]];  // color for level 19

To set the color of a pixel x, y to correspond to a signal level of say 's' (a value between 0 and 19) the code would be:

  var i = = (y * canvasWidth + x) * 4;
  imgData.data[i] = colMap[s][0];    // red component
  imgData.data[i+1] = colMap[s][1];  // green component
  imgData.data[i+2] = colMap[s][2];  // blue component
  imgData.data[i+3] = colMap[s][3];  // alpha  = 255

  // modified imgData is then written back to the canvas context using putImageData(imgData, 0, 0)

Jet Colormap

A very popular colormap often used in signal processing applications is the Matlab Jet colormap which maps low level signal levels to green-blue colors and increasingly high signal levels to green, yellow and through orange to red at the highest level. The formula for generating the colormap are described here. So a color map with any number of input signal levels can be created.

The example that follow will use a 256 level Jet color map which has been saved as a JSON file JetColormap256.JSON

Canvas ImageData example

As an example of using these directly draw methods Fig 1 shows a sine wave signal with its amplitude mapped to colors and these RGBA values written directly into a canvas image data. The full source code is shown on the left of Fig 1.

function testImageData(cvsID) {
  const canvas = document.getElementById(cvsID),
        ctx = canvas.getContext("2d"),
        imgObj = ctx.getImageData(0,0, canvas.width, canvas.height),
        pxPerLine = imgObj.width,
        colMap = [[  0,  0,128,255], [  0,  0,191,255], 
                  [  0,  0,255,255], [  0, 64,255,255], 
                  [  0,128,255,255], [  0,191,255,255], 
                  [  0,255,255,255], [ 64,255,191,255], 
                  [128,255,128,255], [191,255, 64,255], 
                  [255,255,  0,255], [255,191,  0,255], 
                  [255,128,  0,255], [255, 64,  0,255], 
                  [255,  0,  0,255], [191,  0,  0,255], 
                  [168,  0,  0,255]],
        colorLevels = 16,
        dataLine = [];

  // generate a line of signal data
  for (let i=0; i < pxPerLine; i++) {
    dataLine[i] = Math.floor(Math.sin(i*Math.PI/100)**2 * colorLevels);
  }
  for (let row=0; row < 30; row++) {
    for (let px = 0; px < pxPerLine; px++) {
      let i = 4*(row * pxPerLine + px);
      let rgba = colMap[dataLine[px]];  // lookup color rgba values
      
      imgObj.data[i] = rgba[0];   // red
      imgObj.data[i+1] = rgba[1]; // green
      imgObj.data[i+2] = rgba[2]; // blue
      imgObj.data[i+3] = rgba[3]; // alpha
    }
  }
  ctx.putImageData(imgObj, 0, 0);
}