ContinuousVoice.js

ContinuousVoice is a thin wrapper around the Web Speech API's speech recognition events. It exposes three new custom events: interimResults, stableResults, and finalResults. You can listen for these events and use them to trigger different behaviors.

Here are some usage examples:

Events Monitor: Visualizes what the speech events look like.
Closed Captions: Displays speech in an overlay at the bottom of the screen.
Follow Lyrics: Follows along while you read/sing lyrics.
Reading Actions: Follows along while you read and triggers action at specific points in the story.
Pattern Commands: Listens for configurable trigger patterns with optional words, alternatives, and captured parameters.
On-Device Recognition: Runs recognition locally on your device (offline and private), with an explicit install step.

ContinuousCommands.js builds upon ContinuousVoice to listen for certain key words and phrases, and trigger registered functions when the phrases are detected. Because the logic for tracking commands is a bit complicated, this was broken out into a separate class.

Command Listener: Listens for specific phrases in continuous speech events.
RevealJs Integrations: A Work in progress that exposes voice commands to presentations.

All speech events are emitted in the same format, with id storing a unique identifier for each transcript, new storing the diff between the current and previous events, and interim/stable/transcript storing variations of the speech heard.


    speechEvent = { 
        id: int,
        transcript: string,
        stable: string,
        interim: string,
        new: string,
        isInterim: bool,
        isStable: bool,
        isFinal: bool,
    }

InterimResults emit immediately, but the interim portion may contain partial words or words that could be changed or reinterpretted in future events. For Interim events, the transcript contains both the stable and interim strings concatenated together. The idea being that if you are interested in interm events you would want to act on all possbile parts. The new string contains only the new words of the interim parameter since the last event.


    <script src="./continuousVoice.js" \>

    interimSpeechEvent = { 
        id: 0,
        transcript: "all words in the transcript",
        stable: "all words in",
        interim: "the transcript",
        new: "transcript",
        isInterim: true,
        isStable: false,
        isFinal: false,
    }

    ContinuousVoice.addEventListener( 'interimResult', (interimSpeechEvent) => {
        if ( interimSpeechEvent.detail.isInterim ) {
            console.log( interimSpeechEvent.detail.interim );
        }
    });

StableResults emit a little behind the interim results and the stable portion contains results that will (most likely) persist throughout future events. For Stable events, the transcript contains only the stable string and is a duplicate of that parameter. The idea being that if you are interested in stable events you would not want to act on the interim parts. The new string contains only the new words of the stable parameter since the last event.


    <script src="./continuousVoice.js" \>

    stableSpeechEvent = { 
        id: 1,
        transcript: "all words in",
        stable: "all words in",
        interim: "the transcript",
        new: "words in",
        isInterim: false,
        isStable: true,
        isFinal: false,
    }

    ContinuousVoice.addEventListener( 'stableResult', (stableSpeechEvent) => {
        if ( stableSpeechEvent.detail.isStable ) {
            console.log( stableSpeechEvent.detail.stable );
        }
    });

FinalResults emit last and all word parameters transcript, interim, stable, and new, are the same containing the full text captured since beginning the transcript. This most simulates the experience of how Alexa works.


    <script src="./continuousVoice.js" \>

    finalSpeechEvent = { 
        id: 2,
        transcript: "all words in the transcript",
        stable: "all words in the transcript",
        interim: "all words in the transcript",
        new: "all words in the transcript",
        isInterim: false,
        isStable: false,
        isFinal: true,
    }

    ContinuousVoice.addEventListener( 'finalResult', (finalSpeechEvent) => {
        if ( finalSpeechEvent.detail.isFinal ) {
            console.log( finalSpeechEvent.detail.interim );
        }
    });