p5.speech
created for p5.js.
written by R. Luke DuBois
The ABILITY lab
New York University
p5.speech is a JavaScript library that provides simple, clear access to the Web Speech and Speech Recognition APIs, allowing for the easy creation of sketches that can talk and listen.
Simple example (Synthesis):
var foo = new P5.Speech(); // speech synthesis object
foo.speak('hi there'); // say something
Simple example (Recognition):
var foo = new P5.SpeechRec(); // speech recognition object (will prompt for mic access)
foo.onResult = showResult; // bind callback function to trigger when speech is recognized
foo.start(); // start listening
function showResult()
{
console.log(foo.resultString); // log the result
}
EXAMPLES:
Speech synthesis:
Simple speech synthesis (source)
Speech box example (uses p5.dom) (source)
Callback example (source)
Speech recognition (requires Chrome):
Simple speech recognition (source)
Continuous recognition example (source)
DOWNLOAD:
Download latest copy of the library.
View on Github.
REFERENCE:
p5.Speech
- constructor
default_voice: optional argument to set the default synthesizer voice by number (see listVoices()) or by name.
- methods
cancel(): silently cancels the current utterance and clears any queued utterances.
listVoices(): debugging statement. Lists available synthesis voices to the JavaScript console.
pause(): pause the current utterance. The onPause() callback will fire.
resume(): resumes the current utterance. The onResume() callback will fire.
setLang(language): sets the language interpreter for the synthesizer voice. Argument is BCP-47; Default is 'en-US'.
setPitch(pitch): sets playback pitch of synthesized speech from 0.01 (very low) to 2.0 (very high). Default is 1.0; not supported by all browser / OS combinations.
setRate(rate): sets rate of speech production from 0.1 (very slow) to 2.0 (very fast). Default is 1.0; not supported by all browser / OS combinations.
setVoice(voice): sets synthesizer voice by number (see listVoices()) or by name; equivalent to the default_voice parameter passed with the constructor.
setVolume(volume): sets synthesizer volume in the range of 0.0 (silent) to 1.0 (default=max volume).
speak(utterance): instructs the synthesizer to speak the string encoded in utterance. Depending on the interrupt property, additional calls to speak() will queue after or interrupt speech actively being synthesized. When synthesis begins, the onStart() callback will fire; when synthesis ends, the onEnd() callback will fire.
stop(): stops the current utterance. The onEnd() callback will fire.
- properties
interrupt: boolean to set whether the speak() method will interrupt (true) or queue after (false = default) existing speech currently being synthesized.
onEnd: function sets callback to fire when an utterance is finished.
onLoad: function sets callback to fire when synthesis voices are loaded.
onPause: function sets callback to fire when an utterance is paused.
onResume: function sets callback to fire when an utterance is resumed.
onStart: function sets callback to fire when synthesis is begun.
p5.SpeechRec
- constructor
default_language: optional argument to set the default BCP-47 language / region to for the speech recognition system.
- methods
start(): instructs the speech recognition system to begin listening. use continuous mode rather than multiple calls to start() for multiple recognition tokens within the same site.
- properties
continuous: boolean to set whether the speech recognition engine will give results continuously (true) or just once (false = default).
interimResults: boolean to set whether the speech recognition engine will give faster, partial results (true) or wait for the speaker to pause (false = default).
onEnd: function sets callback to fire when speech recognition ends.
onError: function sets callback to fire when an error occurs on the client side in recording and transmitting the speech.
onResult: function sets callback to fire when synthesis engine has reported a result.
onStart: function sets callback to fire when speech recognition has begun.
resultConfidence: float value (0.0-1.0) representing the confidence level of the speech synthesizer that resultString is what was actually spoken by the user.
resultJSON: JSON object containing a full set of data returned by the speech recognition system.
resultString: String containing the most recently detected speech.
resultValue: boolean value containing a status flag reported by the server (true = speech successfully recognized).