The Hermes protocol specifies the format of MQTT messages for a voice assistant platform.
Adapted from Snips.ai.
This documentation is currently an incomplete version of the Snips Hermes protocol.
This will activate the Wake Word component of the Snips Platform.
Accepts the following message:
{
"siteId": "default"
}
This will deactivate the Wake Word component of the Snips Platform. Consequently, nothing will be triggered when a wake word is pronounced.
Accepts the following message:
{
"siteId": "default"
}
This message will be sent by the Snips Platform when the Wake Word component has detected that a specific wake word has been uttered.
Replace with the id of your
wake word if multiple wake words are present,
or default if there is only one wake word.
Accepts the following message:
{
"siteId": "default",
"modelId": "pvporcupine",
"modelVersion": 1,
"modelType": "universal",
"currentSensitivity": 0.7
}
This will activate the ASR component, subsequently enabling to start listening for voice (using the startListening call described below).
Accepts the following message:
{}
This will deactivate the ASR component.
Accepts the following message:
{}
This will explicitly tell the ASR component to start listening for voice input.
Accepts the following message:
{
"siteId": "default",
"sessionId": "string"
}
This will explicitly tell the ASR component to stop listening for voice input.
Accepts the following message:
{
"siteId": "default",
"sessionId": "string"
}
When the ASR is listening, it transcribes voice to text in real time. This process stops when a longer period of silence is detected, after which the transcription results are posted, as described here.
Accepts the following message:
{
"text": "turn on the lights",
"likelihood": 0.85,
"seconds": 2.1,
"siteId": "default",
"sessionId": "string"
}
In order to extract an intent and slots
from a piece of text, send it directly to
the NLU component as follows. The NLU
component will subsequently publish a message
to the hermes/nlu/intentParsed topic,
described below.
Accepts the following message:
{
"input": "turn on the lights",
"intentFilter": [
"turnOnLights",
"turnOffLights"
],
"id": "string",
"sessionId": "string"
}
When text has been sent to the NLU, the result of the intent resolution is sent back as follows.
Accepts the following message:
{
"id": "string",
"input": "turn on the lights",
"intent": {
"intentName": "lights.on",
"confidenceScore": 0.999
},
"slots": [
{
"confidence": 0.859,
"raw_value": "living room",
"value": "Living Room",
"entity": "room",
"slotName": "location",
"range": {
"start": 14,
"end": 25
}
}
],
"sessionId": "string"
}
When the NLU was unable to parse a chunk of text, it publishes a message telling so.
Accepts the following message:
{
"id": "string",
"input": "show me the weather",
"sessionId": "string"
}
When an error has occurred in the NLU component, it publishes a message telling so.
Accepts the following message:
{
"sessionId": "string",
"error": "string",
"context": "string"
}
Sending text to be spoken by the TTS component (low-level API)
Accepts the following message:
{
"text": "I can't do that, Dave.",
"lang": "en_US",
"id": "string",
"siteId": "default",
"sessionId": "string"
}
When TTS has finished speaking some text, it will publish a message as follows (low-level API)
Accepts the following message:
{
"id": "string",
"sessionId": "string"
}
You may send a WAV sound to be played on a specific site as follows.
The site on which to play the sound
An id to be passed back on
hermes/audioServer/{siteId}/playFinished
Accepts the following message:
The WAV file to play.
When the audio service has finished playing a sound, it publishes a message as follows.
The site on which the sound was played
Accepts the following message:
{
"id": "string",
"sessionId": "string"
}
You can also stream the sound you want to play instead of sending it all on one go. This is for example used by the TTS service to start playing the generated voice before the end of its generation.
The site on which to play the sound
The id to be passed back to
hermes/audioServer/{siteId}/streamFinished
Number of the send chunk, starting at 0
0, except for last chunk, where it is 1
Accepts the following message:
The chunk, encoded as WAV
When the audio service has finished streaming a sound, it publishes a message as follows.
The site on which the stream was played
Accepts the following message:
{
"id": "string",
"siteId": "string"
}
Every time the platform captures an audio frame, it publishes a message as follows.
The site on which the sound frame was captured
Accepts the following message:
The WAV of the frame.
This is the main message the handler code should subscribe to. It is sent by the Dialogue Manager when an intent has been detected.
Note that it is the handler's responsibility to inform the Dialogue
Manager of what it should do with the current session by sending
either a Continue Session or an End Session with the current sessionId.
The name of the detected intent
Accepts the following message:
{
"sessionId": "string",
"customData": "string",
"siteId": "default",
"input": "turn on the lights",
"intent": {
"intentName": "lights.on",
"confidenceScore": 0.999
},
"slots": [
{
"confidence": 0.859,
"raw_value": "living room",
"value": "Living Room",
"entity": "room",
"slotName": "location",
"range": {
"start": 14,
"end": 25
}
}
],
"asrTokens": [
{
"value": "string",
"confidence": 0,
"rangeStart": 10,
"rangeEnd": 15,
"time": {
"start": 2.53,
"end": 2.77
}
}
],
"asrConfidence": 0.876,
"alternatives": [
{
"intentName": "string",
"confidenceScore": 0.65,
"slots": [
{
"confidence": 0.859,
"raw_value": "living room",
"value": "Living Room",
"entity": "room",
"slotName": "location",
"range": {
"start": 14,
"end": 25
}
}
]
}
]
}
You can send this message to programmatically initiate a new session. The Dialogue Manager will start the session by asking the TTS to say the text (if any) and wait for the answer of the end user.
Accepts the following message:
{
"siteId": "default",
"init": {
"type": "action",
"text": "Hello",
"canBeEnqueued": false,
"intentFilter": [
"lights.on",
"lights.off"
],
"sendIntentNotRecognized": true
},
"customData": "string"
}
This message is sent by the Dialogue Manager when it receives a Start Session message and the corresponding site is busy. Only Start Session messages with a notification initialisation or an action initialisation with the canBeEnqueued flag set to true can be enqueued. When the site is free again, this session will be started.
Accepts the following message:
{
"sessionId": "string",
"siteId": "default",
"customData": "string"
}
This message is sent by the Dialogue Manager when a new session is started, either following a wakeword or programmatically.
Accepts the following message:
{
"sessionId": "string",
"siteId": "default",
"customData": "string"
}
You should send this after receiving received an Intent when you want to continue the session for example to ask additional information to the end user.
Be sure to use the same sessionId as the one in the Intent message.
Accepts the following message:
{
"sessionId": "string",
"text": "What light should I turn on?",
"intentFilter": [
"lights.on",
"lights.off"
],
"customData": "string",
"sendIntentNotRecognized": true,
"slot": "string"
}
You should send this after receiving received an Intent when you want to end the session. Be sure to use the same sessionId as the one in the Intent message.
Accepts the following message:
{
"sessionId": "string",
"text": "string"
}
This message is sent by the Dialogue Manager when a session is ended.
Accepts the following message:
{
"sessionId": "string",
"customData": "string",
"siteId": "default",
"termination": {
"reason": "nominal"
}
}
This message is sent by the dialogue manager when it failed to
recognize and intent AND you requested the dialogue manager to
notify you of such events using the sendIntentNotRecognized flag
in the last Start Session or Continue Session
Accepts the following message:
{
"sessionId": "string",
"customData": "string",
"siteId": "string",
"input": "open the pod bay doors"
}
You can send this message to programmatically configure the scope of intents that are enabled at a given time. By default, all intents are enabled by default unless specified otherwise in the console. Refers to the dedicated Console tutorial for further details. The behaviour implemented by the intentFilter attributes for the start session and continue session locally overwrites the set of enabled/disabled intents only for the next user turn.
Accepts the following message:
{
"siteId": "string",
"intents": [
{
"intentId": "string",
"enable": true
}
]
}
The WAV file to play.
The chunk, encoded as WAV
The WAV of the frame.