Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding gesture to wake word detection - is there a better way? #249

Open
mjhewitt1 opened this issue Aug 23, 2023 · 2 comments
Open

Adding gesture to wake word detection - is there a better way? #249

mjhewitt1 opened this issue Aug 23, 2023 · 2 comments

Comments

@mjhewitt1
Copy link

I am looking to add a gesture to the wake word detection process. The gesture recognition happens on a smart watch and when a gesture is recognised, a 1 is sent over esp now to the ESP-BOX.

To make it so that the ESP-BOX only wakes up if the wake word is detected AND a gesture is recognised, I have edited the audio_recorder_update_state() function in willow/deps/esp-adf/components/audio_recorder/audio_recorder.c so that the state only changes from IDLE if the wakeword is detected && gest_now == 1. I have added another function to audio_recorder.c so that the value of gest_now is updated when data is received from the watch device. The changes are outlined below.

It currently works, however I am wondering if you know of a better way to do this whereby I do not need to alter the esp-adf library? Thanks :)


willow/deps/esp-adf/components/audio_recorder/audio_recorder.c - additions:

typedef struct struct_message {
uint8_t wake;
} struct_message;
struct_message gesture;

uint8_t gest_now;

void onDataReceived(const uint8_t *macaddr, const uint8_t *data, int dataLen) {
memcpy(&gesture, data, sizeof(gesture));
gest_now = gesture.wake;
ESP_LOGE(TAG, "GEST NOW %d", gest_now);
}

static void audio_recorder_update_state(audio_recorder_t *recorder, int event)
{
switch (recorder->state) {
case RECORDER_ST_IDLE: {
if (event == RECORDER_EVENT_WWE_DECT && gest_now == 1) {
…….
}
break;
}
…………..
}

willow/main/main.c - additions

void app_main(void)
{
….
if (esp_now_init() != ESP_OK) {
ESP_LOGI(TAG, "ESP_NOW_INIT FAILED");
return;
}
….
}

@kristiankielhofner
Copy link
Contributor

Thanks, this is very interesting!

Generally speaking we try to avoid patching esp-adf wherever possible. @stintel is currently on vacation but we'll take a look at this when he gets back; we may be able to come up with a way to implement this in a more generic fashion (and potentially without patching esp-adf).

We're looking at the ability to programatically trigger wake start/stop and vad start/stop with esp-adf which would likely enable functionality like this without patching esp-adf for each use case.

Stay tuned!

@mjhewitt1
Copy link
Author

Thanks for the response! Sounds good - not having to patch would be ideal. Will stay tuned :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants