MAX98357A Integration with Raspberry Pi CM5

[danic85]

As part of the proposed changes for Cody, the Raspberry Pi 5 will be replaced with a Pi Compute Module 5. This will allow a carrier board to be created that can house the core modules needed for the robot in a much smaller package.

This discussion details the potential use of the MAX98357A I2S amplifier, and will hopefully allow the module to be embedded on the carrier board directly as discrete components.

This image shows the pinout and includes an example of the MEMS mic currently used in Buddy:

GPIO pins 18, 19 & 21 are used (where 18 & 19 are shared with the other I2S device as expected).

Datasheet:

https://www.analog.com/media/en/technical-documentation/data-sheets/MAX98357A-MAX98357B.pdf

[danic85]

Details of a custom board pinout with discrete components: https://raspberrypi.stackexchange.com/questions/144434/cm4-custom-board-with-max98357

[danic85]

It has taken a while to get this component to play nicely with the INMP441 microphones, but I have an installer script (or three) that should work. I will keep testing as the installation modifies the same files which can cause conflicts.

The installers need to be executed in the following order, with multiple restarts during the process (which is why they are not a single install):

• install-mics.sh
• install-speech-recognition.sh
• install-amp.sh

The scripts may require re-running after restart, so I'll try to make that clear from the debug output.

Full commit showing changes: fbe3eae

More information

It seems as though the adafruit i2s installer scripts are a good baseline for this, and I have created my own bash scripts to do the following:

1. Install the INMP441 microphones
2. Configure the microphones to support speech recognition
3. Install the MAX98357A amplifier.

The original mic installer attempts to make and install Raspberry-Pi-Installer-Scripts/i2s_mic_module, but this resulted in a number of errors I couldn't resolve. Fortunately excluding this step hasn't caused any problems. Reference of error here

There is a line added to /etc/asound.conf dtoverlay=max98357a on the original install script, but this appears to block the microphones so they no longer appear in arecord -l. Removing this line resolves this and doesn't appear to impact the amplifier.

Testing

I have tested the speakers using speaker-test -l5 -c2 -t wav -D plughw:0 where plughw is the card number from aplay -l. Occasionally after install the card number is shown as 2, but a restart will reset it to the default of 0.

cm5@cm5:~/modular-biped $ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: sndrpigooglevoi [snd_rpi_googlevoicehat_soundcar], device 0: Google voiceHAT SoundCard HiFi voicehat-hifi-0 [Google voiceHAT SoundCard HiFi voicehat-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: vc4hdmi0 [vc4-hdmi-0], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: vc4hdmi1 [vc4-hdmi-1], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Testing the microphones and speech recognition can be done by enabling the speechinput module in the yaml and enabling run-on-boot in that same file's config. You can also quickly check that the microphones are accessible using arecord- l

cm5@cm5:~/modular-biped $ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: sndrpigooglevoi [snd_rpi_googlevoicehat_soundcar], device 0: Google voiceHAT SoundCard HiFi voicehat-hifi-0 [Google voiceHAT SoundCard HiFi voicehat-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

[danic85]

Scratch that. On a second attempt this doesn't work well at all.

I had an issue with the CM5 and ended up reinstalling the bootloader and re-flashing the eMMC, but this time using the install scripts hasn't been successful. I can consistently get the mics working but the amp module will break the mics and then not work itself either....

Fun!

[danic85]

A little dated but I'll take a look at this as a possible improvement adafruit/Raspberry-Pi-Installer-Scripts#322

Also this
https://forums.raspberrypi.com/viewtopic.php?t=189152

[EngineerJoe2016]

So just to ask, do you have any data for the failures? Like error logs or any output files? I started to look into it but I wanted to see if there is anything I could review.

[danic85]

No unfortunately not, besides it not working the only thing that changes is that the arecord - l is empty after I apply the amp install.

I haven't looked at the above links yet but I'll get to it soon.

[danic85]

I have been trying a few things to resolve this including re-running the mic installer, and even creating a modified amp installer (I didn't keep this, but here it is):

#!/bin/bash

set -e

# Variables
BLACKLIST="/etc/modprobe.d/raspi-blacklist.conf"
PRODUCT_NAME="I2S Amplifier and Microphone"
FLAG_FILE="i2s_amp_mic_installed.flag"
CONFIG="/boot/config.txt"
ALT_CONFIG="/boot/firmware/config.txt"
OVERLAY_NAME="dual_i2s"
OVERLAY_FILE="/boot/overlays/${OVERLAY_NAME}.dtbo"

# Function to check if a command exists
command_exists() {
    command -v "$1" >/dev/null 2>&1
}

# Function to prompt user
prompt() {
    read -p "$1 [y/N]: " response
    case "$response" in
        [yY][eE][sS]|[yY]) 
            return 0
            ;;
        *)
            return 1
            ;;
    esac
}

# Function to check if running on Raspberry Pi
is_raspberry_pi() {
    if [ -f /proc/device-tree/model ]; then
        grep -q "Raspberry Pi" /proc/device-tree/model
    else
        return 1
    fi
}

# Function to write a file
write_file() {
    echo -e "$2" > "$1"
}

# Ensure running as root
if [ "$EUID" -ne 0 ]; then
    echo "Please run as root."
    exit 1
fi

# Check for installer flag
if [ -f "$FLAG_FILE" ]; then
    echo "Installer has already been run. Exiting."
    exit 0
fi

# Ensure running on Raspberry Pi
if ! is_raspberry_pi; then
    echo "Non-Raspberry Pi board detected. Exiting."
    exit 1
fi

# Determine config file
if [ -f "$ALT_CONFIG" ]; then
    CONFIG="$ALT_CONFIG"
fi

if [ ! -f "$CONFIG" ]; then
    echo "No Device Tree detected. Not supported."
    exit 1
fi

# Create and compile DTS file for dual soundcard
echo "Creating and compiling DTS file for dual I2S soundcard."
DTS_FILE="${OVERLAY_NAME}.dts"
write_file "$DTS_FILE" "/dts-v1/;
/plugin/;

/ {
    compatible = \"brcm,bcm2708\";

    fragment@0 {
        target-path = \"/\";
        __overlay__ {
            dual_soundcard: dual-soundcard {
                compatible = \"simple-audio-card\";
                simple-audio-card,name = \"Dual I2S\";
                status=\"okay\";

                capture_link: simple-audio-card,dai-link@0 {
                    format = \"i2s\";
                    r_cpu_dai: cpu {
                        sound-dai = <&i2s>;
                    };
                    r_codec_dai: codec {
                        sound-dai = <&codec_in>;
                    };
                };

                playback_link: simple-audio-card,dai-link@1 {
                    format = \"i2s\";
                    p_cpu_dai: cpu {
                        sound-dai = <&i2s>;
                    };
                    p_codec_dai: codec {
                        sound-dai = <&codec_out>;
                    };
                };

            };
        };
    };

    fragment@1 {
        target-path = \"/\";
        __overlay__ {
            codec_out: spdif-transmitter {
                #address-cells = <0>;
                #size-cells = <0>;
                #sound-dai-cells = <0>;
                compatible = \"linux,spdif-dit\";
                status = \"okay\";
            };
            codec_in: spdif-receiver {
                #address-cells = <0>;
                #size-cells = <0>;
                #sound-dai-cells = <0>;
                compatible = \"linux,spdif-dir\";
                status = \"okay\";
            };
        };
    };

    fragment@2 {
        target = <&i2s>;
        __overlay__ {
            #sound-dai-cells = <0>;
            status = \"okay\";
        };
    };
};"

dtc -@ -I dts -O dtb -o "$OVERLAY_FILE" "$DTS_FILE"
rm "$DTS_FILE"
echo "Overlay compiled and saved to $OVERLAY_FILE."

# Modify config.txt to use the overlay
if ! grep -q "^dtoverlay=$OVERLAY_NAME" "$CONFIG"; then
    echo "dtoverlay=$OVERLAY_NAME" >> "$CONFIG"
    REBOOT=true
fi

# Configure ALSA
echo "Configuring ALSA for dual soundcard."
ALSACONF="""
# Dual I2S Soundcard Configuration
pcm.amplifier {
   type hw
   card 0
   device 0
}

pcm.microphone {
   type hw
   card 0
   device 1
}
"""
if [ ! -f "/etc/asound.conf" ]; then
    write_file "/etc/asound.conf" "$ALSACONF"
else
    echo "Warning: /etc/asound.conf already exists. Please check manually to avoid conflicts."
fi

# Test both playback and capture
echo "Testing playback (amplifier) and capture (microphone)."
if command_exists "speaker-test"; then
    echo "Testing playback..."
    speaker-test -Dplug:amplifier -c2 -t sine -l1 || echo "Playback test failed."
fi
if command_exists "arecord"; then
    echo "Testing capture..."
    arecord -Dplug:microphone -f cd -d 5 test.wav || echo "Capture test failed."
    echo "Recording saved as test.wav"
fi

# Final message
echo "Installation complete! A reboot is required to apply changes."
touch "$FLAG_FILE"
if prompt "Reboot now?"; then
    reboot
fi

The mics work, but the speaker seems to hit a 'device busy' error when I test them with speaker-test

Playback open error: -16,Device or resource busy

Running sudo fuser -k /dev/snd/* to force release of audio devices seems to resolve the 'device busy' issue on the amp when it occurs. I also had to check that /etc/asound.conf was referencing the right device:

# I2S Amplifier Configuration
pcm.speakerbonnet {
   type hw card 0
}

Card should match the card number in aplay -l (in my case this is 2, not 0). UPDATE: I'm not sure how relevant this is as this is currently misconfigured and the below test is working...

Testing has been difficult as the device configuration is changing after restart. I managed to do a complete test of speech recognition and echoing out via TTS by adding the speech recognition module to the TTS module main execution:

if __name__ == '__main__':
    # tts = TTSModule()
    # tts.speak('Test') # broken currently, thinks espeak-ng is not installed
    
    tts2 = TTSModule(service='elevenlabs', voice_id='pMsXgVXv3BLzUgSXRplE')
    tts2.speak('Test')
    
    import speechinput as speechinput
    speech = speechinput.SpeechInput(device_name='pulse', start_on_boot=True)
        

The device name is 'pulse' in this case as the 'lp' device was not present. But this allowed me to echo back the speech heard by adding pub.sendMessage('tts', msg='I heard ' + val.lower()) to the appropriate place in the speech recognition module.

Furthermore, running elevenlabs from a module (or even just hardcoding a module to run) results in an error:

Traceback (most recent call last):
  File "/home/cm5/modular-biped/main.py", line 102, in <module>
    main()
  File "/home/cm5/modular-biped/main.py", line 29, in main
    module_instances = loader.load_modules()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/cm5/modular-biped/module_loader.py", line 71, in load_modules
    instance = getattr(mod, module_name)(**instance_config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cm5/modular-biped/modules/audio/ttsmodule.py", line 34, in __init__
    self.init_elevenlabs(self.voice_id)
  File "/home/cm5/modular-biped/modules/audio/ttsmodule.py", line 67, in init_elevenlabs
    self.ttsclient = elevenlabs.ElevenLabs(
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cm5/modular-biped/myenv/lib/python3.11/site-packages/elevenlabs/client.py", line 75, in __init__
    super().__init__(
  File "/home/cm5/modular-biped/myenv/lib/python3.11/site-packages/elevenlabs/base_client.py", line 102, in __init__
    else httpx.Client(timeout=_defaulted_timeout, follow_redirects=follow_redirects)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Client.__init__() got an unexpected keyword argument 'follow_redirects'

This error doesn't happen if I run the module directly myenv/bin/python3 modules/audio/ttsmodule.py, but any time it is referenced from another file (main.py etc) the above error occurs. Dependencies are up to date and not different based on the execution method. So I am confused.

Definitely a lot of work to be done in the code but at least the hardware is capable of using both.
The above is more of a hack to get it tested than a fix, so I'll need to clean this process up and find a way to reliably install both modules on a pi in order to release this.

Current config for above successful test:

aplay -l:

**** List of PLAYBACK Hardware Devices ****
card 0: vc4hdmi0 [vc4-hdmi-0], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: sndrpigooglevoi [snd_rpi_googlevoicehat_soundcar], device 0: Google voiceHAT SoundCard HiFi voicehat-hifi-0 [Google voiceHAT SoundCard HiFi voicehat-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: vc4hdmi1 [vc4-hdmi-1], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

arecord -l

**** List of CAPTURE Hardware Devices ****
card 1: sndrpigooglevoi [snd_rpi_googlevoicehat_soundcar], device 0: Google voiceHAT SoundCard HiFi voicehat-hifi-0 [Google voiceHAT SoundCard HiFi voicehat-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

I ran the mic installer again most recently, and then restarted. The device numbers seem to change on restart, which doesn't help with hardcoding configurations.

speaker-test -l5 -c2 -t wav -D plughw:1 works to demo speaker

On restart playback cards 1 and 2 swap, but the above test still works and speaker-test works if plughw is set to 2.

[danic85]

Update: The error unexpected keyword argument 'follow_redirects' above was caused by a duplicate version of httpx being installed on my pi in the local directory modular-biped. It was an earlier version which didn't support that method call.

This was because I had previously installed modules outside of the virtual environment as part of the viam config steps. I have since archived those dependency files and that small issue, at least, is fixed.

[danic85]

I definitely find that running the mic installer fixes a few problems, but none of this is particularly clean.

I've pushed a version of the feature/cm5 branch that allows for voice recognition, GPT completion, and TTS output via elevenlabs. Tested as working on the CM5.
The speech recognition seems to stop a little early so it only listens to half a phrase sometimes, but that is adjustable in code.