Document how to install custom DSP programs

[archi]

As the title says, the documentation on how to install custom DSP programs (that is after programming them in SigmaStudio) is currently not documented.

My current state is that I adapted BeocreateUniversal-v10.dspproj for a linear-phase crossover, but

1. I don't know how to get the actual program onto the DSP using the WebGUI, and
2. how to tell the system the new addresses for those controls I didn't remove.

Would be cool to get at least some rudimentary pointers - I've already written a parser for the various SigmaStudio output files, so I'm not eniterely new to this ;-)

Best regards, Sebastian

[tuomashamalainen]

Hi Sebastian and thanks for opening this issue.

The functionality of uploading and creating custom DSP programs is indeed lacking at the moment, as there has been little demand for it so far. I need to do this properly when I can, but to get you started, here are some tips:

• HiFiBerry have pretty good documentation on how to generate the XML files that are used to program the DSP in the following link (note: just save the XML and export system files for now, no need to use the dsptoolkit commands): https://github.com/hifiberry/hifiberry-dsp/blob/master/doc/dspprofiles.md
• Once you have written the program to EEPROM via SigmaStudio, please run dsptoolkit get-checksum command on the speaker via SSH and note down the signature. This is the fingerprint of your custom DSP program. Make sure that Remote Login (SSH) is turned on in General -> Remote Login, to be able to use SSH.
• When you have your XML and exported system files, you can find "DSP Parameter Reader" in our repository, which you can use to prepare metadata for your DSP program. You can open the HTML file in a web browser to access it (you don't need a server, just open it from its folder on your hard drive), and then drag-and-drop the generated .params file to see all the blocks in your DSP program.
• The addresses for the "stock" parts of your program should be automatically recognised by the reader and added to program metadata. Metadata can be accessed from the upper right corner.
• Remember to change the programID and profileName fields to your own to avoid any conflicts. Also make sure the correct sampling rate is listed. Finally, add in the checksum/fingerprint from the second step (it's listed as AddMe by default).
• Note that whilst the reader will show your whole DSP program, your own custom blocks will not show up in the metadata. They will be a part of the program itself and functional, rest assured. The metadata is just a way for our software to know where the settings that we can control exist in the memory.
• The generated metadata (you can copy it from the XML view) should go into the XML file that you exported from SigmaStudio. You may take a look at the existing programs to see how the metadata is placed in the file. Might also be a good idea to compare with those programs to see that all the default functionality that you intended to be there is there.
• The GUI is currently missing the function that enables you to do this, but you can upload the programs manually over SFTP. Upload your programs to /etc/beocreate/beo-dsp-programs. After uploading, run systemctl restart beocreate2 to have it show up in the list of DSP programs.
• At this point, your new program might already be shown as "active" (with a checkmark) in the DSP Programs menu. To make sure that all parts of the system (the HiFiBerry parts) know about it, you should open it up from the list and install it again with the button in the panel.
• Beocreate 2 should be smart enough to cope behind the scenes with DSP features that you took out, but the GUI might in some instances show features that actually aren't there. This area needs improvement, as I've mostly worked with our standard DSP programs.

Hopefully this can get you forward. These instructions are by no means exhaustive, so if you run into any roadblocks or questions, don't hesitate to reach out and I'll be happy to look into it further.

Best,
Tuomas from Bang & Olufsen

[archi]

Thanks Toumas, I'm giving it a try. That link to the hifiberry people is also quite useful. Also, I've to check what dsptoolkit can do.

However, is there some way to get the firmware on the DSP without programming the EEPROM via TCP? The thing is, I'm running my "daily" hifiberryos in a container (so I can use the Pi for more tasks, e.g. as a PiHole), and SigmaTCP doesn't seem to be too happy about this.

Regarding the checksum: I've written an assembler for the ADAU1452(!) DSP firmware (from the SigmaStudio export), maybe you can use something like this to produce the DSP firmware and compute the correct checksum plus the programmable firmware image? You can see the (really simple!) assembler algorithm in my repo, implemented in perl or alternatively over at the freeDSP-aurora in C++/Qt.

[tuomashamalainen]

Hi Sebastian and thanks for the response.

The way things currently are, you will need to flash the EEPROM over TCP (or USB, if you own a USBi programmer and have the appropriate pins on the board), as this is the way we can record the correct write events that can be saved to the XML, which the DSPToolkit essentially "plays back" to the DSP when you want to install a program from file on the Pi without SigmaStudio.

@hifiberry this assembler thing seems interesting but is beyond my paygrade, could you maybe have a look if it's something we could use/learn from?

Best,
Tuomas from Bang & Olufsen

[archi]

The freeDSP-aurora uses its ESP32 to flash the ADAU1452 EEPROM. I suppose one could do the same with the RPi4 and ADAU1451 ;-) That's much more comfortable than XML copy&pasting - though of course extracting and verifying the parameters for the web interface would still be necessary.

Judging from the XML, the actual flashing could be done quite easily from the export (one file contains the section information, essentially size and offset into memory; the other contains the actual bytes to be written). I find the XML export/playback variant to be rather... adventure-ish :P

[begin edit] Since the input files are textual, I think you could even implement this in JS. Though the output is a binary file. Glue that to your current Parameter Reader, and let it emit a single, packed file, and you're set for a "upload this single DSP firmware via the beocreate web interface". [end edit]

btw, the active program is now "Double Four" :) Thanks for the pointers. Let's see how it sounds - then only one or two or a dozen measurements to get the PEQ right ^^

[tuomashamalainen]

We definitely need to give this some thought, the current solution is most certainly not very elegant or user-friendly.

If you've got some specific DSP-related features in mind that you would like to see in the GUI, feel free to drop suggestions. Can't promise significant near-term changes, but I'm always interested in hearing how our customers are using the product and how we could improve it to better suit their needs. Our goal is to cover the most common things, as the door will always be open to run more exotic, custom DSP flows 😊

Best,
Tuomas from Bang & Olufsen

[archi]

Hi Toumas, my DSP program defines two 1800 tap FIR filters (each in stereo configuration, so the program actually handles 4*1800 = 7200 taps; the ADAU1451 limit is ~8192 when doing nothing else). Maybe having a way to upload new values for FIRs using the GUI would be nice? The RePhase output format is just a file with 1800 lines of floats (and can be directly used by SigmaStudio), but I don't know how complex the programming-the-values-into-the-DSP end of business is.

Though for now I can just use SigmaStudio, connected via sigmatcp, so no need to rush this for me. I disabled the create GUI anyway for now, the whole hifiberryos code is not too happy running in a podman/docker container (startup takes an eternity, services fail,...) :'( Though, I've already created Arch Linux packages for hifiberry-dsp, which includes sigmatcp and dsptoolkit. Need to clean then up and then I'll push them to AUR. The create GUI will surely follow at some point as well ;-)

My usage scenario is similar to what you seem to be doing with the B&O speakers: The FIR acts an active 250 Hz crossover for my FAST/WAF, with linear-phase (here it differs from the B&O speakers). My testsetup also had a low-shelv filter, plus 15 band PEQ all integrated into 16k taps FIRs - besides acting as the active XO, these filters boosted the frequency response down to 40Hz at the expense of max SPL (but still enough). I need to do some measurements again, and then hope to do this with the Beocreate.

[hifiberry]

@archi It would be cool if you could share some documentation to your project on the HiFiBerry forum. I'm pretty sure other users would be interested in this.
https://support.hifiberry.com/hc/en-us/community/topics