user_config_override.h can't override the config set in tasmota_configurations.h

[Frogmore42]

PROBLEM DESCRIPTION

A clear and concise description of what the problem is.
As an example: I want to disable (undef) USE_MPU_ACCEL
This is defined in tasmota\include\tasmota_configurations_ESP32.h, which is included by tasmota\include\tasmota_configurations.h, which is included by tasmota_options.h, but only after tasmota/my_user_config.h which is what includes tasmota\user_config_override.h.

This means if I want to turn off the default implementation of xsns_85_mpu6886.ino I HAVE to modify a low level header file.

It seems like it should be possible to fix this by reversing the order of the includes in tasmoata_globals.h so my_user_config.h comes last and therefore wins.

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

• [ x] Read the Contributing Guide and Policy and the Code of Conduct
• [ x] Searched the problem in issues
• Searched the problem in discussions
• [x ] Searched the problem in the docs
• Searched the problem in the chat
• [x ] Device used (e.g., Sonoff Basic): _____m5 Atom Matrix (esp32)
• [x ] Tasmota binary firmware version number used: _____14.3.0.7
  • Pre-compiled
  • [ x] Self-compiled
• [ x] Flashing tools used: _____
• Provide the output of command: Backlog Template; Module; GPIO 255:

  Configuration output here:

• If using rules, provide the output of this command: Backlog Rule1; Rule2; Rule3:

  Rules output here:

• Provide the output of this command: Status 0:

  STATUS 0 output here:

• Set weblog to 4 and then, when you experience your issue, provide the output of the Console log:

  Console output here:

TO REPRODUCE

Steps to reproduce the behavior:

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen.
I can choose to create a binary that does not include the MPU code just by editing user_config_override.h

It looks like it might not be as simple as just reversing the order of the includes in tasmota_options.h (since when I did that I got a linker error for something unrelated in berry).

SCREENSHOTS

If applicable, add screenshots to help explain your problem.

ADDITIONAL CONTEXT

Add any other context about the problem here.

(Please, remember to close the issue when the problem has been addressed)

[Jason2866]

You need to define your own firmware setup. See https://tasmota.github.io/docs/Compile-your-build/#defining-multiple-custom-firmwares

[Frogmore42]

@Jason2866 thanks for the tip on multiple custom firmware, but it too has the same issue.

Everything in user_config_override.h gets processed before everything in tasmota_configurations*.h, which means it is impossible to actually override what they (tasmota_configurations*.h) do. You can add things using user_config_override.h, but you can't remove something that tasmota_configurations*.h add, since they are processed later.

So it is fine to say won't fix for this issue. I can think of several ways that I can get whatever behavior I want. Since this is open source, I can change whatever I want to get it to do what I want. I filed this issue, since I figured the whole point of user_config_override.h was to make it so I did NOT have to change the underlying code. I also recall Sonoff-Tasmota from the days before user_config_override.h when my_user_config.h was the way to do it (of course, it still has the same problem). Tasmota has gotten a lot more complex (because it does a LOT more than it used to) and the configuration is much more complicated. As long as it generally works, it probably doesn't matter, but it is a lot harder to understand.

[sfromis]

What you have in tasmota_configurations_ESP32.h does not have to be used at all, unless you build for one of the precomposed build environments with #define symbols for FIRMWARE_xxxxx. You can create a custom build environment without any of those, but then you have to add features you specifically want to have included.

[barbudor]

@Frogmore42
While you are right that in tasmota32 (vs tasmota) you can't remove features from tasmota32 by #undef, if you make your own custom environment, you can totally remove the features of tasmota32 by NOT defining FIRMWARE_TASMOTA32

Default tasmota32 uses:

[env:tasmota32]
extends                 = env:tasmota32_base
build_flags             = ${env:tasmota32_base.build_flags}
                          -DFIRMWARE_TASMOTA32
                          -DOTA_URL='"http://ota.tasmota.com/tasmota32/release/tasmota32.bin"'```

While my own tasmota32-barbudor uses:

[env:tasmota32-barbudor]
extends                 = env:tasmota32_base
build_flags             = ${env:tasmota32_base.build_flags} -DFW_BARBUDOR  -DFW_IMAGE=\"barbudor\"

As FIRMWARE_TASMOTA32 is not defined, I don't the default features. then I can fully controlled what's baked in

[Jason2866]

Thx @barbudor for the explanation. I thought my link is enough. It wasn't ;-) It is wanted to make it not possible to define or undefine some stuff for predefined firmware variants. Adding every define to the variants Tasmota and Tasmota32 is possible. Removing is not. The only way to remove mostly anything is via the way from barbudor described.

[Frogmore42]

@Jason2866 thanks for the final sentence of your last post. That is not at all clear from the documentation. It also is clear that removing stuff is hard. It also appears that there is no minimal version (that can compile in under a min or two on a decent machine).

I wrote the following when this was an issue:

I do appreciate the help, so please take these comments in the way they are intended (to help others with similar problems/questions).

I have a long history both with Tasmota and C/C++ development (having done the later professionally) so I do know how to get a compiler to do what I want (and not just what I told it to do ;-)).

Having been around Tasmota for a long time, I know that I should use user_config_override.h and not (the now poorly named) my_user_config.h to make changes that are specific to my environment. I have been doing this for years and it has generally been fine. Recently I decided to make more use of some M5 Stack Atom Matrix devices that I have. These have a built in MPU that I figured would work well as a vibration sensor (prime use case is my automatic standby generator, which is supposed to exercise once a week, did it do that or not?). Seems simple enough. I find the location in the code that is processing the sensor. I discover that it is taking many mins to compile the source code and almost as long to download it (since the Atom won't work reliably above 115200 baud). So, this development is going to be hard with a 15+min turnaround for each change.

I then discover berry (which I had seen before, but didn't have a reason to try it). It even has an example driver with the MPU6886, so just what I need. Since I don't want what I am doing in berry to conflict with what the rest of the code is doing I try and remove the MPU6886. Turns out that is not so easy using what should be the easy way to do it.

#ifdef USE_MPU_ACCEL
#undef USE_MPU_ACCEL
#endif

So I searched the code to understand WHY this is not working. A quick set of searches reveals the complicated set of includes (listed in the original issue) which makes it impossible to actually override things that are set by tasmota_configurations.h. The simple solution of changing the order of the two includes in tasmota_options.h did not work for me, which means there must be something strange (or I did something wrong/strange). The other option is to have a different override file that is included after tasmota_configurations.h to let it actually override things that tasmota_configurations.h sets without having to set all the other things that it does.

Having looked more, your comment helps, since you are probably correct that the better option for me might be to figure out exactly what I want to include and only include that, since the default FIRMARE_TASMOTA32 includes WAY more than I need and helps explain with it takes 15min to compile it all. This solution does have a much higher bar to understanding/implementing than just adding the simple undef to user_config_override.h

So I took your advice and did this:

[env:tasmota32-frogmore]
extends                     = env:tasmota32_base
build_flags                 = ${env:tasmota32_base.build_flags}
                              -DFIRMWARE_FROGMORE32
                              -DCODE_IMAGE_STR='"tasmota32-frogmore"'
                              -DOTA_URL='""'

and this:

#ifdef FIRMWARE_FROGMORE32
  #warning **** Build: FROGMORE32 ****
  #define USE_DS18x20                              // Add support for DS18x20 sensors with id sort, single scan and read retry (+1k3 code)
  #define USE_I2C                                  // I2C using library wire (+10k code, 0k2 mem, 124 iram)
    #define USE_INA226                             // [I2cDriver35] Enable INA226 (I2C address 0x40, 0x41 0x44 or 0x45) Low voltage and current sensor (+2k3 code)
#endif

The log indicates that it did use my special config (since I see the compiler warning, MANY times) but the log also indicates that it compiled a LOT more stuff than I was expecting. Some of that is because the way configs are set up, lots of stuff gets added by default. But, the log also indicates that it compiled:

Compiling .pio\build\tasmota32-frogmore\lib5fc\MPU_accel\MPU_accel.cpp.o
Archiving .pio\build\tasmota32-frogmore\lib5fc\libMPU_accel.a

The good news is that it appears the code for that MPU didn't actually make it to the binary. It didn't save me much time in compiling, so not clear how worthwhile this is. This pretty much highlights the fundamental difference between Tasmota and ESPHome. Tasmota binaries give you everything including several kitchen sinks, while ESPHome gives only what you asked for. Having said that, I did switch the one thing I was using ESPHome for back to Tasmota since I found ESPHome not so reliable and the support for counters in Tasmota got a lot better.

I will say that the way options are handled is complicated, luckily most people don't need to use them as the default binaries are generally usable as they are.

[Jason2866]

Sadly Platformio fails in correctly detecting which libs needs to be included and compiled. So it compiles everything :-(
You can only trim down build time by including only needed libs and / or disable not needed.
It is done this way for the preconfigured build variants. For example the Tasmota32 zigbee bridge pro build

[Frogmore42]

So I spent a little time optimizing build time. Here is where I ended up:

in user_config_override.h

// Smart Meter Interface https://tasmota.github.io/docs/Smart-Meter-Interface/
#ifndef FIRMWARE_SENSORS
  // Not compatible with tasmota-sensors
  #ifndef USE_SCRIPT
    #define USE_SCRIPT
  #endif
  #ifndef USE_SML_M
    #define USE_SML_M
  #endif
  #ifdef USE_RULES
    #undef USE_RULES
  #endif
#endif

#ifdef FIRMWARE_FROGMORE32
  // #warning **** Build: FROGMORE32 ****
  #ifdef USE_IR_REMOTE
    #undef USE_IR_REMOTE // xdrv_05_irremote.ino missing IRremoteESP8266.h
  #endif
  #ifdef USE_ENERGY_SENSOR
    #undef USE_ENERGY_SENSOR // xnrg_05_pzem_ac.ino  xnrg_06_pzem_ac.ino missing TasmotaModbus.h
  #endif
  #ifdef USE_SHUTTER
    #undef USE_SHUTTER // xdrv_27_esp32_shutter.ino uses energy sensor stuff
  #endif
  #define USE_DS18x20                              // Add support for DS18x20 sensors with id sort, single scan and read retry (+1k3 code)
  #define USE_I2C                                  // I2C using library wire (+10k code, 0k2 mem, 124 iram)
    #define USE_INA219                             // [I2cDriver14] Enable INA219 (I2C address 0x40, 0x41 0x44 or 0x45) Low voltage and current sensor (+1k code)
    // #define USE_INA226                             // [I2cDriver35] Enable INA226 (I2C address 0x40, 0x41 0x44 or 0x45) Low voltage and current sensor (+2k3 code)
#endif

and in platformio_tasmota_cenv.ini

[env:tasmota32-frogmore]
extends                     = env:tasmota32_base
build_flags                 = ${env:tasmota32_base.build_flags}
                              -DFIRMWARE_FROGMORE32
                              -DCODE_IMAGE_STR='"tasmota32-frogmore"'
                              -DOTA_URL='""'
lib_ignore                  = ${env:tasmota32_base.lib_ignore}
                              Micro-RTSP
                              TasmotaModbus
                              ESP Mail Client
                              IRremoteESP8266
                              EEPROM
                              EEPROM 24C128_256_521
                              MFRC522
                              JPEGDEC
                              universal display Library
                              ESP8266Audio
                              ESP8266SAM
                              mp3_shine_esp32
                              Adafruit_LED_Backpack
                              Adafruit-GFX-Library
                              SevenSegmentTM1637
                              Display renderer
                              Waveshare esp 2.9 inch e-paper display driver
                              Waveshare esp 4.2 inch e-paper display driver
                              FT5206_Library
                              LiquidCrystal_I2C
                              TM1638plus
                              universal display Library
                              RA8876
                              XPT2046_Touchscreen
                              A4988_Stepper
                              Adafruit Fingerprint Sensor Library
                              AMS Parser
                              C2Programmer
                              DS2480 Library
                              ESP KNX IP Library
                              LibTeleinfo
                              NewPing
                              OpenTherm Library
                              Process_control
                              MFRC522
                              stm32_flash
                              Adafruit CCS811 Library
                              Adafruit MAX31865 library
                              Adafruit MCP9808 Library
                              Adafruit PM25 AQI Sensor
                              Adafruit seesaw Library
                              Adafruit SGP30 Sensor
                              Adafruit SGP40 Sensor
                              Adafruit TSL2591
                              Adafruit VEML7700 Library
                              Sensirion I2C SGP41
                              BME8563
                              BME68x
                              FrogmoreScd30
                              FrogmoreScd40
                              HPMA115S0 Arduino Library
                              MPU6050 I2C
                              Joba Tsl2561 Library
                              LOLIN_HP303B
                              MLX90640
                              MPU_accel
                              Grove - Multichannel Gas Sensor
                              ScioSense ENS16x
                              ScioSense ENS210
                              Sensirion Core
                              Sensirion I2C SEN5X
                              SPL06-007
                              VL53L0X
                              VL53L1X
                              cc1101
                              KeeloqLib
                              LoRa
                              RadioLib
                              rc-switch
                              RF24
                              epdiy
                              UdpListener
                              DHT sensor library

That removes about half the libraries and makes compilation much faster. I believe there are more that could be removed. The log indicates that the Adafruit GFX library still got compiled so LVGL might be doing something to get it back. Berry seems to have a dependency on LVGL and it wasn't obvious how to remove it (in 30s or less) so I just left LVGL in there, since I did want berry.

So, it is possible to make it faster, but I go back to: the configuration is much too complicated to fit into anybody's head. I don't expect it to change, since Tasmota is WAY down the be everything to everybody path. The cost of that is complexity.

Thanks for the help and encouragement in learning how it fits together. I now understand it a bit better.