LDF downloads always all "lib" dependencies regardless of lib_deps

[BlueAndi]

Configuration

Operating system:
Win10 + VSCode + PIO

PlatformIO Version (platformio --version):
6.1.11

Description of problem

The platformio.ini contains several environments. Every environment requires a different set of libraries, which are configured by lib_deps. The problem is that ALL dependencies are downloaded for every environment, means .pio/libdeps/<ENV> contains always all of them.

Steps to Reproduce

platformio.ini

[env:testA]
board = esp32doit-devkit-v1
board_build.filesystem = littlefs
platform = espressif32 @ ~6.5.0
framework = arduino
lib_deps =
    A

[env:testB]
board = esp32doit-devkit-v1
board_build.filesystem = littlefs
platform = espressif32 @ ~6.5.0
framework = arduino
lib_deps =
    B

lib/A/library.json

{
    "name": "A",
    "dependencies": [{
        "owner": "makuna",
        "name": "NeoPixelBus",
        "version": "~2.7.3"
    }]
}

lib/B/library.json

{
    "name": "B",
    "dependencies": [{
        "owner": "bodmer",
        "name": "TFT_eSPI",
        "version": "~2.5.31"
    }]
}

Actual Results

.pio/libdeps/testA/ folder contains NeoPixelBus and TFT_eSPI libraries
.pio/libdeps/testB/ folder contains NeoPixelBus and TFT_eSPI libraries

Expected Results

.pio/libdeps/testA/ folder contains ONLY NeoPixelBus library
.pio/libdeps/testB/ folder contains ONLY TFT_eSPI library

Notes

It seems LDF loops over every library in ./lib instead of following environment wise.

[ivankravets]

Do you run only the specified environment? pio run -e testB

[BlueAndi]

The environments are exclusively used, depended on the use case. Not sure whether I understood your question right. :-)

[ivankravets]

Please provide a simple project (as ZIP attached to this issue) to reproduce this issue.

[BlueAndi]

pioLibTest.zip

[ivankravets]

I can't reproduce this issue:

$ pio run  -e testA

Processing testA (board: esp32doit-devkit-v1; platform: espressif32; framework: arduino)
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Library Manager: Installing makuna/NeoPixelBus @ ~2.7.3
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
Library Manager: Installing bodmer/TFT_eSPI @ ~2.5.31
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
Verbose mode can be enabled via `-v, --verbose` option
....

[gabryelreyes]

I believe you did reproduce the issue. lib/A/library.json contains ONLY NeoPixelBus as a dependency, however in the libdeps folder you have TFT_eSPI too. This is what the issue describes on the first comment.

[ivankravets]

Does NeoPixelBus depend on TFT_eSPI?

[gabryelreyes]

No, it does not.

[ivankravets]

Thanks, we reproduced this issue. This is a normal behaviour, see https://github.com/platformio/platformio-core/blob/develop/platformio/package/commands/install.py#L258

We resolve dependencies BEFORE LDF starts (because we need all manifests). Please note that libraries installed/copied to the libdeps folder are not always built (LDF decides). Also, external libraries from the registry are cached locally, there is not "redownloading" that could lead to additional compilation time.

[gabryelreyes]

Can you please test the following project?

pioLibTest2.zip

Library "A" depends on ArduinoJson @ ~6.16.1 but instead it gets compiled with ArduinoJson @ ~7.0.3, which is a dependency of "B".

Scanning dependencies...
Dependency Graph
|-- A
|-- ArduinoJson @ 7.0.3

[ivankravets]

This is a bug. Could I ask you to file a new issue https://github.com/platformio/platformio-core ?

[gabryelreyes]

Sure, I can file a new issue. However, is this issue not already the description of the problem? The reason of this behavior is still the LDF loading all libraries.

If this issue is not enough, please detail what the new issue should say and contain in order to be different from this one.

Thank you for your support

[ivankravets]

This issue relates to "all dependencies are downloaded from the "lib" folder" and this is normal behavior.

Library "A" depends on ArduinoJson @ ~6.16.1 but instead it gets compiled with ArduinoJson @ ~7.0.3, which is a dependency of "B".

This a bug. Please open a new issue and refer to this issue. Thank you so much for your attention and participation.

[BlueAndi]

@ivankravets If always all dependencies from the lib folder are downloaded, its an performance issue for projects with a large number of environments, using always only a part of the libs. To get around, the only chance is to move the meta dependencies info from the libs to the environment directly, or do you recommend something else?

[ivankravets]

I reopen this issue with updated subject. Indeed, we can improve PlatformIO Core behaviour installing local "lib" dependencies to the separate common folder.

Anyway, please file a new issue which actually a bug regarding LDF that picks wrong library version when requirements are declared and multiple versions of the same library are installed.