Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible Bug: merge is overwriting instead of appending #50

Open
HackXIt opened this issue Apr 4, 2023 · 1 comment
Open

Possible Bug: merge is overwriting instead of appending #50

HackXIt opened this issue Apr 4, 2023 · 1 comment

Comments

@HackXIt
Copy link

HackXIt commented Apr 4, 2023

Hello,

I'm using aviator (version 1.9.0) to create a complex, or rather lengthy, pipeline.
The pipeline consists of over a dozen resources, and it would be hard to maintain or add to the pipeline if created by hand.

My aviator file is supposed to create an all_resources.yml, but it's not working as intended.
The file contains the full list of resources with their respective repo-names and repo-urls, and additionally it also contains a job, where all those resources are fetched with get as input to a compilation task.
I managed to get the part of the resources: section working and made a really identical part for the plan: section, but SOMEHOW, the - get lines overwrite each other, which ends up with only the last resource in the final file.

I'm really at a dead end here, because I can't see anything I'm doing wrong.
Can you help me out here?

This is my aviator.yml file:

spruce:
  # First we generate all resources to use in the pipeline
  - base: factory/meta-resource-stub.yml
    prune:
      - meta
    for_each:
      in: meta-resources/
      except:
        - example-resource.yml
      regexp: ".*.(yml)"
    to_dir: temp/resources/
  # Merge all resources into one file
  - base: factory/all-resources-stub.yml
    merge:
      - with_all_in: temp/resources/
        regexp: ".*.(yml)"
    to: temp/all-resources.yml

And these are the stubs:
meta-resource.stub.yml

---
jobs:
  - get: (( grab meta.git.repo-name ))
    trigger: (( grab meta.git.trigger ))
    params:
      depth: 1

resources:
  - name: (( grab meta.git.repo-name ))
    type: git
    source:
      uri: (( grab meta.git.http ))
      username: bot-tt
      password: ((sesource.bot-tt-pw))
      branch: (( grab meta.git.branch ))

all-resources-stub.yml

---
jobs:
  -
resources:
  -

When run, it correctly produces the all-resources.yml but inside, it failed to add each resource in the jobs section, instead it overwrites itself, so only the last resource was added.
The resources section correctly produces ALL generated resources.

I don't know what's wrong, since the two sections are near-identical
And also, I know that the resulting file currently is not correct in terms of concourse syntax, but I reduced it to a minimal example to figure out what's wrong.

@HackXIt
Copy link
Author

HackXIt commented Apr 4, 2023

I think I figured it out.

According to https://github.com/geofffranks/spruce/blob/main/doc/array-merging.md the identifier 'name' is special and in the default behavior of merge, it will merge existing items based on the value of the key.

I would need the ability to use the (( merge in <key> )) function inside aviator, to tell it to merge on the key get, so that I can add multiple inputs.

@HackXIt HackXIt changed the title Possible Bug: merge is overwriting instead of adding to the map Possible Bug: merge is overwriting instead of appending Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant