Skip to content

S3 Event Notifications - Configuration is ambiguously defined #59

Open
@kai-ten

Description

@kai-ten

The Problem

When creating Matano with byob, I was unable to deploy DPMainStack with the error "Configuration is ambiguously defined"

Ultimately this was because my existing dev bucket already had an SNS Event Notification on it for an existing workflow.

Side effects of this problem include:

  • Must remove the existing S3 Event Notification + deploy Matano (this causes an outage)
  • Repoint any existing SQS queue(s) to the new Matano SNS topic
    • If a user was using any Event trigger types other than Object Creation, they would then have to go and add their Event triggers again, causing stack drift / manual point of failure

Discussed solutions at the time of writing

  • Allow users to bring their own SNS Topics if there is already an S3 Event Notification
  • In the Transformer, skip events that don't match an acceptable Struct for ingest

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions