feat: Include Object Paths in S3 sync summary

[bbernays]

This PR adds in a column that will include the full keys of all objects synced in the sync

objects_synced contains a map[string][]string where the map key is the table name and the array of strings are the object keys

{
  "_cq_source_name": "aws",
  "_cq_sync_time": "2024-03-11 17:11:16.369293",
  "cli_version": "development",
  "destination_name": "s3",
  "destination_path": "localhost:7778",
  "destination_version": null,
  "errors": 9,
  "objects_synced": {
    "aws_s3_buckets": [
      "aws_s3_buckets/8c862045-ed27-43f4-a70d-2b5f955e0748.parquet"
    ]
  },
  "resources": 29,
  "source_name": "aws",
  "source_path": "cloudquery/aws",
  "source_version": "v25.2.0",
  "sync_id": "a61ca899-1210-497b-ba1d-f442c26792a3",
  "warnings": 1
}

This is an extension of #17112

[erezrokah]

Shouldn't this be under the option to enable the summary?

[erezrokah]

This seems too tightly coupled with the CLI that uses the same name for the table

[bbernays]

It is designed to be an extension of the CLI summary table, all this functionality is doing is adding a column to that table.

[yevgenypats]

Few questions:

1. do we already have sync summary message? is it new
2. will objectkeys will run out of memory and it is not bound by anything?

depends on 1 and 2 we might want to implement sync summary only on the CLI in that case and then only add file that sais this sync finished and have a path of {{sync_id}} in the config instead of sending summary to every destination (unless we already do that?)

[bbernays]

Few questions:

1. do we already have sync summary message? is it new

It will be added in #17112

2. will objectkeys will run out of memory and it is not bound by anything?

I have added bounding that will limit it to only log the first 10,000 keys

depends on 1 and 2 we might want to implement sync summary only on the CLI in that case and then only add file that sais this sync finished and have a path of {{sync_id}} in the config instead of sending summary to every destination (unless we already do that?)

Adding {{sync_id}} only partially fixes the problem... S3 paths are used for at least 2 distinct things:

1. A means to search for objects: you can search for a path prefix to find all objects with said prefix
2. A means to partition data in S3/Athena for time based data. Adding the UUID into that means that it would be much harder to do because the UUID must be included in the definition

With the above solution assuming less than 10,000 objects in a single sync we get best of both worlds