[FileCopy based bootstrap] GetMetadata Api

[DevenAhluwalia]

Ticket

BUG=AMBRY-13007

Summary

Adds GetMetadata Api required for bootstrap replication improvements using file copy based protocol.

Testing Done

Test logs :-

2025/01/24 07:51:22.080 INFO [AmbryServer] [main] [ambry-server] [] Dw: Request: FileCopyGetMetaDataRequest[PartitionId=146, HostName=hostName]
2025/01/24 07:51:22.569 INFO [AmbryServer] [main] [ambry-server] [] Dw: Response: FileCopyGetMetaDataResponse[NumberOfLogfiles=2, logInfoList[LogInfo[FileName=0_62, FileSizeInBytes=4001180, IndexFiles=[FileInfo[FileName=0_62_18_index, FileSizeInBytes=364]], BloomFilters=[FileInfo[FileName=0_62_18_bloom, FileSizeInBytes=50024]]], LogInfo[FileName=24_0, FileSizeInBytes=1349928533, IndexFiles=[FileInfo[FileName=24_0_18_index, FileSizeInBytes=36572]]]]]

[piyujai]

What are we using this for?

[DevenAhluwalia]

This is used in a generic infra defined in RequestMetricsUpdater

This powers some common metrics like qps for a particular request type defined here.

[piyujai]

Do we need this to be unmodifiableList?

[DevenAhluwalia]

This ensures that the list cannot be modified from outside the class and helps in maintaining the immutability of the list and prevents accidental or intentional modifications.
Checked the demo master branch and this works well there too with FileCopyHandler caller code.

[justinlin-linkedin]

Let's use one LogInfo class so we don't need to do this conversion. Maybe we can put the LogInfo to ambry-api package.

[DevenAhluwalia]

Request handler for this api is in ambry-server module.
Protocol class is required to have network stream SerDe (alongwith readFrom, prepareBuffer etc) same as any other protocol request/response classes in ambry-protocol module.
BlobStore/FileStore need their Pojos (LogInfo, FileInfo) in ambry-store module as it doesn't have a dependency on Protocol module (and vice-versa)

Eventually, we need similar constructs in ambry-store and ambry-protocol modules.

[DevenAhluwalia]

This pattern is same in some other apis where Store and Protocol class conversion is happening.
fe, check this BatchDelete Api code

List<BlobDeleteStatus> blobDeleteStatuses = new ArrayList<>();
for (StoreKey storeKey : batchDeletePartitionRequestInfo.getBlobIds()){
    blobDeleteStatuses.add(new BlobDeleteStatus((BlobId)storeKey, ServerErrorCode.Unknown_Error));
}

BlobDeleteStatus is a protocol class while StoreKey is a Store class and we are converting from Store to Protocol class. Just that in my case, I have named both classes same (LogInfo) and created a private converter util.

[justinlin-linkedin]

you can put these classes under ambry-api modules. Just like FindToken. This way, you don't need two identical classes.

[DevenAhluwalia]

Ty for pointing this out @justinlin-linkedin
PTAL at the latest diff.

Changelog :-
Fixes the consolidation of pojos. Protocol/pojos no longer exist.
New interfaces are in common place api-module and Store, Protocol classes implement it.

[piyujai]

The @nonnull annotation will already be doing this check. We should remove this.

[DevenAhluwalia]

@nonnull works as a static warning. Shows up as red line in the calling code if null is passed but it doesn't give a compilation error. I tried the same. From tests, I was able to pass null. Hence, I added a requireNonNull check.

[piyujai]

nit: fileCopyGetMetaDataRequest != null

[DevenAhluwalia]

Yoda conditions

[mudit-saxena]

Can you remove redundant comments

[mudit-saxena]

Have a default access modifier

[DevenAhluwalia]

The default access modifier would be public in an interface.
the default keyword would mean that a default impl exists for this method (which btw, doesn't).

Can you pls elaborate what could be the default access modifier here?

[mudit-saxena]

yes , i meant public modifier

[mudit-saxena]

Why not this?

{
  return "LogInfo{" +
      "fileName='" + fileName + '\'' +
      ", fileSizeInBytes=" + fileSizeInBytes +
      ", indexFiles=" + indexFiles +
      ", bloomFilters=" + bloomFilters +
      '}';
}

[DevenAhluwalia]

I vaguely remember this.
iirc, it should have worked but It was not recursively printing indexFiles and bloomFilters.

Indexfiles and bloomfiles are of type List<FileInfo> and List wont get strigified as is. So, I needed to iterate over these values and then delegate to indexfile/bloomfile-toString()