fix(kafka): Handle message indices in proto data also for Glue Schema Registry

[karthikpswamy]

Issue #, if available:

Description of changes:

• Adding support for different schema registry types (Confluent and Glue)
• Improving the handling of message indices in protobuf data
• Fixing deserialization logic for different schema registry formats
• Adding appropriate tests for both Confluent and Glue schema registry scenarios

Schema Registry Type Detection and Deserialization Logic:

No Schema Registry Integration:

• When KafkaEvent contains no key/value metadata
• Uses entire byte array as raw data for deserialization
• AWS Glue Schema Registry:

Identified by 16-byte schema ID in key/value metadata

• Skips first byte of data
• Uses remaining bytes for Protobuf deserialization
• Confluent Schema Registry:

Default case when neither of above conditions are met

• Follows Confluent wire format specification
• Dynamically skips 1 or 2 bytes based on message index
• Uses remaining bytes for Protobuf deserialization

Checklist

• Meet tenets criteria
• Update tests
• Update docs
• PR title follows conventional commit semantics

Breaking change checklist

RFC issue #:

• Migration process documented
• Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[phipag]

Hey @karthikpswamy. Thanks again for contributing this logic. To unblock work during my timezone I created a copy of your PR here: #1907. Please review it. I did some minor changes but the key change is that I updated the GLUE Schema ID size to 36 because this is length of the uuids. Can you let me know if this is correct?