Topic Name Convention with Azure Schema Registry

[sridharanss]

I'm having my topic name is "domain.network.inbound.topic", what should be the name for schema registry ?

If I register the schema with the name "DomainNetworkInboundTopicValue", then the nussknacker UI is giving the topic name as "domain-network-inbound-topic". Character '-' is being added in between, but my actual topic name should have '.' not '-'.

Please let me know how to solve this ?

[arkadius]

Hi, we stick with the convention that topic name is in the lower-hyphen format. It is defined starting from SchemaNameTopicMatchStrategy:47:

val topicName = CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_HYPHEN, cleanedSchemaName)

The reason why we have such a convention is because Azure's schema registry doesn't have a concept similar to subjects in the Confluent Schema Registry and we can't easily determine how schemas are connected with topics. I see that we need to improve also the documentation around this area (https://nussknacker.io/documentation/docs/integration/KafkaIntegration/#schema-registry-comparison) because it is written very laconically.

I think that we could introduce some improvements to make this heuristic better e.g. 1. we could scan event hubs with names with different delimiter or 2. add the configuration parameter that would allow changing the delimiter from hyphen to any other.

For now, as a workaround, I can only propose changing the topic name.

[sridharanss]

Thank you for the Response,

Change Kafka topic name is not possible for us, there are bunch of topics and all are already being consumed by many other apps. Adding configuration parameter to change the delimiter would work for us. Are you planning to make this change ? if possible can you please make this change. Thank you.

[gskrobisz]

Hi, I've started working on a solution.

[sridharanss]

Thank you so much @gskrobisz. Please notify here, once you have created the new version with this solution.

[gskrobisz]

Hi @sridharanss ,
fix for naming strategy is ready to be published in v1.14. Right now you can find it at https://hub.docker.com/r/touk/nussknacker/tags in release_1.14-latest.

Be sure you have configured Kafka properties and Azure Schema Registry properties described in setting-up-nussknacker-cloud. With that basic configuration you should be able to use schema DomainNetworkInboundTopicValue with event hub domain.network.inbound.topic

Tested with schemas

and eventhubs / topics:

Please let us know if this solution is suitable for your environment.

[arkadius]

@sridharanss FYI, we released public 1.14 version with this patch. Take a look at changelog for the full list of changes.