[Theme] Dev Services

[ThomasVitale]

Description

This theme is for collecting requirements and tasks to get the Arconia Dev Services ready for the 1.0 release. Separate issues will be used to track the actual implementation work. We'll use this theme to gather requirements and discuss ideas. For starters, I'm adding details here, but we'll progressively move them to dedicated issues.

General

• Consider if each Dev Service module should be marked as a development tool by adding Spring-Boot-Jar-Type: development-tool to the MANIFEST.MF file. See spring-projects/spring-boot@27715de for more context.
• Service connection beans in Spring Boot can only be configured via annotations, such as @RestartScope. The new BeanRegistrar API doesn't support registering them programmatically. The permutations of different possible annotations on a service connection bean leads to cumbersome auto-configuration. Currently, each Dev Service comes with two different configurations: restart and no-restart. If we wanted to support TLS configuration, we would end up with many different configurations as multiple annotations are used by Spring Boot for that purpose (@Ssl, @JksKeyStore, @JksTrustStore, @PemKeyStore and @PemTrustStore). Explore options to overcome this challenge. Handled in [Dev Services] Consolidate bean registration #160 and df1574d
• Lots of configurations are duplicated across all Dev Services module. That should be consolidated for the sake of clarity and maintenance. Handled in [Dev Services] Consolidate bean registration #160
• Lots of test code is duplicated across all Dev Services module. That should be consolidated for the sake of clarity and maintenance. Handled in 1f53561

Testcontainers

• Each Arconia Dev Services module should define an Arconia subclass of the Container class used in that module to ensure there's a unique API for each service (e.g. ArconiaRabbitMQContainer). Currently, some of them use GenericContainer (like Redis and Phoenix), which limits customisations. This change is for enabling use cases such as allowing bean override (using the @ConditionalOnMissingBean annotation in the auto-configuration) and logging information like the URLs where certain UIs are made available. Handled in [Dev Services] Consolidate bean registration #160.
• Consider if Dev Services should support TLS connection between the application and the container. Spring Boot introduced TLS support for service connections (see Consider how to provide SSL from ConnectionDetails spring-projects/spring-boot#41137) specifically due to issues with the Elasticsearch Testcontainer (see Improve Elasticsearch ServiceConnection spring-projects/spring-boot#35926). Even though there is support on the Spring Boot side for the client connection and we might be able to support that, the container itself also needs to be customised internally with the TLS certificates. Only Elasticsearch comes built-in with this support, from what I understand. For the rest, it's manual, explicit configuration which differs for each container image. Dev Services follow the principles of Spring Boot auto-configuration: sensible defaults out-of-the-box, but with the possibility to override if needed. So it might be acceptable to let users interested in TLS between app and dev services to define their own service connection bean, as documented in https://docs.spring.io/spring-boot/reference/testing/testcontainers.html#testing.testcontainers.service-connections. Further investigation is needed for this point before proceeding.
• Configurable fixed exposed ports. Handled in [Dev Services] Configurable fixed exposed ports #161
• Configurable network aliases. Handled in df1574d
• Configurable shared networks [Dev Services] Configurable shared networks #146
• The Reusable Containers feature is a bit confusing. Refactor it so that it can be controlled via a boolean value. When true, the containers will be made reusable only when the application runs in dev mode. Handled in df1574d
• Configurable resource mappings (classpath/file system resources copied into a container, read-only) c281d37
• Configurable file system bindings (mounting file system paths into a container, read/write) [Dev Services] Support file system bindings for dev service containers #171 1f53561

[jimisola]

What logic is behind the default value for private Shared shared = Shared.NEVER?

I noticed that dev services have different values which probably is not obvious to users.
Since, Arconia is used for dev and test my thought is that it should be as.. "relaxed" as possible to allow for zero-config.

[ThomasVitale]

What logic is behind the default value for private Shared shared = Shared.NEVER?

@jimisola The shared functionality is currently based on the Testcontainers Reusable Containers feature. The three states are described here, as well as the fact that each Dev Service has a different default value: https://docs.arconia.io/arconia/latest/dev-services/#_sharing_dev_services

In the upcoming 0.22.0 release, that setup is simplified to a boolean condition, considered only when running the application in dev mode.

The reasoning behind why certain Dev Services are shareable by default is that they are usually used across applications, such as message brokers or observability services.

I noticed that dev services have different values which probably is not obvious to users.
Since, Arconia is used for dev and test my thought is that it should be as.. "relaxed" as possible to allow for zero-config.

That is exactly the reason behind the default values. For example, when using the Grafana Dev Service, and I have two applications running in dev mode, I would likely want to have it shared so I can trace requests between them. On the other hand, when using the PostgreSQL Dev Service, I would likely want to use it specifically for my application or else I would run into problems in terms of initialisation, schemas, Flyway/Liquibase setups, and so on.

I will extend the docs to include a bit more background on the decision, thanks for bringing this up!

[jimisola]

@ThomasVitale Ok. So, how does the new Shared setup with a boolean actually work? How about when running tests?
Haven't thought about this before, come to think of it is so that you never want to share when running tests?

Yes, the logic makes sense. Reason why it came to my attention was that I was not sure what to default to for keycloak.

[ThomasVitale]

That's indeed what is simplified in the upcoming release: containers can be optionally shared only in dev mode (the option for sharing them also in test mode has been removed).

When running tests, there won't be any sharing for two main reasons: to keep the tests self-contained and reliable, and to avoid leaving running containers after the test execution.

I have extended the documentation to explain better the feature and the reasoning behind certain choices.