[BUG][Opensearch] Cannot merge custom config opensearch.yml parameters with default ones: Ignoring non-table value

[jetnet]

Describe the bug
I'd like to merge the default Opensearch configuration parameters with my custom ones from the config values. The custom parameters are defined as a map, but getting an error.

To Reproduce
Steps to reproduce the behavior:

1. Create a values file _values-master.yaml:

config:
  opensearch.yml:
    indexing_pressure.memory.limit: "50%"

2. Create templates:

helm template --values _values-master.yaml  --version 2.26.0 -n opensearch-test opensearch-test-master opensearch/opensearch

3. See error

coalesce.go:289: warning: destination for opensearch.config.opensearch.yml is a table. Ignoring non-table value (cluster.name: opensearch-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0

# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# Implicitly done if ".singleNode" is set to "true".
# discovery.type: single-node

# Start OpenSearch Security Demo Configuration
# WARNING: revise all the lines below before you go into production
plugins:
  security:
    ssl:
      transport:
        pemcert_filepath: esnode.pem
        pemkey_filepath: esnode-key.pem
        pemtrustedcas_filepath: root-ca.pem
        enforce_hostname_verification: false
      http:
        enabled: true
        pemcert_filepath: esnode.pem
        pemkey_filepath: esnode-key.pem
        pemtrustedcas_filepath: root-ca.pem
    allow_unsafe_democertificates: true
    allow_default_init_securityindex: true
    authcz:
      admin_dn:
        - CN=kirk,OU=client,O=client,L=test,C=de
    audit.type: internal_opensearch
    enable_snapshot_restore_privilege: true
    check_snapshot_restore_write_privileges: true
    restapi:
      roles_enabled: ["all_access", "security_rest_api_access"]
    system_indices:
      enabled: true
      indices:
        [
          ".opendistro-alerting-config",
          ".opendistro-alerting-alert*",
          ".opendistro-anomaly-results*",
          ".opendistro-anomaly-detector*",
          ".opendistro-anomaly-checkpoints",
          ".opendistro-anomaly-detection-state",
          ".opendistro-reports-*",
          ".opendistro-notifications-*",
          ".opendistro-notebooks",
          ".opendistro-asynchronous-search-response*",
        ]
######## End OpenSearch Security Demo Configuration ########
)

Generated config map:

---
# Source: opensearch/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: opensearch-cluster-master-config
  labels:
    helm.sh/chart: opensearch-2.26.0
    app.kubernetes.io/name: opensearch
    app.kubernetes.io/instance: opensearch-test-master
    app.kubernetes.io/version: "2.17.1"
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: opensearch-cluster-master
data:
  opensearch.yml: |
    indexing_pressure.memory.limit: 50%

Expected behavior
The custom parameter from the _values-master.yaml gets merged with the default Chart's opensearch.yml configuration

Chart Name
opensearch/opensearch

Host/Environment (please complete the following information):

• Helm Version: v3.14.0
• (not applicable) Kubernetes Version: client v1.28.2, server: v1.30.7

[DandyDeveloper]

@jetnet This is by design because the yml is passed as a string.

config:
  opensearch.yml: |
    cluster.name: opensearch-cluster

    # Bind to all interfaces because we don't know what IP address Docker will assign to us.
    network.host: 0.0.0.0
  ...

This is by design.

There's probably 100 reasons on both sides why this is a good or bad idea, but I don't expect it to change unless there's significant callouts. Especially as it has a breaking change impact on existing users of the helm chart.

I think there's been a few discussions back when the Helm chart was in its infancy and it was decided to proceed with the string instead of the map (I don't remember fully though).

[jetnet]

@jetnet This is by design because the yml is passed as a string.

Then, I was confused by the description of the config parameter - "either replaced or merged":

In case of string format, the whole content of the config file will be replaced by new config file value

when in case of using map format content of configuration file will be a result of merge.

and I was trying to merge.

[DandyDeveloper]

@jetnet Yeah, I have a feeling this is an older doc string that would have been wrong since moving from a map to a string type on that value.

It'll need to be updated. I'll see if I can quickly sneak in a README update.