Adjusting the merge policy setting to make merges less aggressive

[shatejas]

Description

The default setting for opensearch changed, this caused faiss graph creation to use more CPU possibly affecting searches for cases where merge and search is happening on the same cluster. This change makes sure k-NN indices have less aggressive defaults

Related Issues

None

Testing details

Index created:

{
  "settings": {
    "index": {
			"knn": "true",
      "number_of_shards": 1,
      "number_of_replicas": 0,
			"knn.derived_source.enabled": false
    }
  },
  "mappings": {
    "properties": {
      "location": {
        "type": "knn_vector",
        "dimension": 768,
        "method": {
          "name": "hnsw",
          "space_type": "cosinesimil",
          "engine": "{{ _.engine }}",
          "parameters": {
            "ef_construction": 200,
            "m": 16
          }
        }
      },
			"tenant": {
				"type" : "text"
			}
    }
  }
}

{
	"target_index": {
		"settings": {
			"index": {
				"replication": {
					"type": "DOCUMENT"
				},
				"number_of_shards": "1",
				"provided_name": "target_index",
				"merge": {
					"policy": {
						"floor_segment": "2mb",
						"max_merge_at_once": "10"
					}
				},
				"knn": "true",
				"creation_date": "1771961852031",
				"number_of_replicas": "0",
				"uuid": "U4m6NE8KS0SSg-GqCLUdvQ",
				"version": {
					"created": "137277827"
				},
				"knn.derived_source": {
					"enabled": "false"
				}
			}
		}
	}
}

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[navneet1v]

we should move this to enhancement section

[navneet1v]

also lets have a IT to validate that index settings are getting set with correct values for index.knn true and false

[shatejas]

added

[Vikasht34]

LGTM

[kotwanikunal]

nit: We can use the settings key here as well

[navneet1v]

@shatejas Lets fix the CI issues.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.44%. Comparing base (a2372a3) to head (ca24f9d).

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #3128   +/-   ##
=========================================
  Coverage     82.43%   82.44%           
  Complexity     3874     3874           
=========================================
  Files           418      418           
  Lines         14474    14479    +5     
  Branches       1851     1851           
=========================================
+ Hits          11932    11937    +5     
  Misses         1782     1782           
  Partials        760      760           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.