Fix vllm sampling params

[malte-aws]

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

PR information

Updates loading of generation config from file.
Previously the _load_generation_config function did not load all configuration from a generation_config.json file.

With this pull request _load_generation_config loads all configuration from a generation_config.json file. The _prepare_generation_config now uses the sampling parameters loaded from file and overwrites the ones provided in the request config.

This allows to configure guided decoding in a generation_config.json file. Here is an example generation_config.json file:

{
  "bos_token_id": 151643,
  "do_sample": true,
  "eos_token_id": 151645,
  "max_new_tokens": 2048,
  "pad_token_id": 151643,
  "temperature": 0.01,
  "top_k": 1,
  "top_p": 0.001,
  "transformers_version": "4.46.0.dev0",
  "guided_decoding": {
    "backend":"lm-format-enforcer",
    "json":{
      "type": "object",
      "properties": {
        "foo": { "type": "string"},
      },
      "required": ["foo"]
    }
  }
}