Update sft trainer to include better packing

[Ishan-Kumar2]

What does this PR do?

Fixes #2466

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines.
• [] Did you write any new necessary tests?

Who can review?

@qgallouedec @shirinyamani @AIR-hl

Hi everyone, I am new to TRL, have attempted a fix at the issue 2466 here.
The greedy knapsack tries to fit as many examples as possible without overshooting, the remaining ones are chunked as before by breaking it up into smaller example size (<= max_seq_len). Also I havent made this the default packing and kept it under a flag.
Please let me know if this is close to what was expected, I can add tests then, thanks!

[qgallouedec]

Thanks for your contribution, and sorry for the late reply. I've made some comments. Can you please also:

• include a test for pack_example_smater
• modify the packing argument in SFTConfig to allow for this packing method

[qgallouedec]

wdyt of

Suggested change

	def pack_examples_smarter(examples: dict[str, list[list]], seq_length: int) -> dict[str, list[list]]:
	def pack_examples_knapsack(examples: dict[str, list[list]], seq_length: int) -> dict[str, list[list]]:

or even

Suggested change

	def pack_examples_smarter(examples: dict[str, list[list]], seq_length: int) -> dict[str, list[list]]:
	def pack_examples_knapsack_greedy(examples: dict[str, list[list]], seq_length: int) -> dict[str, list[list]]:

[qgallouedec]

maybe this is better:

Suggested change

	packing: bool,
	formatting_func: Optional[Callable[[dict], str]],
	dataset_name: str,
	pack_smart: bool = False,
	packing: Union[bool, str],
	formatting_func: Optional[Callable[[dict], str]],
	dataset_name: str,

and use if packing == "knapsack":