what is the reason set pad_token to unused token?

[blackcherry88]

Any special reason for it? This is regarding the following code in sft.py "tokenizer.pad_token = "<|fim_pad|>"

[Muennighoff]

Just to be sure that some code does not accidentally mask tokens that are actually used when it tries to mask all padding tokens

[sangmandu]

@Muennighoff Can you explain a bit more? Will using an existing pad token cause problems with masking?

[Muennighoff]

i think you should use a token for padding that you dont expect to appear in the regular prompt / completion

[sangmandu]

sorry but why? what is the reason..??