Correspondence between paper formulas and code

[a7217339]

Hello, sorry to bother you. I am a beginner and I have some shallow questions to ask you. May I ask why, in the compute_logprobe function, multiple tokens are indexed on responsive_log_probes corresponding to the probability distribution of a single token? How do these log_tokengsprob sums correspond to [1y | | π ref (· | x)] in the paper? I am looking forward to receiving your reply. Thank you

[jinnaiyuu]

Thank you very much for your question.

The KL divergence between the identity function 1_y and the reference policy \pi_ref results in the log probability of the text y. We will make this clearer in future revisions.

Please let us know if you have any other questions.