Steps to reproduce LM+FM result in Table 3

[tongyao-zhu]

Thanks for your great work!

Using the checkpoints you released, I am able to get the result of SEAL (LM+FM, intersective) in Table 3. However, I am wondering how to obtain the result of SEAL (LM, |n| = 5) and SEAL (LM+FM) in the table. There are several flags in SEALSearcher in retrieval.py. Which ones should be set?

In addition, I notice that in retrieval.py, the found_keys are always rescored by rk.rescore_keys. May I know what does this function do? I don't seem to find it in the paper.

Thanks in advance!

[mbevila]

rescore_keys simplifies scoring at the expense of compute. Instead of getting the scores directly out of the beam search, we refeed the generated ngrams to the decoder and recompute logprobs. It shouldn't change the performances.