Scoring Dialog using Hugging Face DialoGPT

[hckh]

Hi,

I would like to be able to do 2 things that I think should be straight forward, but am having trouble figuring them out in the Hugging Face model.

1. I would like to provide a prompt, and in addition to returning response produced by the model, I would also like to return the word level score/model probabilities.

2. I would like to enter a prompt and a response and use DialoGPT to get the word level score/model probabilities of that response (the 'force-decoded' score).

I have tried looking through the Hugging Face documentation, but it seems like different models have different APIs, and I can't quite figure out how to do either of these.

Thanks!

[dreasysnail]

Thanks for your interest! We have release a model for dialog evaluation. Please find the link below
https://github.com/golsun/DialogRPT