Replies: 1 comment 1 reply
-
|
对，直接使用 dynamic ntk 可以外推到 200K，没有在 200k 语料上训练过，后续的 tech report 会介绍细节，目前可以先参考 https://zhuanlan.zhihu.com/p/678871288 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
考虑过适配YaRN等方法吗？ |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
描述问题
“[QA] LMDeploy 实现了 dynamic ntk，支持长文本外推。使用如下代码，可以把 InternLM2 的文本外推到 200K：”。针对这句话的疑问，请解答下
Beta Was this translation helpful? Give feedback.
All reactions