FP8 PTQ With Physical Dependency 0515 #64333
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
Sorry to inform you that 3bb1c63's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Category
Execute Infrastructure
PR Types
New features
Description
添加适配fp8的量化反量化层，代码体现在python/paddle/nn/quant/fp8format.py中以及python/paddle/quantization/init.py中
添加调用fp8量化反量化的ptq逻辑，代码体现在python/paddle/quantization/fp8ptq.py python/paddle/quantization/fp8quantize.py
Paddle库设计逻辑：
python/paddle/nn/quant/fp8format.py
定义FP8ConvertibleQuantedLayer、FP8LinearQuanterDequanter、FP8LinearQuanter、FP8LinearDequanter用以根据在PaddleSlim中新定义的FP8UniformObserver统计的abs_max和fp8的取值范围计算出的scale执行量化和反量化运算。
python/paddle/quantization/fp8quantize.py
定义FP8QuantedLinear该类继承自FP8ConvertibleQuantedLayer，用以在执行_convert_to_quant_layers函数的时候将普通的Linear替换，定义FP8Quantization用以对接只属于FP8量化的逻辑
python/paddle/quantization/fp8ptq.py
定义FP8PTQ该类继承自FP8Quantization，用以暴露给PaddleNLP中定义的llm/fp8quant.py执行FP8 PTQ逻辑
python/paddle/quantization/init.py
将FP8PTQ添加到__init__中方便外部调用