针对rv1126芯片量化mobilenet-0.25 ，得到的quant_model.onnx，使用rknntoolkit-1.7.1加载失败 #17

zjd1988 · 2023-07-06T01:16:57Z

按照example里关于rv平台的量化示例，对mobilenet模型进行量化，能正常的到量化后的onnx模型，但是用rknn-toolkit转换失败

看报错信息，应该是官方不支持gemm量化后的算子，我查看了官方rknntoolkit仓库里关于加载量化模型的示例，发现瑞芯微官方提供的shufflenet模型最后的gemm前后确实也没有加quant/dequant op
https://github.com/rockchip-linux/rknn-toolkit/tree/master/examples/common_function_demos/load_quantized_model/onnx

quant_model.zip
mobilenet.zip

Tracin · 2023-07-06T08:46:27Z

使用onnxruntime量化后的模型进行部署不确定性较大，推荐rv hybrid模式，使用dipoorlet生成的量化参数覆写rk工具生成的json

zjd1988 · 2023-07-06T09:11:04Z

@Tracin 你好，方便提供下详细的步骤吗，我看rv hybrid模式，step1 生成三个文件，torch_jit.json torch_jit.data torch_jit.quantization.cfg，Dipoorlet 生成的rv_quantized_param.yaml可以用来替换torch_jit.quantization.cfg里的内容，其他两个文件（torch_jit.json torch_jit.data ）还需要做修改吗？

附件是step1的中间结果和 Dipoorlet的结果
step1.zip

Tracin · 2023-07-06T09:19:33Z

只需要修改量化参数，注意不要替换全部内容，需要根据名字匹配修改min/max/scale/zp，可以参考https://github.com/ModelTC/Dipoorlet/blob/main/example/rv.md

zjd1988 · 2023-07-06T09:20:40Z

@Tracin 嗯嗯，灰常谢谢，我后边试试。

zjd1988 · 2023-07-06T09:53:31Z

@Tracin 尝试了一下，目前rknntoolkit-1.7.1 hybrid模式step1生成的节点名称，会在最前边加上节点类型，

Dipoorlet生成的json文件没有加节点类型

所以用https://github.com/ModelTC/Dipoorlet/blob/main/example/rv.md 这个脚本有点问题

zjd1988 · 2023-07-07T02:14:24Z

@Tracin 修改代码适配新的节点名字之后，执行hybrid_step1 和 hybrid_step2，并且执行精度分析看着精度不太好，
下面是Dipoorlet的精度模拟

/stem/stem.0/conv/Conv
Layer  cos: 0.99984
/stem/stem.0/relu/Relu
Layer  cos: 0.99983
/stem/stem.1/depthwise/depthwise.0/Conv
Layer  cos: 0.99867
/stem/stem.1/depthwise/depthwise.2/Relu
Layer  cos: 0.99924
/stem/stem.1/pointwise/pointwise.0/Conv
Layer  cos: 0.99775
/stem/stem.1/pointwise/pointwise.2/Relu
Layer  cos: 0.99801
/conv1/conv1.0/depthwise/depthwise.0/Conv
Layer  cos: 0.99633
/conv1/conv1.0/depthwise/depthwise.2/Relu
Layer  cos: 0.99688
/conv1/conv1.0/pointwise/pointwise.0/Conv
Layer  cos: 0.99426
/conv1/conv1.0/pointwise/pointwise.2/Relu
Layer  cos: 0.99464
/conv1/conv1.1/depthwise/depthwise.0/Conv
Layer  cos: 0.98980
/conv1/conv1.1/depthwise/depthwise.2/Relu
Layer  cos: 0.99008
/conv1/conv1.1/pointwise/pointwise.0/Conv
Layer  cos: 0.98599
/conv1/conv1.1/pointwise/pointwise.2/Relu
Layer  cos: 0.98736
/conv2/conv2.0/depthwise/depthwise.0/Conv
Layer  cos: 0.97897
/conv2/conv2.0/depthwise/depthwise.2/Relu
Layer  cos: 0.98137
/conv2/conv2.0/pointwise/pointwise.0/Conv
Layer  cos: 0.97854
/conv2/conv2.0/pointwise/pointwise.2/Relu
Layer  cos: 0.98086
/conv2/conv2.1/depthwise/depthwise.0/Conv
Layer  cos: 0.97220
/conv2/conv2.1/depthwise/depthwise.2/Relu
Layer  cos: 0.97512
/conv2/conv2.1/pointwise/pointwise.0/Conv
Layer  cos: 0.97208
/conv2/conv2.1/pointwise/pointwise.2/Relu
Layer  cos: 0.97411
/conv3/conv3.0/depthwise/depthwise.0/Conv
Layer  cos: 0.96590
/conv3/conv3.0/depthwise/depthwise.2/Relu
Layer  cos: 0.96956
/conv3/conv3.0/pointwise/pointwise.0/Conv
Layer  cos: 0.96503
/conv3/conv3.0/pointwise/pointwise.2/Relu
Layer  cos: 0.96880
/conv3/conv3.1/depthwise/depthwise.0/Conv
Layer  cos: 0.95286
/conv3/conv3.1/depthwise/depthwise.2/Relu
Layer  cos: 0.95817
/conv3/conv3.1/pointwise/pointwise.0/Conv
Layer  cos: 0.95736
/conv3/conv3.1/pointwise/pointwise.2/Relu
Layer  cos: 0.95975
/conv3/conv3.2/depthwise/depthwise.0/Conv
Layer  cos: 0.94178
/conv3/conv3.2/depthwise/depthwise.2/Relu
Layer  cos: 0.94906
/conv3/conv3.2/pointwise/pointwise.0/Conv
Layer  cos: 0.94918
/conv3/conv3.2/pointwise/pointwise.2/Relu
Layer  cos: 0.95384
/conv3/conv3.3/depthwise/depthwise.0/Conv
Layer  cos: 0.93601
/conv3/conv3.3/depthwise/depthwise.2/Relu
Layer  cos: 0.94546
/conv3/conv3.3/pointwise/pointwise.0/Conv
Layer  cos: 0.94466
/conv3/conv3.3/pointwise/pointwise.2/Relu
Layer  cos: 0.94837
/conv3/conv3.4/depthwise/depthwise.0/Conv
Layer  cos: 0.92799
/conv3/conv3.4/depthwise/depthwise.2/Relu
Layer  cos: 0.93851
/conv3/conv3.4/pointwise/pointwise.0/Conv
Layer  cos: 0.93460
/conv3/conv3.4/pointwise/pointwise.2/Relu
Layer  cos: 0.94114
/conv3/conv3.5/depthwise/depthwise.0/Conv
Layer  cos: 0.92225
/conv3/conv3.5/depthwise/depthwise.2/Relu
Layer  cos: 0.93377
/conv3/conv3.5/pointwise/pointwise.0/Conv
Layer  cos: 0.93424
/conv3/conv3.5/pointwise/pointwise.2/Relu
Layer  cos: 0.94100
/conv4/conv4.0/depthwise/depthwise.0/Conv
Layer  cos: 0.93205
/conv4/conv4.0/depthwise/depthwise.2/Relu
Layer  cos: 0.94296
/conv4/conv4.0/pointwise/pointwise.0/Conv
Layer  cos: 0.94205
/conv4/conv4.0/pointwise/pointwise.2/Relu
Layer  cos: 0.94703
/conv4/conv4.1/depthwise/depthwise.0/Conv
Layer  cos: 0.93303
/conv4/conv4.1/depthwise/depthwise.2/Relu
Layer  cos: 0.93837
/conv4/conv4.1/pointwise/pointwise.0/Conv
Layer  cos: 0.93148
/conv4/conv4.1/pointwise/pointwise.2/Relu
Layer  cos: 0.94145
/fc/Gemm
Layer  cos: 0.99676
The smallest cos value of 10 layers: 
/conv3/conv3.5/depthwise/depthwise.0/Conv-/conv3/conv3.5/depthwise/depthwise.0/Conv_output_0 cos : 0.92225
/conv3/conv3.4/depthwise/depthwise.0/Conv-/conv3/conv3.4/depthwise/depthwise.0/Conv_output_0 cos : 0.92799
/conv4/conv4.1/pointwise/pointwise.0/Conv-/conv4/conv4.1/pointwise/pointwise.0/Conv_output_0 cos : 0.93148
/conv4/conv4.0/depthwise/depthwise.0/Conv-/conv4/conv4.0/depthwise/depthwise.0/Conv_output_0 cos : 0.93205
/conv4/conv4.1/depthwise/depthwise.0/Conv-/conv4/conv4.1/depthwise/depthwise.0/Conv_output_0 cos : 0.93303
/conv3/conv3.5/depthwise/depthwise.2/Relu-/conv3/conv3.5/depthwise/depthwise.2/Relu_output_0 cos : 0.93377
/conv3/conv3.5/pointwise/pointwise.0/Conv-/conv3/conv3.5/pointwise/pointwise.0/Conv_output_0 cos : 0.93424
/conv3/conv3.4/pointwise/pointwise.0/Conv-/conv3/conv3.4/pointwise/pointwise.0/Conv_output_0 cos : 0.93460
/conv3/conv3.3/depthwise/depthwise.0/Conv-/conv3/conv3.3/depthwise/depthwise.0/Conv_output_0 cos : 0.93601
/conv4/conv4.1/depthwise/depthwise.2/Relu-/conv4/conv4.1/depthwise/depthwise.2/Relu_output_0 cos : 0.93837

下面是rknn的精度模拟

尝试一：
操作步骤：
（1）使用rv_quantized_param.json替换xxxx.quantization.cfg
（2）使用原始的onnx模型执行step1
（3）执行step2和精度分析

input_58_out0_nhwc_1_224_224_3.npy                              eculidean_norm=0.004127 cosine_norm=0.999991    eculidean=2.089898      cosine=0.999991
Conv__stem_stem.0_conv_Conv_57_out0_nhwc_1_224_224_8.npy        eculidean_norm=0.021166 cosine_norm=0.999776    eculidean=8.163392      cosine=0.999776
Relu__stem_stem.0_relu_Relu_56_out0_nhwc_1_224_224_8.npy        eculidean_norm=0.021166 cosine_norm=0.999776    eculidean=8.163392      cosine=0.999776
Conv__stem_stem.1_depthwise_depthwise.0_Conv_55_out0_nhwc_1_224_224_8.npy       eculidean_norm=0.053986 cosine_norm=0.998543    eculidean=16.932333     cosine=0.998543
Relu__stem_stem.1_depthwise_depthwise.2_Relu_54_out0_nhwc_1_224_224_8.npy       eculidean_norm=0.053986 cosine_norm=0.998543    eculidean=16.932333     cosine=0.998543
Conv__stem_stem.1_pointwise_pointwise.0_Conv_53_out0_nhwc_1_224_224_16.npy      eculidean_norm=0.088534 cosine_norm=0.996081    eculidean=48.532619     cosine=0.996081
Relu__stem_stem.1_pointwise_pointwise.2_Relu_52_out0_nhwc_1_224_224_16.npy      eculidean_norm=0.088534 cosine_norm=0.996081    eculidean=48.532619     cosine=0.996081
Conv__conv1_conv1.0_depthwise_depthwise.0_Conv_51_out0_nhwc_1_112_112_16.npy    eculidean_norm=0.154437 cosine_norm=0.988075    eculidean=44.775547     cosine=0.988075
Relu__conv1_conv1.0_depthwise_depthwise.2_Relu_50_out0_nhwc_1_112_112_16.npy    eculidean_norm=0.154437 cosine_norm=0.988075    eculidean=44.775547     cosine=0.988075
Conv__conv1_conv1.0_pointwise_pointwise.0_Conv_49_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.189053 cosine_norm=0.982130    eculidean=65.665627     cosine=0.982130
Relu__conv1_conv1.0_pointwise_pointwise.2_Relu_48_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.189053 cosine_norm=0.982130    eculidean=65.665627     cosine=0.982130
Conv__conv1_conv1.1_depthwise_depthwise.0_Conv_47_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.282762 cosine_norm=0.960023    eculidean=106.238625    cosine=0.960023
Relu__conv1_conv1.1_depthwise_depthwise.2_Relu_46_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.282762 cosine_norm=0.960023    eculidean=106.238625    cosine=0.960023
Conv__conv1_conv1.1_pointwise_pointwise.0_Conv_45_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.307300 cosine_norm=0.952783    eculidean=126.205544    cosine=0.952783
Relu__conv1_conv1.1_pointwise_pointwise.2_Relu_44_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.307300 cosine_norm=0.952783    eculidean=126.205544    cosine=0.952783
Conv__conv2_conv2.0_depthwise_depthwise.0_Conv_43_out0_nhwc_1_56_56_32.npy      eculidean_norm=0.374900 cosine_norm=0.929725    eculidean=78.079353     cosine=0.929725
Relu__conv2_conv2.0_depthwise_depthwise.2_Relu_42_out0_nhwc_1_56_56_32.npy      eculidean_norm=0.374900 cosine_norm=0.929725    eculidean=78.079353     cosine=0.929725
Conv__conv2_conv2.0_pointwise_pointwise.0_Conv_41_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.379721 cosine_norm=0.927906    eculidean=108.951294    cosine=0.927906
Relu__conv2_conv2.0_pointwise_pointwise.2_Relu_40_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.379721 cosine_norm=0.927906    eculidean=108.951294    cosine=0.927906
Conv__conv2_conv2.1_depthwise_depthwise.0_Conv_39_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.463514 cosine_norm=0.892577    eculidean=117.783615    cosine=0.892577
Relu__conv2_conv2.1_depthwise_depthwise.2_Relu_38_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.463514 cosine_norm=0.892577    eculidean=117.783615    cosine=0.892577
Conv__conv2_conv2.1_pointwise_pointwise.0_Conv_37_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.457422 cosine_norm=0.895383    eculidean=117.017387    cosine=0.895383
Relu__conv2_conv2.1_pointwise_pointwise.2_Relu_36_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.457422 cosine_norm=0.895383    eculidean=117.017387    cosine=0.895383
Conv__conv3_conv3.0_depthwise_depthwise.0_Conv_35_out0_nhwc_1_28_28_64.npy      eculidean_norm=0.503716 cosine_norm=0.873135    eculidean=63.907375     cosine=0.873135
Relu__conv3_conv3.0_depthwise_depthwise.2_Relu_34_out0_nhwc_1_28_28_64.npy      eculidean_norm=0.503716 cosine_norm=0.873135    eculidean=63.907375     cosine=0.873135
Conv__conv3_conv3.0_pointwise_pointwise.0_Conv_33_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.545430 cosine_norm=0.851253    eculidean=95.238365     cosine=0.851253
Relu__conv3_conv3.0_pointwise_pointwise.2_Relu_32_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.545430 cosine_norm=0.851253    eculidean=95.238365     cosine=0.851253
Conv__conv3_conv3.1_depthwise_depthwise.0_Conv_31_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.626550 cosine_norm=0.803717    eculidean=109.186737    cosine=0.803717
Relu__conv3_conv3.1_depthwise_depthwise.2_Relu_30_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.626550 cosine_norm=0.803717    eculidean=109.186737    cosine=0.803717
Conv__conv3_conv3.1_pointwise_pointwise.0_Conv_29_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.608508 cosine_norm=0.814859    eculidean=101.878395    cosine=0.814859
Relu__conv3_conv3.1_pointwise_pointwise.2_Relu_28_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.608508 cosine_norm=0.814859    eculidean=101.878395    cosine=0.814859
Conv__conv3_conv3.2_depthwise_depthwise.0_Conv_27_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.670188 cosine_norm=0.775424    eculidean=115.712540    cosine=0.775424
Relu__conv3_conv3.2_depthwise_depthwise.2_Relu_26_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.670188 cosine_norm=0.775424    eculidean=115.712540    cosine=0.775424
Conv__conv3_conv3.2_pointwise_pointwise.0_Conv_25_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.628073 cosine_norm=0.802762    eculidean=103.884956    cosine=0.802762
Relu__conv3_conv3.2_pointwise_pointwise.2_Relu_24_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.628073 cosine_norm=0.802762    eculidean=103.884956    cosine=0.802762
Conv__conv3_conv3.3_depthwise_depthwise.0_Conv_23_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.657071 cosine_norm=0.784129    eculidean=116.070755    cosine=0.784129
Relu__conv3_conv3.3_depthwise_depthwise.2_Relu_22_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.657071 cosine_norm=0.784129    eculidean=116.070755    cosine=0.784129
Conv__conv3_conv3.3_pointwise_pointwise.0_Conv_21_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.638838 cosine_norm=0.795943    eculidean=104.514961    cosine=0.795943
Relu__conv3_conv3.3_pointwise_pointwise.2_Relu_20_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.638838 cosine_norm=0.795943    eculidean=104.514961    cosine=0.795943
Conv__conv3_conv3.4_depthwise_depthwise.0_Conv_19_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.685335 cosine_norm=0.765158    eculidean=114.731880    cosine=0.765158
Relu__conv3_conv3.4_depthwise_depthwise.2_Relu_18_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.685335 cosine_norm=0.765158    eculidean=114.731880    cosine=0.765158
Conv__conv3_conv3.4_pointwise_pointwise.0_Conv_17_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.676670 cosine_norm=0.771059    eculidean=113.743423    cosine=0.771059
Relu__conv3_conv3.4_pointwise_pointwise.2_Relu_16_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.676670 cosine_norm=0.771059    eculidean=113.743423    cosine=0.771059
Conv__conv3_conv3.5_depthwise_depthwise.0_Conv_15_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.679760 cosine_norm=0.768963    eculidean=124.497566    cosine=0.768963
Relu__conv3_conv3.5_depthwise_depthwise.2_Relu_14_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.679760 cosine_norm=0.768963    eculidean=124.497566    cosine=0.768963
Conv__conv3_conv3.5_pointwise_pointwise.0_Conv_13_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.672946 cosine_norm=0.773572    eculidean=115.792709    cosine=0.773572
Relu__conv3_conv3.5_pointwise_pointwise.2_Relu_12_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.672946 cosine_norm=0.773572    eculidean=115.792709    cosine=0.773572
Conv__conv4_conv4.0_depthwise_depthwise.0_Conv_11_out0_nhwc_1_14_14_128.npy     eculidean_norm=0.660555 cosine_norm=0.781833    eculidean=60.376953     cosine=0.781833
Relu__conv4_conv4.0_depthwise_depthwise.2_Relu_10_out0_nhwc_1_14_14_128.npy     eculidean_norm=0.660555 cosine_norm=0.781833    eculidean=60.376953     cosine=0.781833
Conv__conv4_conv4.0_pointwise_pointwise.0_Conv_9_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.642327 cosine_norm=0.793708    eculidean=84.062347     cosine=0.793708
Relu__conv4_conv4.0_pointwise_pointwise.2_Relu_8_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.642327 cosine_norm=0.793708    eculidean=84.062347     cosine=0.793708
Conv__conv4_conv4.1_depthwise_depthwise.0_Conv_7_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.663912 cosine_norm=0.779611    eculidean=93.698517     cosine=0.779611
Relu__conv4_conv4.1_depthwise_depthwise.2_Relu_6_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.663912 cosine_norm=0.779611    eculidean=93.698517     cosine=0.779611
Conv__conv4_conv4.1_pointwise_pointwise.0_Conv_5_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.739684 cosine_norm=0.726434    eculidean=115.537247    cosine=0.726434
Relu__conv4_conv4.1_pointwise_pointwise.2_Relu_4_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.739684 cosine_norm=0.726434    eculidean=115.537247    cosine=0.726434
GlobalAveragePool__avg_GlobalAveragePool_3_out0_nhwc_1_1_1_256.npy      eculidean_norm=0.314344 cosine_norm=0.950594    eculidean=2.229387      cosine=0.950594
Reshape__Reshape_2_RKNN_mark_perm_59_out0_nhwc_1_256_1_1.npy    eculidean_norm=0.314344 cosine_norm=0.950594    eculidean=2.229387      cosine=0.950594
Reshape__Reshape_2_out0_nhwc_1_256.npy                          eculidean_norm=0.314344 cosine_norm=0.950594    eculidean=2.229387      cosine=0.950594
Gemm__fc_Gemm_1_out0_nhwc_1_10.npy                              eculidean_norm=0.345889 cosine_norm=0.940180    eculidean=3.018753      cosine=0.940180
attach_Gemm__fc_Gemm_out0_0_out0_nhwc_1_10.npy                  eculidean_norm=0.345889 cosine_norm=0.940180    eculidean=3.018753      cosine=0.940180

尝试二：
操作步骤：
（1）使用rv_quantized_param.json替换xxxx.quantization.cfg
（2）使用原始的adaround.onnx模型执行step1
（3）执行step2和精度分析

input_58_out0_nhwc_1_224_224_3.npy                              eculidean_norm=0.004127 cosine_norm=0.999991    eculidean=2.089898      cosine=0.999991
Conv__stem_stem.0_conv_Conv_57_out0_nhwc_1_224_224_8.npy        eculidean_norm=0.018000 cosine_norm=0.999838    eculidean=6.798223      cosine=0.999838
Relu__stem_stem.0_relu_Relu_56_out0_nhwc_1_224_224_8.npy        eculidean_norm=0.018000 cosine_norm=0.999838    eculidean=6.798223      cosine=0.999838
Conv__stem_stem.1_depthwise_depthwise.0_Conv_55_out0_nhwc_1_224_224_8.npy       eculidean_norm=0.051185 cosine_norm=0.998690    eculidean=16.421238     cosine=0.998690
Relu__stem_stem.1_depthwise_depthwise.2_Relu_54_out0_nhwc_1_224_224_8.npy       eculidean_norm=0.051185 cosine_norm=0.998690    eculidean=16.421238     cosine=0.998690
Conv__stem_stem.1_pointwise_pointwise.0_Conv_53_out0_nhwc_1_224_224_16.npy      eculidean_norm=0.084619 cosine_norm=0.996420    eculidean=47.497707     cosine=0.996420
Relu__stem_stem.1_pointwise_pointwise.2_Relu_52_out0_nhwc_1_224_224_16.npy      eculidean_norm=0.084619 cosine_norm=0.996420    eculidean=47.497707     cosine=0.996420
Conv__conv1_conv1.0_depthwise_depthwise.0_Conv_51_out0_nhwc_1_112_112_16.npy    eculidean_norm=0.147569 cosine_norm=0.989112    eculidean=43.876987     cosine=0.989112
Relu__conv1_conv1.0_depthwise_depthwise.2_Relu_50_out0_nhwc_1_112_112_16.npy    eculidean_norm=0.147569 cosine_norm=0.989112    eculidean=43.876987     cosine=0.989112
Conv__conv1_conv1.0_pointwise_pointwise.0_Conv_49_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.181115 cosine_norm=0.983598    eculidean=64.457291     cosine=0.983599
Relu__conv1_conv1.0_pointwise_pointwise.2_Relu_48_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.181115 cosine_norm=0.983598    eculidean=64.457291     cosine=0.983599
Conv__conv1_conv1.1_depthwise_depthwise.0_Conv_47_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.276782 cosine_norm=0.961696    eculidean=106.292648    cosine=0.961696
Relu__conv1_conv1.1_depthwise_depthwise.2_Relu_46_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.276782 cosine_norm=0.961696    eculidean=106.292648    cosine=0.961696
Conv__conv1_conv1.1_pointwise_pointwise.0_Conv_45_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.297474 cosine_norm=0.955755    eculidean=126.161407    cosine=0.955755
Relu__conv1_conv1.1_pointwise_pointwise.2_Relu_44_out0_nhwc_1_112_112_32.npy    eculidean_norm=0.297474 cosine_norm=0.955755    eculidean=126.161407    cosine=0.955755
Conv__conv2_conv2.0_depthwise_depthwise.0_Conv_43_out0_nhwc_1_56_56_32.npy      eculidean_norm=0.356959 cosine_norm=0.936290    eculidean=76.733688     cosine=0.936290
Relu__conv2_conv2.0_depthwise_depthwise.2_Relu_42_out0_nhwc_1_56_56_32.npy      eculidean_norm=0.356959 cosine_norm=0.936290    eculidean=76.733688     cosine=0.936290
Conv__conv2_conv2.0_pointwise_pointwise.0_Conv_41_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.363486 cosine_norm=0.933939    eculidean=108.036804    cosine=0.933939
Relu__conv2_conv2.0_pointwise_pointwise.2_Relu_40_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.363486 cosine_norm=0.933939    eculidean=108.036804    cosine=0.933939
Conv__conv2_conv2.1_depthwise_depthwise.0_Conv_39_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.444320 cosine_norm=0.901290    eculidean=116.232605    cosine=0.901290
Relu__conv2_conv2.1_depthwise_depthwise.2_Relu_38_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.444320 cosine_norm=0.901290    eculidean=116.232605    cosine=0.901290
Conv__conv2_conv2.1_pointwise_pointwise.0_Conv_37_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.437830 cosine_norm=0.904152    eculidean=114.794678    cosine=0.904152
Relu__conv2_conv2.1_pointwise_pointwise.2_Relu_36_out0_nhwc_1_56_56_64.npy      eculidean_norm=0.437830 cosine_norm=0.904152    eculidean=114.794678    cosine=0.904152
Conv__conv3_conv3.0_depthwise_depthwise.0_Conv_35_out0_nhwc_1_28_28_64.npy      eculidean_norm=0.484129 cosine_norm=0.882810    eculidean=62.654198     cosine=0.882809
Relu__conv3_conv3.0_depthwise_depthwise.2_Relu_34_out0_nhwc_1_28_28_64.npy      eculidean_norm=0.484129 cosine_norm=0.882810    eculidean=62.654198     cosine=0.882809
Conv__conv3_conv3.0_pointwise_pointwise.0_Conv_33_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.521382 cosine_norm=0.864080    eculidean=92.550476     cosine=0.864080
Relu__conv3_conv3.0_pointwise_pointwise.2_Relu_32_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.521382 cosine_norm=0.864080    eculidean=92.550476     cosine=0.864080
Conv__conv3_conv3.1_depthwise_depthwise.0_Conv_31_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.600567 cosine_norm=0.819660    eculidean=105.605957    cosine=0.819660
Relu__conv3_conv3.1_depthwise_depthwise.2_Relu_30_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.600567 cosine_norm=0.819660    eculidean=105.605957    cosine=0.819660
Conv__conv3_conv3.1_pointwise_pointwise.0_Conv_29_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.585402 cosine_norm=0.828652    eculidean=98.466415     cosine=0.828653
Relu__conv3_conv3.1_pointwise_pointwise.2_Relu_28_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.585402 cosine_norm=0.828652    eculidean=98.466415     cosine=0.828653
Conv__conv3_conv3.2_depthwise_depthwise.0_Conv_27_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.648339 cosine_norm=0.789828    eculidean=112.009468    cosine=0.789828
Relu__conv3_conv3.2_depthwise_depthwise.2_Relu_26_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.648339 cosine_norm=0.789828    eculidean=112.009468    cosine=0.789828
Conv__conv3_conv3.2_pointwise_pointwise.0_Conv_25_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.609681 cosine_norm=0.814144    eculidean=101.186188    cosine=0.814145
Relu__conv3_conv3.2_pointwise_pointwise.2_Relu_24_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.609681 cosine_norm=0.814144    eculidean=101.186188    cosine=0.814145
Conv__conv3_conv3.3_depthwise_depthwise.0_Conv_23_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.647330 cosine_norm=0.790482    eculidean=113.995216    cosine=0.790482
Relu__conv3_conv3.3_depthwise_depthwise.2_Relu_22_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.647330 cosine_norm=0.790482    eculidean=113.995216    cosine=0.790482
Conv__conv3_conv3.3_pointwise_pointwise.0_Conv_21_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.635823 cosine_norm=0.797864    eculidean=102.715630    cosine=0.797865
Relu__conv3_conv3.3_pointwise_pointwise.2_Relu_20_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.635823 cosine_norm=0.797864    eculidean=102.715630    cosine=0.797865
Conv__conv3_conv3.4_depthwise_depthwise.0_Conv_19_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.677077 cosine_norm=0.770783    eculidean=111.587761    cosine=0.770783
Relu__conv3_conv3.4_depthwise_depthwise.2_Relu_18_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.677077 cosine_norm=0.770783    eculidean=111.587761    cosine=0.770783
Conv__conv3_conv3.4_pointwise_pointwise.0_Conv_17_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.677754 cosine_norm=0.770325    eculidean=111.414268    cosine=0.770325
Relu__conv3_conv3.4_pointwise_pointwise.2_Relu_16_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.677754 cosine_norm=0.770325    eculidean=111.414268    cosine=0.770325
Conv__conv3_conv3.5_depthwise_depthwise.0_Conv_15_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.685440 cosine_norm=0.765086    eculidean=122.818817    cosine=0.765086
Relu__conv3_conv3.5_depthwise_depthwise.2_Relu_14_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.685440 cosine_norm=0.765086    eculidean=122.818817    cosine=0.765086
Conv__conv3_conv3.5_pointwise_pointwise.0_Conv_13_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.682958 cosine_norm=0.766784    eculidean=114.223099    cosine=0.766784
Relu__conv3_conv3.5_pointwise_pointwise.2_Relu_12_out0_nhwc_1_28_28_128.npy     eculidean_norm=0.682958 cosine_norm=0.766784    eculidean=114.223099    cosine=0.766784
Conv__conv4_conv4.0_depthwise_depthwise.0_Conv_11_out0_nhwc_1_14_14_128.npy     eculidean_norm=0.670988 cosine_norm=0.774888    eculidean=59.889061     cosine=0.774888
Relu__conv4_conv4.0_depthwise_depthwise.2_Relu_10_out0_nhwc_1_14_14_128.npy     eculidean_norm=0.670988 cosine_norm=0.774888    eculidean=59.889061     cosine=0.774888
Conv__conv4_conv4.0_pointwise_pointwise.0_Conv_9_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.644046 cosine_norm=0.792602    eculidean=81.873611     cosine=0.792602
Relu__conv4_conv4.0_pointwise_pointwise.2_Relu_8_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.644046 cosine_norm=0.792602    eculidean=81.873611     cosine=0.792602
Conv__conv4_conv4.1_depthwise_depthwise.0_Conv_7_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.673401 cosine_norm=0.773266    eculidean=90.893112     cosine=0.773266
Relu__conv4_conv4.1_depthwise_depthwise.2_Relu_6_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.673401 cosine_norm=0.773266    eculidean=90.893112     cosine=0.773266
Conv__conv4_conv4.1_pointwise_pointwise.0_Conv_5_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.759542 cosine_norm=0.711548    eculidean=112.248756    cosine=0.711548
Relu__conv4_conv4.1_pointwise_pointwise.2_Relu_4_out0_nhwc_1_14_14_256.npy      eculidean_norm=0.759542 cosine_norm=0.711548    eculidean=112.248756    cosine=0.711548
GlobalAveragePool__avg_GlobalAveragePool_3_out0_nhwc_1_1_1_256.npy      eculidean_norm=0.309183 cosine_norm=0.952203    eculidean=2.011168      cosine=0.952203
Reshape__Reshape_2_RKNN_mark_perm_59_out0_nhwc_1_256_1_1.npy    eculidean_norm=0.309183 cosine_norm=0.952203    eculidean=2.011168      cosine=0.952203
Reshape__Reshape_2_out0_nhwc_1_256.npy                          eculidean_norm=0.309183 cosine_norm=0.952203    eculidean=2.011168      cosine=0.952203
Gemm__fc_Gemm_1_out0_nhwc_1_10.npy                              eculidean_norm=0.291019 cosine_norm=0.957654    eculidean=2.744588      cosine=0.957654
attach_Gemm__fc_Gemm_out0_0_out0_nhwc_1_10.npy                  eculidean_norm=0.291019 cosine_norm=0.957654    eculidean=2.744588      cosine=0.957654

gushiqiao · 2023-07-07T05:59:25Z

@zjd1988
感谢指出https://github.com/ModelTC/Dipoorlet/blob/main/example/rv.md 脚本存在的问题，我们会尽快修改。
从Dipoorlet的结果来看，有些层的cosine不够高，建议使用Dipoorlet中更先进的离线量化算法，例如Qdrop和Brecq，具体使用方式是在运行dipoorlet的时候加上 --brecq 或 --brecq --drop，并使用产出的brecq.onnx部署。

zjd1988 · 2023-07-07T06:53:48Z

@gushiqiao 好的，我后边尝试一下，谢谢。

Tracin · 2023-07-07T07:25:08Z

RV分析结果还是很奇怪的，非对称u8量化模式下RV与DP模拟差距有些大
可以尝试使用--we调整

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

针对rv1126芯片量化mobilenet-0.25 ，得到的quant_model.onnx，使用rknntoolkit-1.7.1加载失败 #17

针对rv1126芯片量化mobilenet-0.25 ，得到的quant_model.onnx，使用rknntoolkit-1.7.1加载失败 #17

zjd1988 commented Jul 6, 2023

Tracin commented Jul 6, 2023 •

edited

Loading

zjd1988 commented Jul 6, 2023 •

edited

Loading

Tracin commented Jul 6, 2023

zjd1988 commented Jul 6, 2023

zjd1988 commented Jul 6, 2023 •

edited

Loading

zjd1988 commented Jul 7, 2023 •

edited

Loading

gushiqiao commented Jul 7, 2023 •

edited

Loading

zjd1988 commented Jul 7, 2023

Tracin commented Jul 7, 2023

针对rv1126芯片量化mobilenet-0.25 ，得到的quant_model.onnx，使用rknntoolkit-1.7.1加载失败 #17

针对rv1126芯片量化mobilenet-0.25 ，得到的quant_model.onnx，使用rknntoolkit-1.7.1加载失败 #17

Comments

zjd1988 commented Jul 6, 2023

Tracin commented Jul 6, 2023 • edited Loading

zjd1988 commented Jul 6, 2023 • edited Loading

Tracin commented Jul 6, 2023

zjd1988 commented Jul 6, 2023

zjd1988 commented Jul 6, 2023 • edited Loading

zjd1988 commented Jul 7, 2023 • edited Loading

gushiqiao commented Jul 7, 2023 • edited Loading

zjd1988 commented Jul 7, 2023

Tracin commented Jul 7, 2023

Tracin commented Jul 6, 2023 •

edited

Loading

zjd1988 commented Jul 6, 2023 •

edited

Loading

zjd1988 commented Jul 6, 2023 •

edited

Loading

zjd1988 commented Jul 7, 2023 •

edited

Loading

gushiqiao commented Jul 7, 2023 •

edited

Loading