Skip to content
This repository was archived by the owner on Jan 15, 2024. It is now read-only.

Commit 27331cb

Browse files
authored
[DOC] add int8 command (#1174)
1 parent 50e5278 commit 27331cb

File tree

1 file changed

+12
-12
lines changed

1 file changed

+12
-12
lines changed

scripts/bert/index.rst

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -223,22 +223,22 @@ version of `mxnet-mkl <https://apache-mxnet.s3-us-west-2.amazonaws.com/dist/inde
223223
Sentence Classification
224224
+++++++++++++++++++++++
225225

226-
+-----------+-------------------+---------------+---------------+---------+---------+-----+---------+
227-
| Dataset | Model | FP32 Accuracy | INT8 Accuracy | FP32 F1 | INT8 F1 | Log | Command |
228-
+===========+===================+===============+===============+=========+=========+=====+=========+
229-
| MRPC | bert_12_768_12 | 87.01 | 87.01 | 90.97 | 90.88 | | |
230-
+-----------+-------------------+---------------+---------------+---------+---------+-----+---------+
231-
| SST-2 | bert_12_768_12 | 93.23 | 93.00 | | | | |
232-
+-----------+-------------------+---------------+---------------+---------+---------+-----+---------+
226+
+-----------+-------------------+---------------+---------------+---------+---------+------------------------------------------------------------------------------------------------------------------------+
227+
| Dataset | Model | FP32 Accuracy | INT8 Accuracy | FP32 F1 | INT8 F1 | Command |
228+
+===========+===================+===============+===============+=========+=========+========================================================================================================================+
229+
| MRPC | bert_12_768_12 | 87.01 | 87.01 | 90.97 | 90.88 |`command <https://github.com/dmlc/web-data/blob/master/gluonnlp/logs/bert/calibration_MRPC_base_mx1.6.0b20200125.sh>`__ |
230+
+-----------+-------------------+---------------+---------------+---------+---------+------------------------------------------------------------------------------------------------------------------------+
231+
| SST-2 | bert_12_768_12 | 93.23 | 93.00 | | |`command <https://github.com/dmlc/web-data/blob/master/gluonnlp/logs/bert/calibration_SST_base_mx1.6.0b20200125.sh>`__ |
232+
+-----------+-------------------+---------------+---------------+---------+---------+------------------------------------------------------------------------------------------------------------------------+
233233

234234
Question Answering
235235
++++++++++++++++++
236236

237-
+-----------+-------------------+---------+---------+---------+---------+-----+---------+
238-
| Dataset | Model | FP32 EM | INT8 EM | FP32 F1 | INT8 F1 | Log | Command |
239-
+===========+===================+=========+=========+=========+=========+=====+=========+
240-
| SQuAD 1.1 | bert_12_768_12 | 81.18 | 80.32 | 88.58 | 88.10 | | |
241-
+-----------+-------------------+---------+---------+---------+---------+-----+---------+
237+
+-----------+-------------------+---------+---------+---------+---------+----------------------------------------------------------------------------------------------------------------------------+
238+
| Dataset | Model | FP32 EM | INT8 EM | FP32 F1 | INT8 F1 | Command |
239+
+===========+===================+=========+=========+=========+=========+============================================================================================================================+
240+
| SQuAD 1.1 | bert_12_768_12 | 81.18 | 80.32 | 88.58 | 88.10 |`command <https://github.com/dmlc/web-data/blob/master/gluonnlp/logs/bert/calibration_squad1.1_base_mx1.6.0b20200125.sh>`__ |
241+
+-----------+-------------------+---------+---------+---------+---------+----------------------------------------------------------------------------------------------------------------------------+
242242

243243
For all model settings above, we use a subset of evaluation dataset for calibration.
244244

0 commit comments

Comments
 (0)