- Multimodal intent benchmark results on multi-class and binary classification
Methods ACC_20 F1_20 P_20 R_20 ACC_2 F1_2 P_2 R_2 Human 85.51 85.07 86.37 85.74 94.72 94.67 94.74 94.82 Text-classifier 70.88 67.40 68.07 67.44 88.09 87.96 87.95 88.09 MAG-BERT 72.65 68.64 69.08 69.28 89.24 89.10 89.10 89.13 MulT 72.52 69.25 70.25 69.24 89.19 89.07 89.02 89.18 MISA 72.29 69.32 70.85 69.24 89.21 89.06 89.12 89.06 - Results (macro F1-score) of each fine-grained classes in "Express emotions and attitudes"
Methods Complain Praise Apologise Thank Criticize Care Agree Taunt Flaunt Oppose Joke Human 80.08 93.44 96.15 96.90 72.21 96.09 87.21 65.55 78.10 69.04 72.22 Text-classifier 64.36 85.69 97.93 97.22 47.06 87.42 94.26 15.53 46.12 32.32 27.42 MAG-BERT 67.65 86.03 97.76 96.52 49.02 85.59 91.60 15.78 47.09 33.97 37.54 MulT 65.48 84.72 97.93 96.83 49.72 88.12 92.23 26.12 48.91 34.68 33.95 MISA 63.91 86.63 97.78 98.03 53.44 87.14 92.05 22.15 46.44 36.15 38.74 - Results (macro F1-score) of each fine-grained classes in "Achieve goals"
Methods Inform Advise Arrange Introduce Comfort Leave Prevent Greet Ask for help Human 79.69 87.14 81.40 84.09 95.95 97.06 86.43 94.15 88.54 Text-classifier 67.74 67.68 64.67 68.64 77.05 73.37 82.47 84.90 66.20 MAG-BERT 71.00 69.30 63.82 67.42 76.43 75.77 85.07 91.06 64.44 MulT 70.85 69.43 65.44 71.19 76.44 75.58 81.68 86.65 69.12 MISA 70.18 69.56 67.32 67.22 78.78 77.23 83.30 82.71 67.57