Add benchmark of Aquila2 34B AWQ in README.md. (#126)

Signed-off-by: ldwang <[email protected]>
Co-authored-by: ldwang <[email protected]>

README.md

@@ -258,6 +258,24 @@ generation_output = model.generate(
 | 1 | 1024 | 1024 | 2256.22 | 94.0237 | 4.69 GB (19.78%) |
 | 1 | 2048 | 2048 | 1831.71 | 94.2032 | 6.83 GB (28.83%) |
 
+### Aquila2 34B
+
+- Note: Fast generation, fast context processing
+- GPU: NVIDIA A100-SXM4-40GB
+- Command: `python examples/benchmark.py --model_path casperhansen/aquilachat2-34b-awq --quant_file pytorch_model.bin.index.json`
+- Version: GEMM
+
+| Batch Size | Prefill Length | Decode Length | Prefill tokens/s | Decode tokens/s | Memory (VRAM) |
+|-------------:|-----------------:|----------------:|-------------------:|------------------:|:------------------|
+| 1 | 32 | 32 | 36.7505 | 23.423 | 18.26 GB (46.12%) |
+| 1 | 64 | 64 | 516.544 | 23.3536 | 18.26 GB (46.12%) |
+| 1 | 128 | 128 | 643.968 | 23.3803 | 18.26 GB (46.12%) |
+| 1 | 256 | 256 | 736.236 | 23.389 | 18.34 GB (46.32%) |
+| 1 | 512 | 512 | 829.405 | 23.3889 | 18.54 GB (46.84%) |
+| 1 | 1024 | 1024 | 836.023 | 23.3757 | 18.95 GB (47.87%) |
+| 1 | 2048 | 2048 | 802.632 | 23.3777 | 20.25 GB (51.15%) |
+| 1 | 4096 | 4096 | 722.49 | 23.4252 | 25.38 GB (64.12%) |
+
 ## Reference
 
 If you find AWQ useful or relevant to your research, you can cite their [paper](https://arxiv.org/abs/2306.00978):