Reads out FP16 parameters after quantization

[Jason202268]

After using scrips like

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

# Specify paths and hyperparameters for quantization
quant_path = "./models/test"
quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM" }

# Load your tokenizer and model with AutoAWQ
model = AutoAWQForCausalLM.from_pretrained(model_path, device_map="auto", safetensors=True)
model.quantize(tokenizer, quant_config=quant_config, calib_data=formatted_data)
model.save_quantized(quant_path, safetensors=True, shard_size="4GB")
tokenizer.save_pretrained(quant_path)

I manually checked the parameters using safetensor library and the dtype shows FP16, is this normal?