AWQ-Bert / 4-bit Bert

[michaelfeil]

Hoping to add a implementation of 4bit Bert, potentially in casper-hansen/AutoAWQ#328. Contributions welcome

[casper-hansen]

Hi @michaelfeil, any chance you will look more closely into quantizing BERT models with AWQ? Your PR was off to a great start, but needs more experimentation to figure out how to scale a BERT model.

[michaelfeil]

@casper-hansen open for collaboration, but no further progress unfortunately.