About activation binarization

[AmirAliEbrahimi]

I used QuantIdentity with bit_width=1 and QuantType.BINARY, specifying a range of -1 to 1 for the activations binarization. This configuration uses brevitas.core.quant.binary.BinaryQuant, which uses binary_sign_ste for binarization. The binary_sign_ste implements STE as applies binary_sign during forward propagation and passes the gradient during backward propagation.
Inspired by this, I wrote my own implementation of binary_sign_ste activation module as:

class STEFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input):
        positive_mask = torch.ge(input, 0.0)
        negative_mask = torch.lt(input, 0.0)
        y = positive_mask.to(input.dtype) - negative_mask.to(input.dtype)
        return y

    @staticmethod
    def backward(ctx, grad_output):
        return grad_output


class StraightThroughEstimator(nn.Module):
    def __init__(self):
        super(StraightThroughEstimator, self).__init__()

    def forward(self, x):
        x = STEFunction.apply(x)
        return x

I am achieving approximately 96% accuracy with QuantIdentity, but when I use my own version, I am only getting around 92% accuracy. Is there something that I am missing? Or does QuantIdentity perform additional operations on the input/output tensors?

[AmirAliEbrahimi]

I was able to find a solution on my own. Instead of using BinaryQuant, my QuantIdentity utilized ClampedBinaryQuant. To achieve the desired outcome, I added a tensor clamp before applying the sign function. As a result, I am now obtaining the same results as intended.