Skip to content

Error with Pytorch metrics during PTQ Static #2087

Open
@DGP1607

Description

@DGP1607

Encountering an issue while PTQ Static on Pytorch Model. The process involves utilizing pytorch metrics for benchmarking such as ['Accuracy','F1']. The workflow is executed within the VS code Jupyter extension.

The issue arises as follows :

  • During the first execution of the entire notebook, everything runs smoothly without any issues.
  • However, upon executing the notebook for a second time, I encounter an error.Upon restarting the kernel in VS code it is running fine. But this should not be the case everytime why a restart of kerenel is required for metric.
  • The traceback for the error is as follows:
    AssertionError Traceback (most recent call last)
    Cell In[91], line 10
    7 metrics = METRICS('pytorch')
    8 top2 = metrics'Accuracy'
    ---> 10 q_model = quantization.fit(
    11 model=model,
    12 conf=conf,
    13 calib_dataloader=dataloader,
    14 eval_dataloader=dataloader,
    15 eval_metric=top2,
    16 )

    in fit(model, conf, calib_dataloader, calib_func, eval_func, eval_dataloader, eval_metric, **kwargs)
    151 wrapped_model = Model(model, conf=conf)
    153 if eval_metric is not None:
    --> 154 metric = register_customer_metric(eval_metric, conf.framework)
    155 else:
    156 metric = None
    in register_customer_metric(user_metric, framework)
    1681 metric_cfg = {name: id(user_metric)}
    1682 metrics = METRICS(framework)
    -> 1683 metrics.register(name, metric_cls)
    1684 return metric_cfg
    ...
    221 """
    --> 222 assert name not in self.metrics.keys(), "registered metric name already exists."
    223 self.metrics.update({name: metric_cls})

AssertionError: registered metric name already exists.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions