Unable to use Seq2SeqTrainingArguments and Seq2SeqTrainer

[shivanraptor]

System Info

I am using the following versions:

• tensorflow 2.18.0
• tensorboard 2.18.0
• transformers 4.49.0
• keras 3.8.0
• Python 3.10.0

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

I want to use Seq2SeqTrainingArguments and Seq2SeqTrainer. The code was working using old version of transformers 4.28.1.

Here is the minimal executable example:

import transformers
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer

training_args = Seq2SeqTrainingArguments()

The main error is:

RuntimeError: Failed to import transformers.trainer_seq2seq because of the following error (look up to see its traceback):
Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tensorflow._api.v2.compat.v2.__internal__' has no attribute 'register_load_context_function'

I am using the following versions:

• tensorflow 2.18.0
• tensorboard 2.18.0
• transformers 4.49.0
• keras 3.8.0
• Python 3.10.0

But the official example uses similar codes: https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/run_translation.py#L595

My question is: apart from downgrading the transformers, how do I fix this issue? Thanks.

---

Full error is:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1863, in _LazyModule._get_module(self, module_name)
   1862 try:
-> 1863     return importlib.import_module("." + module_name, self.__name__)
   1864 except Exception as e:

File /opt/tljh/user/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/.local/lib/python3.10/site-packages/transformers/modeling_tf_utils.py:38
     37 from . import DataCollatorWithPadding, DefaultDataCollator
---> 38 from .activations_tf import get_tf_activation
     39 from .configuration_utils import PretrainedConfig

File ~/.local/lib/python3.10/site-packages/transformers/activations_tf.py:22
     21 try:
---> 22     import tf_keras as keras
     23 except (ModuleNotFoundError, ImportError):

File ~/.local/lib/python3.10/site-packages/tf_keras/__init__.py:3
      1 """AUTOGENERATED. DO NOT EDIT."""
----> 3 from tf_keras import __internal__
      4 from tf_keras import activations

File ~/.local/lib/python3.10/site-packages/tf_keras/__internal__/__init__.py:6
      5 from tf_keras.__internal__ import losses
----> 6 from tf_keras.__internal__ import models
      7 from tf_keras.__internal__ import optimizers

File ~/.local/lib/python3.10/site-packages/tf_keras/__internal__/models/__init__.py:3
      1 """AUTOGENERATED. DO NOT EDIT."""
----> 3 from tf_keras.src.models.cloning import clone_and_build_model
      4 from tf_keras.src.models.cloning import in_place_subclassed_model_state_restoration

File ~/.local/lib/python3.10/site-packages/tf_keras/src/__init__.py:21
     15 """Implementation of the TF-Keras API, the high-level API of TensorFlow.
     16 
     17 Detailed documentation and user guides are available at
     18 [keras.io](https://keras.io).
     19 """
---> 21 from tf_keras.src import applications
     22 from tf_keras.src import distribute

File ~/.local/lib/python3.10/site-packages/tf_keras/src/applications/__init__.py:18
     15 """Keras Applications are premade architectures with pre-trained weights."""
---> 18 from tf_keras.src.applications.convnext import ConvNeXtBase
     19 from tf_keras.src.applications.convnext import ConvNeXtLarge

File ~/.local/lib/python3.10/site-packages/tf_keras/src/applications/convnext.py:33
     32 from tf_keras.src.applications import imagenet_utils
---> 33 from tf_keras.src.engine import sequential
     34 from tf_keras.src.engine import training as training_lib

File ~/.local/lib/python3.10/site-packages/tf_keras/src/engine/sequential.py:24
     23 from tf_keras.src.engine import base_layer
---> 24 from tf_keras.src.engine import functional
     25 from tf_keras.src.engine import input_layer

File ~/.local/lib/python3.10/site-packages/tf_keras/src/engine/functional.py:33
     32 from tf_keras.src.engine import node as node_module
---> 33 from tf_keras.src.engine import training as training_lib
     34 from tf_keras.src.engine import training_utils

File ~/.local/lib/python3.10/site-packages/tf_keras/src/engine/training.py:48
     47 from tf_keras.src.saving import pickle_utils
---> 48 from tf_keras.src.saving import saving_api
     49 from tf_keras.src.saving import saving_lib

File ~/.local/lib/python3.10/site-packages/tf_keras/src/saving/saving_api.py:25
     24 from tf_keras.src.saving import saving_lib
---> 25 from tf_keras.src.saving.legacy import save as legacy_sm_saving_lib
     26 from tf_keras.src.utils import io_utils

File ~/.local/lib/python3.10/site-packages/tf_keras/src/saving/legacy/save.py:27
     26 from tf_keras.src.saving.legacy.saved_model import load as saved_model_load
---> 27 from tf_keras.src.saving.legacy.saved_model import load_context
     28 from tf_keras.src.saving.legacy.saved_model import save as saved_model_save

File ~/.local/lib/python3.10/site-packages/tf_keras/src/saving/legacy/saved_model/load_context.py:68
     65     return _load_context.in_load_context()
---> 68 tf.__internal__.register_load_context_function(in_load_context)

AttributeError: module 'tensorflow._api.v2.compat.v2.__internal__' has no attribute 'register_load_context_function'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1863, in _LazyModule._get_module(self, module_name)
   1862 try:
-> 1863     return importlib.import_module("." + module_name, self.__name__)
   1864 except Exception as e:

File /opt/tljh/user/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/.local/lib/python3.10/site-packages/transformers/integrations/integration_utils.py:36
     34 import packaging.version
---> 36 from .. import PreTrainedModel, TFPreTrainedModel
     37 from .. import __version__ as version

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1851, in _LazyModule.__getattr__(self, name)
   1850 elif name in self._class_to_module.keys():
-> 1851     module = self._get_module(self._class_to_module[name])
   1852     value = getattr(module, name)

File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1865, in _LazyModule._get_module(self, module_name)
   1864 except Exception as e:
-> 1865     raise RuntimeError(
   1866         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1867         f" traceback):\n{e}"
   1868     ) from e

RuntimeError: Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tensorflow._api.v2.compat.v2.__internal__' has no attribute 'register_load_context_function'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1863, in _LazyModule._get_module(self, module_name)
   1862 try:
-> 1863     return importlib.import_module("." + module_name, self.__name__)
   1864 except Exception as e:

File /opt/tljh/user/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/.local/lib/python3.10/site-packages/transformers/trainer_seq2seq.py:29
     28 from .integrations.fsdp import is_fsdp_managed_module
---> 29 from .trainer import Trainer
     30 from .utils import is_datasets_available, logging

File ~/.local/lib/python3.10/site-packages/transformers/trainer.py:42
     40 # Integrations must be imported before ML frameworks:
     41 # isort: off
---> 42 from .integrations import (
     43     get_reporting_integration_callbacks,
     44 )
     46 # isort: on

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1851, in _LazyModule.__getattr__(self, name)
   1850 elif name in self._class_to_module.keys():
-> 1851     module = self._get_module(self._class_to_module[name])
   1852     value = getattr(module, name)

File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1865, in _LazyModule._get_module(self, module_name)
   1864 except Exception as e:
-> 1865     raise RuntimeError(
   1866         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1867         f" traceback):\n{e}"
   1868     ) from e

RuntimeError: Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tensorflow._api.v2.compat.v2.__internal__' has no attribute 'register_load_context_function'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[2], line 2
      1 import transformers
----> 2 from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
      4 training_args = Seq2SeqTrainingArguments()

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1851, in _LazyModule.__getattr__(self, name)
   1849     value = Placeholder
   1850 elif name in self._class_to_module.keys():
-> 1851     module = self._get_module(self._class_to_module[name])
   1852     value = getattr(module, name)
   1853 elif name in self._modules:

File ~/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py:1865, in _LazyModule._get_module(self, module_name)
   1863     return importlib.import_module("." + module_name, self.__name__)
   1864 except Exception as e:
-> 1865     raise RuntimeError(
   1866         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1867         f" traceback):\n{e}"
   1868     ) from e

RuntimeError: Failed to import transformers.trainer_seq2seq because of the following error (look up to see its traceback):
Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tensorflow._api.v2.compat.v2.__internal__' has no attribute 'register_load_context_function'

Steps to reproduce:

1. Install transformers 4.49.0 and other libraries listed above (with their dependencies installed); no conflict is reported.
2. Write the minimal example above in the Jupyter Notebook / a Python script (results are the same).
3. The above errors occurred.

Expected behavior

In transformers 4.28.1, the Seq2SeqTrainingArguments won't cause errors. The translation example in this GitHub is using the same code as well. No error should be expected.

[Rocketknight1]

hi @shivanraptor, the error seems to be occuring when the tf-keras package is imported on your system. Please make sure this package is updated to match your version of tensorflow! https://pypi.org/project/tf-keras/