speaker diarization inference Error: Missing 'speaker_model_cfg' Key in Configuration When Using Pretrained CKPT File

[sipercai]

When running inference with the multiscale_diar_decoder_infer.py script after training the model, an error occurs related to a missing configuration attribute (speaker_model_cfg). The error traceback indicates that the configuration used for inference does not contain the required speaker_model_cfg key.

Steps/Code to reproduce bug

1. Train the model using the following command:

   ./multiscale_diar_decoder.py --config-path=../conf/neural_diarizer --config-name=msdd_5scl_15_05_50Povl_256x3x32x2.yaml model.train_ds.manifest_filepath=/data2/xxx_data/SD/nemo/msdwild/train/msdd_data.50step.json model.validation_ds.manifest_filepath=/data2/xxx_data/SD/nemo/msdwild/test/msdd_data.50step.json model.train_ds.emb_dir=/data2/xxx_data/SD/nemo/msdwild/train model.validation_ds.emb_dir=/data2/xxx_data/SD/nemo/msdwild/test exp_manager.name=sample_train exp_manager.exp_dir=./msdd_exp

2. After training, use the following command to run inference:

   python multiscale_diar_decoder_infer.py --config-path='../conf/inference' --config-name='diar_infer_telephonic.yaml' \
   diarizer.msdd_model.model_path='/home/data/xxx/project/SD/NeMo/examples/speaker_tasks/diarization/neural_diarizer/msdd_exp/sample_train/2024-12-17_12-32-23/checkpoints/test.ckpt' \
   diarizer.oracle_vad=True \
   diarizer.manifest_filepath='/data2/xxx_data/SD/nemo/msdwild/train/msdd_data.50step.json' \
   diarizer.out_dir='output'

3. The following error occurs during inference:

   Traceback (most recent call last):
   File "/home/data/xxx/project/SD/NeMo/examples/speaker_tasks/diarization/neural_diarizer/multiscale_diar_decoder_infer.py", line 31, in main
   diarizer_model = NeuralDiarizer(cfg=cfg).to(cfg.device)
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/asr/models/msdd_models.py", line 1019, in __init__
   self._init_msdd_model(cfg)
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/asr/models/msdd_models.py", line 1114, in _init_msdd_model
   self.msdd_model = EncDecDiarLabelModel.load_from_checkpoint(
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/nemo/core/classes/modelPT.py", line 506, in load_from_checkpoint
   checkpoint = super().load_from_checkpoint(
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/lightning/pytorch/utilities/model_helpers.py", line 125, in wrapper
   return self.method(cls, *args, **kwargs)
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/lightning/pytorch/core/module.py", line 1582, in load_from_checkpoint
   loaded = _load_from_checkpoint(
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/lightning/pytorch/core/saving.py", line 91, in _load_from_checkpoint
   model = _load_state(cls, checkpoint, strict=strict, **kwargs)
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/lightning/pytorch/core/saving.py", line 165, in _load_state
   obj = instantiator(cls, _cls_kwargs) if instantiator else cls(**_cls_kwargs)
   File "/home/data/xxx/anaconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/asr/models/msdd_models.py", line 145, in __init__
   self.msdd._speaker_model = EncDexxxeakerLabelModel.from_config_dict(cfg.speaker_model_cfg)
   omegaconf.errors.ConfigAttributeError: Key 'speaker_model_cfg' is not in struct
   full_key: cfg.speaker_model_cfg
   object_type=dict
   

I found a similar issue in #8463, but I am passing the ckpt file and the issue is not resolved.
Could you please help?