You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have extended FairseqDataset and created a custom dataset implementation. How do I use this custom implementation to train against some particular model architecture using fairseq-train?
What have you tried?
Read the docs regarding Tasks and have a feeling that's the direction to go. Still curious about the extent to which the CLI tool fairseq-train can be used.
What's your environment?
fairseq Version (e.g., 1.0 or main): main
PyTorch Version (e.g., 1.0): 2.2.2
OS (e.g., Linux): Ubuntu 20.04
How you installed fairseq (pip, source): source
Build command you used (if compiling from source): pip installed from source as described in README.
Python version: 3.10
CUDA/cuDNN version: CUDA 11.8
The text was updated successfully, but these errors were encountered:
For anyone encountering the same issue, it probably varies quite a bit depending on your dataset, but I had to implement custom Task and FairseqDataset classes and then override setup_task() and load_dataset() behavior in my custom Task class.
❓ Questions and Help
Before asking:
What is your question?
I have extended FairseqDataset and created a custom dataset implementation. How do I use this custom implementation to train against some particular model architecture using
fairseq-train
?What have you tried?
Read the docs regarding Tasks and have a feeling that's the direction to go. Still curious about the extent to which the CLI tool
fairseq-train
can be used.What's your environment?
main
2.2.2
Ubuntu 20.04
pip
, source):source
3.10
CUDA 11.8
The text was updated successfully, but these errors were encountered: