python3.6 train_lstm.py is giving Illegal instruction (core dumped)

[srini8080]

python3.6 train_lstm.py is giving Illegal instruction (core dumped)

[srini8080]

(567): (568): (571): --- modulename: _bootstrap_external, funcname: create_module
(921): (922): --- modulename: _bootstrap, funcname: _call_with_frames_removed
Illegal instruction (core dumped)

[safakkbilici]

It should work with the dependencies that I mentioned in the project. I think that this error is about hardware utilization. Did you run the code on CPU? May I learn your hardware properties?

[srini8080]

Dell server, 4 CPUs Xeon, 6 cores each, 64GB RAM. I am trying to use the project to help children make titles for their stories. On Tuesday, December 29, 2020, 02:47:16 PM GMT+5:30, Şafak Bilici <[email protected]> wrote: It should work with the dependencies that I mentioned in the project. I think that this error is about hardware utilization. Did you run the code on CPU? May I learn your hardware properties? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[safakkbilici]

Okay, I don't know what is the problem, it works on my computer and my colleagues' computer. I suggest you to train the T5 model to make titles. We used seq2seq lstm as a baseline model, it didn't give good results. But T5 works very good. Also I suggest you to train T5 on colab.

[srini8080]

Thanks, In all honesty, I am very new to the whole LSTM or NLP. I Just work on books. Should I just go into the T5 directory and run the generate and train?  Any datasets you recommend? Thanks On Wednesday, December 30, 2020, 12:39:48 AM GMT+5:30, Şafak Bilici <[email protected]> wrote: Okay, I don't know what is the problem, it works on my computer and my colleagues' computer. I suggest you to train the T5 model to make titles. We used seq2seq lstm as a baseline model, it didn't give good results. But T5 works very good. Also I suggest you to train T5 on colab. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[safakkbilici]

Yes, train.py is given for colab. It also contains script for mounting google drive inside, because it is more practical to save your checkpoints into arbitrary drive directory. By doing that you can use your model after with only loading checkpoints. In train.py model_args has

"best_model_dir": "/content/drive/My Drive/outputs/best_model",
"output_dir": "/content/drive/My Drive/outputs"

That means your checkpoints are saved into "/content/drive/My Drive/outputs" and "/content/drive/My Drive/outputs/best_model". Feel free to change them.

Then you can use generate.py for loading checkpoints with just a line:

model = T5Model("t5","/content/drive/My Drive/outputs/best_model", args=model_args)

But I do not know a dataset which contains strory/title pair :(

[srini8080]

Can I use the Nytimes articles data or aixrv one to start with? On Wednesday, December 30, 2020, 03:36:14 PM GMT+5:30, Şafak Bilici <[email protected]> wrote: Yes, train.py is given for colab. It also contains script for mounting google drive inside, because it is more practical to save your checkpoints into arbitrary drive directory. By doing that you can use your model after with only loading checkpoints. In train.py model_args has "best_model_dir": "/content/drive/My Drive/outputs/best_model", "output_dir": "/content/drive/My Drive/outputs" That means your checkpoints are saved into "/content/drive/My Drive/outputs" and "/content/drive/My Drive/outputs/best_model". Feel free to change them. Then you can use generate.py for loading checkpoints with just a line: model = T5Model("t5","/content/drive/My Drive/outputs/best_model", args=model_args) But I do not know a dataset which contains strory/title pair :( — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[safakkbilici]

If the data has article/title pairs within pandas dataframe, you can use it yes. Do not forget that make your input column name input_text and target column name (in this example titles) target_text.
Btw I haven't heard this data. May you send me a link?

[srini8080]

https://www.kaggle.com/aashita/nyt-comments On Thursday, December 31, 2020, 06:05:46 PM GMT+5:30, Şafak Bilici <[email protected]> wrote: If the data has article/title pairs within pandas dataframe, you can use it yes. Do not forget that make your input column name input_text and target column name (in this example titles) target_text. Btw I haven't heard this data. May you send me a link? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.