error about snakemake

[yoshihiko1218]

Hi, when I ran "snakemake -j 1", it return "Error: no Snakefile found, tried Snakefile, snakefile, workflow/Snakefile, workflow/snakefile."
Is there anything I might miss or should I run this in a specific location? snakemake was installed correctly.
Thank you!

[creisle]

Snakemake expects the default workflow file to be named "Snakefile" and in your current directory unless you pass it an argument to specify otherwise. So in this case it would need to be run in the directory where you cloned this repository. Is that where you ran it from?

[yoshihiko1218]

Thank you! I figured it out. Then in the next step when I run snakemake -j 48, I met following error:

Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 48
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Job stats:
job count min threads max threads

---

all 1 1 1
all_diseases 1 1 1
all_drugs 1 1 1
load_PMC4232638 1 1 1
load_PMC4468049 1 1 1
load_approvals 1 1 1
load_cancerhotspots 1 1 1
load_cgi 1 1 1
load_civic 1 1 1
load_do 1 1 1
load_docm 1 1 1
load_ensembl 1 1 1
load_local 6 1 1
load_moa 1 1 1
load_ncit 1 1 1
load_oncotree 1 1 1
load_refseq 1 1 1
load_uberon 1 1 1
total 23 1 1

Select jobs to execute...

[Thu Mar 3 16:10:21 2022]
rule load_local:
input: snakemake_data/local/chromosomes.json
output: snakemake_data/local-chromosomes.COMPLETE
log: snakemake_logs/local-chromosomes.logs.txt
jobid: 6
wildcards: local=chromosomes
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
rule load_local:
input: snakemake_data/local/evidenceLevels.json
output: snakemake_data/local-evidenceLevels.COMPLETE
log: snakemake_logs/local-evidenceLevels.logs.txt
jobid: 8
wildcards: local=evidenceLevels
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
rule load_local:
input: snakemake_data/local/aacr.json
output: snakemake_data/local-aacr.COMPLETE
log: snakemake_logs/local-aacr.logs.txt
jobid: 10
wildcards: local=aacr
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
rule load_local:
input: snakemake_data/local/asco.json
output: snakemake_data/local-asco.COMPLETE
log: snakemake_logs/local-asco.logs.txt
jobid: 12
wildcards: local=asco
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
rule load_approvals:
output: snakemake_data/fdaApprovals.COMPLETE
log: snakemake_logs/fdaApprovals.logs.txt
jobid: 30
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
rule load_local:
input: snakemake_data/local/signatures.json
output: snakemake_data/local-signatures.COMPLETE
log: snakemake_logs/local-signatures.logs.txt
jobid: 4
wildcards: local=signatures
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
rule load_local:
input: snakemake_data/local/vocab.json
output: snakemake_data/local-vocab.COMPLETE
log: snakemake_logs/local-vocab.logs.txt
jobid: 2
wildcards: local=vocab
resources: tmpdir=/tmp

[Thu Mar 3 16:10:21 2022]
Error in rule load_local:
jobid: 6
output: snakemake_data/local-chromosomes.COMPLETE
log: snakemake_logs/local-chromosomes.logs.txt (check log file(s) for error message)
shell:
node bin/load.js file ontology snakemake_data/local/chromosomes.json &> snakemake_logs/local-chromosomes.logs.txt; cp snakemake_logs/local-chromosomes.logs.txt snakemake_data/local-chromosomes.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Mar 3 16:10:21 2022]
Error in rule load_local:
jobid: 8
output: snakemake_data/local-evidenceLevels.COMPLETE
log: snakemake_logs/local-evidenceLevels.logs.txt (check log file(s) for error message)
shell:
node bin/load.js file ontology snakemake_data/local/evidenceLevels.json &> snakemake_logs/local-evidenceLevels.logs.txt; cp snakemake_logs/local-evidenceLevels.logs.txt snakemake_data/local-evidenceLevels.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Mar 3 16:10:21 2022]
Error in rule load_local:
jobid: 10
output: snakemake_data/local-aacr.COMPLETE
log: snakemake_logs/local-aacr.logs.txt (check log file(s) for error message)
shell:
node bin/load.js file ontology snakemake_data/local/aacr.json &> snakemake_logs/local-aacr.logs.txt; cp snakemake_logs/local-aacr.logs.txt snakemake_data/local-aacr.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Mar 3 16:10:21 2022]
Error in rule load_local:
jobid: 12
output: snakemake_data/local-asco.COMPLETE
log: snakemake_logs/local-asco.logs.txt (check log file(s) for error message)
shell:
node bin/load.js file ontology snakemake_data/local/asco.json &> snakemake_logs/local-asco.logs.txt; cp snakemake_logs/local-asco.logs.txt snakemake_data/local-asco.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Mar 3 16:10:21 2022]
Error in rule load_approvals:
jobid: 30
output: snakemake_data/fdaApprovals.COMPLETE
log: snakemake_logs/fdaApprovals.logs.txt (check log file(s) for error message)
shell:
node bin/load.js api fdaApprovals &> snakemake_logs/fdaApprovals.logs.txt; cp snakemake_logs/fdaApprovals.logs.txt snakemake_data/fdaApprovals.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Mar 3 16:10:21 2022]
Error in rule load_local:
jobid: 4
output: snakemake_data/local-signatures.COMPLETE
log: snakemake_logs/local-signatures.logs.txt (check log file(s) for error message)
shell:
node bin/load.js file ontology snakemake_data/local/signatures.json &> snakemake_logs/local-signatures.logs.txt; cp snakemake_logs/local-signatures.logs.txt snakemake_data/local-signatures.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Mar 3 16:10:21 2022]
Error in rule load_local:
jobid: 2
output: snakemake_data/local-vocab.COMPLETE
log: snakemake_logs/local-vocab.logs.txt (check log file(s) for error message)
shell:
node bin/load.js file ontology snakemake_data/local/vocab.json &> snakemake_logs/local-vocab.logs.txt; cp snakemake_logs/local-vocab.logs.txt snakemake_data/local-vocab.COMPLETE
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/majiayan/software/pori_graphkb_loader/.snakemake/log/2022-03-03T161021.599949.snakemake.log

[creisle]

I will need to see the log file(s) for the job that failed to be able to determine the cause of the error. Could you post the contents of one of the failing jobs? ex snakemake_logs/fdaApprovals.logs.txt

[yoshihiko1218]

Hi! the log looks like this:

node:internal/modules/cjs/loader:936
throw err;
^

Error: Cannot find module 'json-cycle'
Require stack:

• /home/majiayan/software/pori_graphkb_loader/src/graphkb.js
• /home/majiayan/software/pori_graphkb_loader/src/index.js
• /home/majiayan/software/pori_graphkb_loader/bin/load.js
  at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
  at Function.Module._load (node:internal/modules/cjs/loader:778:27)
  at Module.require (node:internal/modules/cjs/loader:999:19)
  at require (node:internal/modules/cjs/helpers:102:18)
  at Object. (/home/majiayan/software/pori_graphkb_loader/src/graphkb.js:5:12)
  at Module._compile (node:internal/modules/cjs/loader:1097:14)
  at Object.Module._extensions..js (node:internal/modules/cjs/loader:1151:10)
  at Module.load (node:internal/modules/cjs/loader:975:32)
  at Function.Module._load (node:internal/modules/cjs/loader:822:12)
  at Module.require (node:internal/modules/cjs/loader:999:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  '/home/majiayan/software/pori_graphkb_loader/src/graphkb.js',
  '/home/majiayan/software/pori_graphkb_loader/src/index.js',
  '/home/majiayan/software/pori_graphkb_loader/bin/load.js'
  ]
  }

Node.js v17.6.0

[creisle]

Are you running it with --use-singularity ? If so are you on the latest version of the loader? This looks similar to the bug we fixed in 6.2.0 for the singularity container where singularity was overriding the working directory of the docker container.

[yoshihiko1218]

My command was "snakemake -j 48". Also, I redownloaded latest version and ran again but got same result.

[yoshihiko1218]

Also, when I try to do npm install, I get error "Cannot read properties of null (reading 'pickAlgorithm')". I did some google research and saw a solution that says I can run "npm install -g npm@7.24.2". However, with this command, I get "11 packages are looking for funding run npm fund for details" and when I run "npm fund". it says "@bcgsc-pori/graphkb-loader@6.1.0". Is this tool need any application for use?
Sorry for bunch of question... I am new to npm : (

[creisle]

My command was "snakemake -j 48". Also, I redownloaded latest version and ran again but got same result.

I have been using node 16 since it tis the latest LTS version. Its possible it is related to the change in node versions. I will try installing node 17 here and see if I can reproduce your error

Also, when I try to do npm install, I get error "Cannot read properties of null (reading 'pickAlgorithm')". I did some google research and saw a solution that says I can run "npm install -g npm@7.24.2". However, with this command, I get "11 packages are looking for funding run npm fund for details" and when I run "npm fund". it says "@bcgsc-pori/graphkb-loader@6.1.0". Is this tool need any application for use?
Sorry for bunch of question... I am new to npm : (

npm fund is just for use if you want to donate to open source packages that ask for funding. It shouldn't affect the install process

[creisle]

I've looked into this a bit and it seems like there are some outstanding issues with node-gyp/node in node 17 still. I've found a number of tickets related to it for different packages but they appear to be outstanding issues still

Would you be able to try node 16, or using the docker container? With node, generally only the even numbered releases are long-term-stable (LTS) releases so we haven't added support for 17 officially and we will probably wait for 18.

If you need to use 17, did you run into any issues during the npm install stage? I suspect it may not have installed correctly

[yoshihiko1218]

Hi, I downgraded node version to stable version. I ran npm install and I think it succeed.
"(venv) (base) root@dell:/home/majiayan/software/pori_graphkb_loader-6.2.0# npm install
npm WARN deprecated xmldom@0.1.27: Deprecated due to CVE-2021-21366 resolved in 0.5.0
npm WARN deprecated uuid@3.3.2: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.

added 1180 packages in 3m

76 packages are looking for funding
run npm fund for details"

When I reran snakemake, it seems that error message has changed:
"(venv) (base) root@dell:/home/majiayan/software/pori_graphkb_loader-6.2.0# cat snakemake_logs/fdaApprovals.logs.txt
usage: load.js [-h] [-g GRAPHKB] [-u USERNAME] -p PASSWORD [--pubmed PUBMED]
[--errorLogPrefix ERRORLOGPREFIX] [--maxRecords MAXRECORDS]
{api,file,civic,cosmic} ...
load.js: error: the following arguments are required: -p/--password"

Are those database permission-required?

[creisle]

@yoshihiko1218 Sorry for the delay! I somehow missed your reply in my notifications. Yes those are the account details for the user that will be loading the data into the database via the API. If you have set up via the demo/dev instructions your username and password should be something like graphkb_importer and secret (note: these are for dev/testing only and users are expected to change them at a later date if they continue using the same instance)