doJobCollection error

[mkrzak]

Hi,
I am running future.batchtools with on LSF cluster and getting error:

Error : BatchtoolsFutureError for BatchtoolsFuture ('future_lapply-1') captured by batchtools: ‘Failed to subset JobCollection using array environment variable 'NA' [='NA']’

I am running it using "Rscript run.R" on terminal where run.R contains:

library(doFuture)
library(future.batchtools)

myProc <- function(size=10000000) {
vec <- rnorm(size)
return(sum(vec))
}

future::plan(
future.batchtools::batchtools_lsf,
template="~/repos/myrepos/parallel-computing/batchtools/lsf.tmpl",
resources=list(
queue="normal",
walltime=60,#minutes
memory=8192,
ncpus=2,
name="merging",
output="out.txt",
chunks.as.arrayjobs=TRUE))

ha = future.apply::future_lapply(1:10, function(n){myProc(n)})

Additionally I have lsf.tmpl file with:

#BSUB -J <%= resources$name %> # Name of the job
#BSUB -o <%= resources$output %> # Output is sent to logfile, stdout + stderr > by default
#BSUB -q <%= resources$queue %> # Job queue
#BSUB -W <%= round(resources$walltime / 60, 1) %> # Walltime (LSF requires minutes, > batchtools uses seconds)
#BSUB -M <%= resources$memory %> # Memory requirements, e.g. "5000KB", > "500MB", "5GB" etc.
export DEBUGME=<%= Sys.getenv("DEBUGME") %>
<%= sprintf("export OMP_NUM_THREADS=%i", resources$omp.threads) -%>
<%= sprintf("export OPENBLAS_NUM_THREADS=%i", resources$blas.threads) -%>
<%= sprintf("export MKL_NUM_THREADS=%i", resources$blas.threads) -%>
Rscript -e 'batchtools::doJobCollection("<%= uri %>")'

And batchtools.conf.R file with:

default.resources=list(walltime=60,
memory=1024,
ncpus=2)

I see jobs are submitted to the cluster, however still getting this error. I checked the log (out.txt) file which shows the jobs and evaluation of doJobCollection functions i.e. :

Rscript -e 'batchtools::doJobCollection("/..pathtojobfile../joba7f53f7835b2d0e33a6e2fe563c6f6da.rds")'

I readRDS that file in R and its not empty , however execution of batchtools::doJobCollection("/..pathtojobfile../joba7f53f7835b2d0e33a6e2fe563c6f6da.rds") inside R is NULL. Do you have any idea if the problem lays in lsf.tmpl file or I am missing calling some other batchtools functions in my run.R script?