Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force single-threaded RcppParallel processing in forked parallelization #482

Closed
HenrikBengtsson opened this issue Mar 25, 2021 · 3 comments
Milestone

Comments

@HenrikBengtsson
Copy link
Collaborator

RcppParallel::setThreadOptions(numThreads = 1L)

https://rcppcore.github.io/RcppParallel/

@HenrikBengtsson HenrikBengtsson transferred this issue from another repository Mar 25, 2021
@HenrikBengtsson HenrikBengtsson changed the title future: Force single-threaded RcppParallel processing in nested parallelization Force single-threaded RcppParallel processing in nested parallelization Apr 4, 2021
@HenrikBengtsson HenrikBengtsson transferred this issue from another repository Apr 4, 2021
@HenrikBengtsson
Copy link
Collaborator Author

Forcing single-threaded processing in nested parallel workers is the most conservative approach. We could relax this a bit by looking at the number of threads per core the CPU supports, cf. futureverse/parallelly#47

@HenrikBengtsson
Copy link
Collaborator Author

HenrikBengtsson commented Apr 9, 2021

Can also just set environment variable RCPP_PARALLEL_NUM_THREADS; that avoids introducing a new package dependency. From RcppParallel 4.4.0 (2018-03-02) NEWS:

  • Allow setting the number of threads to use via RCPP_PARALLEL_NUM_THREADS environment variable.

RcppParallel::setThreadOptions() sets this internally;

> RcppParallel::setThreadOptions
function (numThreads = "auto", stackSize = "auto") 
{
    if (identical(numThreads, "auto")) 
        numThreads <- -1L
    else if (!is.numeric(numThreads)) 
        stop("numThreads must be an integer")
    else numThreads <- as.integer(numThreads)
    if (identical(stackSize, "auto")) 
        stackSize <- 0L
    else if (!is.numeric(stackSize)) 
        stop("stackSize must be an integer")
    else stackSize <- as.integer(stackSize)
    if (!is.null(dllInfo) && isUsingTbb()) 
        setTbbThreadOptions(numThreads, stackSize)
    if (numThreads == -1L) 
        Sys.unsetenv("RCPP_PARALLEL_NUM_THREADS")
    else Sys.setenv(RCPP_PARALLEL_NUM_THREADS = numThreads)
}
<environment: namespace:RcppParallel>

@HenrikBengtsson HenrikBengtsson added this to the Next release milestone Apr 9, 2021
@HenrikBengtsson HenrikBengtsson changed the title Force single-threaded RcppParallel processing in nested parallelization Force single-threaded RcppParallel processing in forked parallelization Jun 1, 2021
@HenrikBengtsson
Copy link
Collaborator Author

Starting by doing it in forked parallel processing when option future.fork.multithreading.enable=FALSE, cf. how with already do it for OpenMP.

HenrikBengtsson added a commit that referenced this issue Jun 1, 2021
variable 'RCPP_PARALLEL_NUM_THREADS' to 1 if using multicore futures
[#482]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant