DOCS: Further clarification on nested parallelization requiring I(.)

[HenrikBengtsson]

From #725 (comment)_:

Apologies just found my way to this issue after being very confused by the example in the "A Comprehensive Overview" vignette. I'm going to guess I'm not the only one that tried to adapt the example shown there, that is I appreciate you have added more complete documentation in the topologies vignette but I'm wondering if it would be worth adding a note or additional explanation in the overview vignette to make this more clear ?

Action

• Add explicit example showing what happens when one does not use I(.) to declare nested parallel workers, e.g.

library("future")
plan(list(
  tweak(multisession, workers = 2),
  tweak(multisession, workers = 4)
))

a %<-% { b %<-% 1 ; b }
a
#> Error in checkNumberOfLocalWorkers(workers) : 
#>  Attempting to set up 4 localhost parallel workers with only 1 CPU cores available for this R process
(per ‘mc.cores’), which could result in a 400% load. The hard limit is set to 300%. Overusing the CPUs
has negative impact on the current R process, but also on all other processes of yours and others running
on the same machine. See help("parallelly.options", package = "parallelly") for how to override the soft
and hard limits

[HenrikBengtsson]

@gowerc, you wrote: "I'm wondering if it would be worth adding a note or additional explanation in the overview vignette to make this more clear". Do you have any suggestions on a sentence or a paragraph that would help clarify this?

[gowerc]

Not sure how useful / accurate / appropriate this is for this section of the vignette but the following are the notes I made for myself to explain this back to me when I inevitably forget 😓 :

The future package will raise a warning if you try to use more cores than what is believed to be available by availableCores(). By default, as a safety measure, future will set the value of availableCores() to be 1 within workers; this means that by default when creating nested workers you will always have a warning raised, note that this doesn't stop multiple nested workers being spawned it is simply just trying to warn you of potentially problematic behaviour. You can suppress this warning message via the use of I() in the call to plan() e.g.

plan(
    list(
        tweak(multisession, workers = 2),
        tweak(multisession, workers = I(3))
    )
)