run future on a cluster of windows-wsl2 machines and ubuntu machines

[Hiroto-Miyoshi]

I used to run a script on a cluster of machines connected by LAN. Two of the machines run the script in wsl2 on windows 11, and the other two machines run ubuntu 22.04LTS,

The script contained nested future (actually, future_map), the first one for allocating lists to different machines and the second one for allocating data to logical cores in each machine. The script worked just fine before.

However, recently I found it did have a problem in communicating between wsl2 machines and ubuntu machines. When wsl2 machine is the main machine, another wsl2 machine works just fine as a slave, but the other two ubuntu machines fail to participate in computing, and when ubuntu machine becomes the main machine, wsl2 machines fail to work. As a result, future_map produces an error, recieving output from failed machines.

I should add that I can ssh from any machine to any other machine without a problem. (I mean wsl2 to wsl2 or ubuntu, and ubuntu to wsl2 or ubuntu.)

The following is a model script which shows my exact problem.

1 library(tidyverse)
1 library(parallelly)
2 library(furrr)
3
4 options(future.globals.maxSize = 700 * 1024^2)
5 #
6 # specify how many nodes are planned to use
7 #
8 node1 <- "192.168.1.10" # hiroto-home-new-wsl
9 node2 <- "192.168.1.11" # VivoBook-wsl
10 node3 <- "192.168.1.12" # hiroto-2013
11 node4 <- "192.168.1.13" # hiroto-ThinkPad
12 cl <- makeClusterPSOCK(c(node1, node2, node3, node4))

13 #
14 # custom_workers
15 #
16 custom_workers <- function() {
17 switch(Sys.info()[["nodename"]],
18 "Hiroto-home-new" = 14L,
19 "VivoBook" = 15L,
20 "hiroto-2013" = 7L,
21 "hiroto-ThinkPad" = 3L,
22 )
23 }
24
24 clstr1 <- 14
23 clstr2 <- 15
22 clstr3 <- 7
21 clstr4 <- 3
20
19 n_cores <- clstr1 + clstr2 + clstr3 + clstr4
18
17 plan(list(
16 tweak(cluster, workers = cl),
15 tweak(multisession, workers = custom_workers)
14 ))

10
9 f_clstr <- function(param_c) {
8 param_n <- split(param_c, param_c$nds)
7 print(param_n)
6 future_map(param_n, f_nds)
5 }
4
3 f_nds <- function(param_n) {
2 n <- I(-100000000)
1 while (n < 10000000) n <- n + 1
50 }
10 nds <- 0:n_cores
9 clstr <- case_when(
8 nds < clstr1 ~ 10,
7 nds < (clstr1 + clstr2) ~ 11,
6 nds < (clstr1 + clstr2 + clstr3) ~ 12
5 nds >= (clstr1 + clstr2 + clstr3) ~ 13
4 )
3 param <- tibble(clstr = clstr, nds = nds) %>%
2 split(., clstr)
1 print(param)
62 future_map(param, f_clstr)

When I ran the script, warning message appeared:

During startup - Warning messages:
1: Setting LC_TIME failed, using "C"
2: Setting LC_MONETARY failed, using "C"
3: Setting LC_PAPER failed, using "C"
4: Setting LC_MEASUREMENT failed, using "C"

What do they mean, why did they appear in recent runs, and how can I solve this problem?
I need your help. Any suggestion would be greatly appreciated.
Thank you.

Hiroto