Issues running on server (with Helix)

[LelsersLasers]

I use Helix a lot on my own laptop, but for my next university class I have to do all my work while logged into a university server. We are allowed to use any terminal based editor and I wanted to use Helix. As we don't have sudo permissions, I installed the latest version of Helix by downloading and scping the AppImage to the server. This all seemed to work and I could use Helix normally. Until I tried to use a picker. The first time I open any picker (file, jump list, etc), it works fine, but the second time I open one, it gives this error:

/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/nucleo-0.5.0/src/worker.rs:63:14:
creating threadpool failed: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

This same error occurs when using the Linux version (the one with the binary + the runtime folder).

I am not really sure what to do to try to fix this as it does not happen on my laptop (when using the same version of Helix, installed the same way). I also tried a blank/default config and I tried turning off the LSP.

The stack trace points to a particular place in a dependency nucleo, which seems to be helix' own implementation of a fuzzy finder. Specifically it is this part in the code: https://github.com/helix-editor/nucleo/blob/master/src/worker.rs#L59C1-L63C51

let pool = rayon::ThreadPoolBuilder::new()
    .thread_name(|i| format!("nucleo worker {i}"))
    .num_threads(worker_threads)
    .build()
    .expect("creating threadpool failed");

I tried running with RAYON_NUM_THREADS=1, but the issue persists. I am unsure what try next.

Thanks in advance for any help!

The full backtrace (doesn't look too much more helpful):

$ RUST_BACKTRACE=full hx
thread 'main' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/nucleo-0.5.0/src/worker.rs:63:14:
creating threadpool failed: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }
stack backtrace:
   0:     0x55a670ff0cd9 - <unknown>
   1:     0x55a6704de823 - <unknown>
   2:     0x55a670fb2a12 - <unknown>
   3:     0x55a670ff45f3 - <unknown>
   4:     0x55a670ff439f - <unknown>
   5:     0x55a6703fcf9f - <unknown>
   6:     0x55a670ff48b9 - <unknown>
   7:     0x55a670ff46c8 - <unknown>
   8:     0x55a670ff4629 - <unknown>
   9:     0x55a670ff461c - <unknown>
  10:     0x55a67039bd1f - <unknown>
  11:     0x55a67039c095 - <unknown>
  12:     0x55a6709b33b6 - <unknown>
  13:     0x55a670b4a62c - <unknown>
  14:     0x55a670a104ba - <unknown>
  15:     0x55a670a05a87 - <unknown>
  16:     0x55a6709e163b - <unknown>
  17:     0x55a670a25ebe - <unknown>
  18:     0x55a6703f0bd9 - <unknown>
  19:     0x55a670472da0 - <unknown>
  20:     0x55a6704449fe - <unknown>
  21:     0x55a670469ddc - <unknown>
  22:     0x55a6704101a3 - <unknown>
  23:     0x55a670475d64 - <unknown>
  24:     0x7f6ae4722d90 - __libc_start_call_main
                               at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  25:     0x7f6ae4722e40 - __libc_start_main_impl
                               at ./csu/../csu/libc-start.c:392:3
  26:     0x55a6703e9c45 - <unknown>
  27:                0x0 - <unknown>

[cuviper]

My guess is that your university server has a tighter limit on the number of user processes -- which includes threads on Linux. See what ulimit -u says, and check if that's lower than on your laptop. It may also be that the server has a much higher number of CPUs, so the rayon pool is attempting to create more threads in the first place.

I tried running with RAYON_NUM_THREADS=1, but the issue persists.

This only has an effect when the threads are otherwise unspecified, but the nucleo code is explicitly setting worker_threads itself. It has an Option parameter for that, but otherwise falls back to std::thread::available_parallelism(), or a final default 4.

AFAICS, the helix picker code always passes None there. It also looks like Picker::new is called from a lot of different places -- I don't know if those are persistent or short-lived, but they're not sharing thread-pools at all. ThreadPool::drop also doesn't wait for its threads (#688), so it's possible those pools have some overlapping existence that increases the overall thread count. (i.e. if they end one and start another, the old threads may still be exiting while it's trying to create new ones.)

I am unsure what try next.

You could set the CPU affinity to reduce available_parallelism, e.g. taskset -c 0-3 hx ... to run only on the first 4 CPUs.

But ultimately, I think you'll have to talk with the Helix developers about the way they're creating their worker pools.

[LelsersLasers]

Firstly, thank you so much for the quick reply!
Secondly, taskset -c 0-3 hx seems to solve the issue! I think the issue was it was trying to use all 208 CPUs and (as you explained) RAYON_NUM_THREADS=1 did not effectively limit the number of threads.

Just for reference:

$ ulimit -u
512

[cuviper]

For future reference, I opened a Helix RFE: helix-editor/helix#12625

[LelsersLasers]

Thanks!