SOLR-13350: multi-threaded search: replace cached with fixed threadpool

[cpoerschke]

https://issues.apache.org/jira/browse/SOLR-13350

[cpoerschke]

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/ThreadPoolExecutor.html documentation says "If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread." and "If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected."

I think this means that if the queue size is generous (relative to the use of the thread pool) then queuing will always be possible and search will remain single-threaded.

[dsmiley]

Can you recommend a way to benchmark this myself, like if I wanted to tweak it to see how my ideas work out?

[cpoerschke]

Can you recommend a way to benchmark this myself, like if I wanted to tweak it to see how my ideas work out?

I've shared the "just pass an executor" patch onto a Solr 9.5 cloud that I used, in case that's useful.

The benchmarking experiments I ran used Python's threading module and (I think under the hood) pysolr also, measuring QTime/elapsed time/CPU utilisation, for example. My experiments were for dense vector search, I haven't yet looked into if/how other searches might use the passed in executor at the Lucene level. But it would be worth mentioning at this point that the collection was quite multi-sharded i.e. that being independent of the type of search being run.

[gus-asf]

Along with this maybe we want to change

 public static final int DEFAULT_INDEX_SEARCHER_EXECUTOR_THREADS = 4;

to

public static final int DEFAULT_INDEX_SEARCHER_EXECUTOR_THREADS = Runtime.getRuntime().availableProcessors();

[cpoerschke]

Along with this maybe we want to change

 public static final int DEFAULT_INDEX_SEARCHER_EXECUTOR_THREADS = 4;

to

public static final int DEFAULT_INDEX_SEARCHER_EXECUTOR_THREADS = Runtime.getRuntime().availableProcessors();

Opened #2569 to separate that less controversial (?) change out from the less obvious (?) change here. Also there's some associated doc and config changes needed to go with making the default runtime aware.