Skip to content

Commit 6445f69

Browse files
committed
doc: document the pool parameter in decorators
Signed-off-by: Matteo Cafasso <[email protected]>
1 parent 56eeb04 commit 6445f69

File tree

1 file changed

+34
-4
lines changed

1 file changed

+34
-4
lines changed

doc/index.rst

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
1919
`Concurrent Module`
2020
-------------------
2121

22-
.. decorator:: concurrent.process(timeout=None, name=None, daemon=True, context=None)
22+
.. decorator:: concurrent.process(timeout=None, name=None, daemon=True, context=None, pool=None)
2323

2424
Runs the decorated function in a concurrent process, taking care of the results and error management.
2525

@@ -33,7 +33,9 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
3333

3434
The *context* parameter can be used to specify the multiprocessing.context_ object used for starting the process.
3535

36-
.. decorator:: concurrent.thread(name=None, daemon=True)
36+
The *pool* parameter accepts a pebble.ProcessPool_ object. If provided, the pool will be used to run the function instead of a dedicated process. The *name*, *daemon* and *context* parameters will be ignored.
37+
38+
.. decorator:: concurrent.thread(name=None, daemon=True, pool=None)
3739

3840
Runs the decorated function in a concurrent thread, taking care of the results and error management.
3941

@@ -43,11 +45,12 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
4345

4446
The *daemon* parameter switches between daemon and non-daemon threads.
4547

48+
The *pool* parameter accepts a pebble.ThreadPool_ object. If provided, the pool will be used to run the function instead of a dedicated thread. The *name* and *daemon* parameters will be ignored.
4649

4750
`Asynchronous Module`
4851
---------------------
4952

50-
.. decorator:: asynchronous.process(timeout=None, name=None, daemon=True, context=None)
53+
.. decorator:: asynchronous.process(timeout=None, name=None, daemon=True, context=None, pool=None)
5154

5255
Runs the decorated function in a concurrent process, taking care of the results and error management.
5356

@@ -61,7 +64,9 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
6164

6265
The *context* parameter can be used to specify the multiprocessing.context_ object used for starting the process.
6366

64-
.. decorator:: asynchronous.thread(name=None, daemon=True)
67+
The *pool* parameter accepts a pebble.ProcessPool_ object. If provided, the pool will be used to run the function instead of a dedicated process. The *name*, *daemon* and *context* parameters will be ignored.
68+
69+
.. decorator:: asynchronous.thread(name=None, daemon=True, pool=None)
6570

6671
Runs the decorated function in a concurrent thread, taking care of the results and error management.
6772

@@ -71,10 +76,13 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
7176

7277
The *daemon* parameter switches between daemon and non-daemon threads.
7378

79+
The *pool* parameter accepts a pebble.ThreadPool_ object. If provided, the pool will be used to run the function instead of a dedicated thread. The *name* and *daemon* parameters will be ignored.
80+
7481

7582
`Pebble Module`
7683
---------------
7784

85+
.. _pebble.ProcessPool:
7886
.. class:: pebble.ProcessPool(max_workers=multiprocessing.cpu_count(), max_tasks=0, initializer=None, initargs=None, context=None)
7987

8088
A Pool allows to schedule jobs into a Pool of Processes which will perform them concurrently.
@@ -130,6 +138,7 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
130138

131139
The *join* function must be called only in the main loop. Calling it in a pebble.ProcessFuture_ callback will result in a deadlock.
132140

141+
.. _pebble.ThreadPool:
133142
.. class:: pebble.ThreadPool(max_workers=multiprocessing.cpu_count(), max_tasks=0, initializer=None, initargs=None)
134143

135144
A ThreadPool allows to schedule jobs into a Pool of Threads which will perform them concurrently.
@@ -474,6 +483,27 @@ If a `timeout` is provided, it will be applied to the whole chunk and not to the
474483

475484
assert list(future.result()) == elements
476485

486+
The `concurrent` and `asynchronous` decorators accept a *pool* parameter. This is useful to control how many instances of decorated functions can be run at the same time.
487+
488+
::
489+
490+
from concurrent.futures import wait
491+
from pebble import concurrent, ProcessPool
492+
493+
pool = ProcessPool(max_workers=4)
494+
495+
@concurrent.process(pool=pool)
496+
def function(arg, kwarg=0):
497+
return arg + kwarg
498+
499+
futures = []
500+
501+
# Maximum 4 executions of `function` will be executed in parallel
502+
for _ in range(100):
503+
futures.append(function(1, kwarg=1))
504+
505+
wait(futures)
506+
477507

478508
Pools and AsyncIO
479509
+++++++++++++++++

0 commit comments

Comments
 (0)