Python multiprocessing
last modified November 17, 2026 Python 3.15
In this article we show how to use the multiprocessing module
for process-based parallelism in Python.
The multiprocessing module runs code in separate processes. Each
process has its own Python interpreter, its own memory, and its own global
interpreter lock (GIL), so CPU-bound code can use all processors of a
machine. The API is similar to the threading module.
Processes run independently; the order in which they finish is not guaranteed. There is no guarantee that the first process to be created will be the first to complete.
The GIL
A global interpreter lock (GIL) is a mechanism used in the CPython interpreter to synchronize the execution of threads so that only one thread executes Python bytecode at a time, even on a multi-core processor.
C extensions such as NumPy can release the GIL during long computations, and the GIL is released during blocking I/O operations. Pure Python code that is CPU-bound does not run faster with threads, however. Python 3.13 introduced an optional free-threaded build without the GIL, and since Python 3.14 it is officially supported, but the standard build still has the GIL. Separate processes avoid the limitation in all builds.
Concurrency and parallelism
Concurrency means that two or more calculations happen within the same time frame. Parallelism means that two or more calculations happen at the same moment. Parallelism is therefore a specific case of concurrency. It requires multiple CPU units or cores.
Python has three main modules for concurrency: multiprocessing,
threading, and asyncio. When the tasks are CPU
intensive, we should consider the multiprocessing module. When
the tasks are I/O bound and require lots of connections, the
asyncio module is recommended. For other types of tasks and when
libraries cannot cooperate with asyncio, the
threading module can be considered. The
concurrent.futures module provides a common high-level interface
to process and thread pools.
Embarrassingly parallel
The term embarrassingly parallel describes a problem or workload that can be easily run in parallel. Not all workloads can be divided into independent subtasks; tasks that need lots of communication among subtasks benefit much less.
The examples of perfectly parallel computations include:
- Monte Carlo analysis
- numerical integration
- rendering of computer graphics
- brute force searches in cryptography
- genetic algorithms
Another situation where parallel computations can be applied is when we run several different computations; for instance, we could compute π with different algorithms or precisions in parallel.
Process vs thread
Both processes and threads are independent sequences of execution. The following table summarizes the differences between a process and a thread.
| Process | Thread |
|---|---|
| runs in separate memory (process isolation) | shares memory with other threads |
| uses more memory | uses less memory |
| data must be pickled to pass between processes | data is shared directly |
| more overhead, slower to create and destroy | less overhead, faster to create and destroy |
| runs Python code in parallel | runs Python code in parallel only in the free-threaded build |
| no shared state by default, fewer race conditions | shared state needs locks, harder to debug |
Process
The Process class represents an activity that is run in a
separate process. It has equivalents of all the methods of
threading.Thread. The Process constructor should
always be called with keyword arguments.
The target argument of the constructor is the callable object to
be invoked by the run method, and args is a tuple of
its arguments. The name is the process name. The
start method starts the process's activity. The
join method blocks until the process terminates; with the
timeout option, it blocks at most the given number of seconds.
The is_alive method tells whether the process is running, and
terminate stops it.
Simple process example
The following is a simple program that uses multiprocessing.
from multiprocessing import Process
def fun(name):
print(f'hello {name}')
def main():
p = Process(target=fun, args=('Peter',))
p.start()
p.join()
if __name__ == '__main__':
main()
We create a new process and pass a value to it.
def fun(name):
print(f'hello {name}')
The function prints the passed parameter.
p = Process(target=fun, args=('Peter',))
p.start()
p.join()
A new process is created. The target option provides the
callable that is run in the new process. The args provides the
data to be passed; note the trailing comma, which makes it a one-element
tuple. The process is started with start and we wait for it
with join.
if __name__ == '__main__':
main()
The code that starts processes is placed inside the
__name__ == '__main__' guard. The next section explains why.
$ python simple.py hello Peter
Start methods and the __main__ guard
There are three ways of starting a process. With fork, the child
is a copy of the parent process. With spawn, a fresh interpreter
is started. With forkserver, a server process is started once and
new processes are forked from it. Since Python 3.14, the default start method
is forkserver on Linux and other POSIX systems except macOS, and
spawn on Windows and macOS. The fork method is no
longer the default because it is unsafe in programs that use threads.
With spawn and forkserver, the child process imports
the main module under the name '__mp_main__' to get the target
function. Code outside the __main__ guard runs again in the
child; without the guard, each child would start new processes.
import multiprocessing as mp
print(f'module loaded as {__name__}')
def fun():
print('running fun')
if __name__ == '__main__':
print(mp.get_start_method())
print(mp.get_all_start_methods())
p = mp.Process(target=fun)
p.start()
p.join()
The program prints the current start method and the methods available on the
platform. The message at the module level is printed twice: once in the main
process, and once in the child, where the module is loaded as
__mp_main__.
$ python start_methods.py module loaded as __main__ forkserver ['forkserver', 'fork', 'spawn'] module loaded as __mp_main__ running fun
Joining processes
The join method blocks the execution of the main process until
the process whose join method is called terminates.
from multiprocessing import Process
import time
def fun():
print('starting fun')
time.sleep(2)
print('finishing fun')
def main():
p = Process(target=fun)
p.start()
p.join()
if __name__ == '__main__':
print('starting main')
main()
print('finishing main')
The example calls join on the newly created process. The
finishing main message is printed after the child process has
finished.
$ python joining.py starting main starting fun finishing fun finishing main
Without join
Without the join method, the main process does not wait at the
point of the call.
from multiprocessing import Process
import time
def fun():
print('starting fun')
time.sleep(2)
print('finishing fun')
if __name__ == '__main__':
print('starting main')
p = Process(target=fun)
p.start()
print('finishing main')
The main process prints its last message before the child process starts working. The interpreter still waits for non-daemon child processes before it exits, so the child finishes its work.
$ python no_join.py starting main finishing main starting fun finishing fun
Join order
It is important to call the join methods after all
start methods. If we start and join each process in turn, the
processes run sequentially.
from multiprocessing import Process
import time
def fun(val):
print(f'starting fun with {val} s')
time.sleep(val)
print(f'finishing fun with {val} s')
def main():
start = time.perf_counter()
processes = [Process(target=fun, args=(val,)) for val in (3, 2, 1)]
for p in processes:
p.start()
for p in processes:
p.join()
print(f'elapsed: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
We start all three processes first and then wait for them. The processes run in parallel, so the program takes about three seconds, the duration of the longest process, instead of six.
$ python join_order.py starting fun with 3 s starting fun with 2 s starting fun with 1 s finishing fun with 1 s finishing fun with 2 s finishing fun with 3 s elapsed: 3.04 s
Process state
The is_alive method determines if the process is running. The
exitcode attribute is None while the process has not
finished; then it holds the exit code.
from multiprocessing import Process
import time
def fun():
time.sleep(1)
def main():
p = Process(target=fun)
print(f'before start: {p.is_alive()}, exitcode {p.exitcode}')
p.start()
print(f'after start: {p.is_alive()}, exitcode {p.exitcode}')
p.join()
print(f'after join: {p.is_alive()}, exitcode {p.exitcode}')
if __name__ == '__main__':
main()
We check the process before it is started, while it is running, and after it has finished. An exit code of 0 means success.
$ python alive.py before start: False, exitcode None after start: True, exitcode None after join: False, exitcode 0
Terminating processes
The join method accepts a timeout. The terminate
method stops a process; on POSIX systems it sends the SIGTERM
signal.
from multiprocessing import Process
import time
def endless():
while True:
time.sleep(0.1)
def main():
p = Process(target=endless)
p.start()
p.join(timeout=1)
print(f'alive after timeout: {p.is_alive()}')
p.terminate()
p.join()
print(f'alive after terminate: {p.is_alive()}')
print(f'exit code: {p.exitcode}')
if __name__ == '__main__':
main()
The child runs an endless loop. We wait for it at most one second; then we
terminate it. A negative exit code -N means that the process was
killed by signal N; SIGTERM is signal 15.
$ python terminating.py alive after timeout: True alive after terminate: False exit code: -15
Process ids
The os.getpid function returns the current process id, while
os.getppid returns the parent's process id.
from multiprocessing import Process
import os
def fun():
print('--------------------------')
print('parent process id:', os.getppid())
print('process id:', os.getpid())
def main():
print('main process id:', os.getpid())
for _ in range(2):
p = Process(target=fun)
p.start()
p.join()
if __name__ == '__main__':
main()
The example runs two child processes. Each of them prints its own id and the id of its parent.
$ python process_id.py main process id: 9296 -------------------------- parent process id: 9298 process id: 9299 -------------------------- parent process id: 9298 process id: 9300
Parent process with forkserver
In the previous output, the parent id of the children differs from the id of
the main process. With the default forkserver start method, the
children are forked from the fork server process, so it is their operating
system parent. The get_context function selects a start method
for a group of processes.
import multiprocessing as mp
import os
def fun():
print(f'parent: {os.getppid()}, child: {os.getpid()}')
def main():
print('main process id:', os.getpid())
for method in ('fork', 'forkserver'):
ctx = mp.get_context(method)
p = ctx.Process(target=fun)
p.start()
p.join()
if __name__ == '__main__':
main()
With fork, the parent of the child is the main process; with
forkserver, it is the server process. Within the
multiprocessing API, the main process is still available as
multiprocessing.parent_process.
$ python process_id_fork.py main process id: 9301 parent: 9301, child: 9302 parent: 9304, child: 9305
Naming processes
With the name parameter of the Process, we can give
the worker a specific name. Otherwise, the module creates a name such as
Process-3.
from multiprocessing import Process, current_process
import time
def worker():
name = current_process().name
print(name, 'starting')
time.sleep(1)
print(name, 'exiting')
def service():
name = current_process().name
print(name, 'starting')
time.sleep(2)
print(name, 'exiting')
if __name__ == '__main__':
processes = [
Process(name='Service 1', target=service),
Process(name='Worker 1', target=worker),
Process(target=worker),
]
for p in processes:
p.start()
for p in processes:
p.join()
In the example, we create three processes; two of them are given a custom
name. The current_process function returns the
Process object of the running process.
$ python naming_workers.py Service 1 starting Worker 1 starting Process-3 starting Worker 1 exiting Process-3 exiting Service 1 exiting
Subclassing Process
When we subclass the Process, we override the run
method. If we override the constructor, we must call the constructor of the
base class first.
import time
from multiprocessing import Process
class Worker(Process):
def __init__(self, delay):
super().__init__()
self.delay = delay
def run(self):
print(f'{self.name} sleeping for {self.delay} s')
time.sleep(self.delay)
print(f'{self.name} done')
def main():
workers = [Worker(2), Worker(1)]
for worker in workers:
worker.start()
for worker in workers:
worker.join()
if __name__ == '__main__':
main()
We create a Worker class which inherits from the
Process. The delay is passed in the constructor and the work is
done in the run method, which is called in the new process by
start.
$ python subclass.py Worker-1 sleeping for 2 s Worker-2 sleeping for 1 s Worker-2 done Worker-1 done
Process pool
The management of the worker processes can be simplified with the
Pool object. It controls a pool of worker processes to which
jobs can be submitted. The pool's map method chops the given
iterable into chunks, which it submits to the process pool as separate tasks.
It is a parallel equivalent of the built-in map function and
blocks until all results are ready.
The Pool takes the number of processes as a parameter. If we do
not provide any value, the number returned by
os.process_cpu_count is used; it is the number of CPUs the
current process may use.
import os
import time
from multiprocessing import Pool
def square(n):
time.sleep(2)
return n * n
def main():
start = time.perf_counter()
print(f'starting computations on {os.process_cpu_count()} cores')
values = (2, 4, 6, 8)
with Pool() as pool:
res = pool.map(square, values)
print(res)
print(f'elapsed time: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
In the example, we create a pool of processes and apply the
square function to four values. The with statement
terminates the pool when the block ends.
$ python worker_pool.py starting computations on 8 cores [4, 16, 36, 64] elapsed time: 2.07 s
More tasks than workers
When there are more tasks than worker processes, the remaining tasks wait until a worker is free.
import time
from multiprocessing import Pool
def square(n):
time.sleep(2)
return n * n
def main():
start = time.perf_counter()
values = (2, 4, 6, 8, 10)
with Pool(4) as pool:
res = pool.map(square, values)
print(res)
print(f'elapsed time: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
We limit the pool to four processes and submit five tasks. The fifth task starts after one of the first four has finished, so the computation takes about four seconds.
$ python small_pool.py [4, 16, 36, 64, 100] elapsed time: 4.06 s
Multiple arguments
To pass multiple arguments to a worker function, we can use the
starmap method. The elements of the iterable are expected to be
iterables that are unpacked as arguments.
import time
from multiprocessing import Pool
def power(x, n):
time.sleep(1)
return x ** n
def main():
start = time.perf_counter()
values = ((2, 2), (4, 3), (5, 5))
with Pool() as pool:
res = pool.starmap(power, values)
print(res)
print(f'elapsed time: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
In this example, we pass two values to the power function: the
value and the exponent.
$ python multi_args.py [4, 64, 3125] elapsed time: 1.07 s
Multiple functions
The following example shows how to run multiple functions in a pool.
from multiprocessing import Pool
import functools
def inc(x):
return x + 1
def dec(x):
return x - 1
def add(x, y):
return x + y
def smap(f):
return f()
def main():
f_inc = functools.partial(inc, 4)
f_dec = functools.partial(dec, 2)
f_add = functools.partial(add, 3, 4)
with Pool() as pool:
res = pool.map(smap, [f_inc, f_dec, f_add])
print(res)
if __name__ == '__main__':
main()
We have three functions, which are run independently in a pool. We use
functools.partial to bind the functions to their arguments
before they are executed. The smap helper then simply calls each
prepared function.
$ python multiple_functions.py [5, 1, 7]
Picklable functions
The function and its arguments are sent to the worker processes with the
pickle module, and so are the results. Functions are pickled by
reference to their module and name, so lambdas and nested functions cannot be
used with a pool.
from multiprocessing import Pool
def main():
with Pool(2) as pool:
try:
print(pool.map(lambda x: x * 2, [1, 2, 3]))
except Exception as e:
print(f'{type(e).__name__}: {e}')
if __name__ == '__main__':
main()
The lambda cannot be pickled and the map call fails. The worker
function must be defined at the top level of a module.
$ python lambda_error.py PicklingError: Can't pickle local object <function main.<locals>.<lambda> at 0x7118b06d7ed0>
Results in completion order
The imap method returns results lazily in the order of the
input, while imap_unordered returns them as soon as they are
ready.
import random
import time
from multiprocessing import Pool
def task(n):
time.sleep(random.uniform(0.1, 1))
return n
def main():
with Pool(4) as pool:
for n in pool.imap(task, range(8)):
print(f'imap: {n}')
for n in pool.imap_unordered(task, range(8)):
print(f'imap_unordered: {n}')
if __name__ == '__main__':
main()
Each task sleeps for a random time. The imap results come in
the input order; the imap_unordered results come in the order in
which the tasks finished.
$ python unordered.py imap: 0 imap: 1 imap: 2 imap: 3 imap: 4 imap: 5 imap: 6 imap: 7 imap_unordered: 2 imap_unordered: 0 imap_unordered: 1 imap_unordered: 3 imap_unordered: 5 imap_unordered: 4 imap_unordered: 6 imap_unordered: 7
ProcessPoolExecutor
The concurrent.futures.ProcessPoolExecutor class is a
higher-level alternative to Pool. The submit method
schedules a call and returns a Future object, and the
as_completed function yields the futures as they finish.
import time
from concurrent.futures import ProcessPoolExecutor, as_completed
def count_primes(limit):
count = 0
for n in range(2, limit):
if all(n % d for d in range(2, int(n ** 0.5) + 1)):
count += 1
return limit, count
def main():
limits = [200_000, 50_000, 100_000, 150_000]
start = time.perf_counter()
with ProcessPoolExecutor() as executor:
futures = [executor.submit(count_primes, limit) for limit in limits]
for future in as_completed(futures):
limit, count = future.result()
print(f'{count:6} primes below {limit:,}')
print(f'elapsed time: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
We count the primes below four limits in parallel. The results are printed
as the tasks complete, so the smallest limit comes first. The same code works
with a ThreadPoolExecutor, which makes it easy to switch between
threads and processes.
$ python executor.py 5133 primes below 50,000 9592 primes below 100,000 13848 primes below 150,000 17984 primes below 200,000 elapsed time: 0.31 s
Computing π sequentially
The π is the ratio of the circumference of any circle to its diameter. It is an irrational number whose decimal form neither ends nor becomes repetitive; it is approximately equal to 3.14159. There are several formulas to calculate π. Computing many digits of π takes a long time, so we can leverage parallel computations. We use the Bailey–Borwein–Plouffe formula.
import time
from decimal import Decimal, getcontext
def pi(precision):
getcontext().prec = precision
return sum(1 / Decimal(16) ** k *
(Decimal(4) / (8 * k + 1) -
Decimal(2) / (8 * k + 4) -
Decimal(1) / (8 * k + 5) -
Decimal(1) / (8 * k + 6)) for k in range(precision))
def main():
start = time.perf_counter()
values = (3000, 3500, 4000)
data = list(map(pi, values))
for val in data:
print(str(val)[:20])
print(f'sequentially: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
First, we calculate three approximations sequentially. The precision is the number of significant digits of the computed π; we print only the first 20 characters of each result.
$ python calc_pi.py 3.141592653589793238 3.141592653589793238 3.141592653589793238 sequentially: 4.15 s
Computing π in parallel
In the following example, we use a pool of processes to calculate the same three approximations.
import time
from decimal import Decimal, getcontext
from multiprocessing import Pool
def pi(precision):
getcontext().prec = precision
return sum(1 / Decimal(16) ** k *
(Decimal(4) / (8 * k + 1) -
Decimal(2) / (8 * k + 4) -
Decimal(1) / (8 * k + 5) -
Decimal(1) / (8 * k + 6)) for k in range(precision))
def main():
start = time.perf_counter()
values = (3000, 3500, 4000)
with Pool(3) as pool:
data = pool.map(pi, values)
for val in data:
print(str(val)[:20])
print(f'in parallel: {time.perf_counter() - start:.2f} s')
if __name__ == '__main__':
main()
We run the calculations in a pool of three processes. The total time is now determined by the slowest of the three computations, the one with precision 4000, and the overhead of starting the processes.
$ python calc_pi2.py 3.141592653589793238 3.141592653589793238 3.141592653589793238 in parallel: 2.12 s
Separate memory in a process
In multiprocessing, each worker has its own memory. The memory is not shared like in threading.
from multiprocessing import Process, current_process
data = [1, 2]
def fun():
data.extend((3, 4, 5))
print(f'Result in {current_process().name}: {data}')
def main():
worker = Process(target=fun)
worker.start()
worker.join()
print(f'Result in main: {data}')
if __name__ == '__main__':
main()
The worker extends the global data list, but the list in the
main process is not modified. The child works with its own copy of the
module's globals.
$ python own_memory_space.py Result in Process-1: [1, 2, 3, 4, 5] Result in main: [1, 2]
Sharing state between processes
Data can be stored in shared memory using Value or
Array. A shared value created with the default
lock=True has a lock, which must be held to update it
safely.
from multiprocessing import Process, Value
def increment(counter, n):
for _ in range(n):
with counter.get_lock():
counter.value += 1
def increment_unsafe(counter, n):
for _ in range(n):
counter.value += 1
def run(target):
counter = Value('i', 0)
processes = [Process(target=target, args=(counter, 10_000)) for _ in range(4)]
for p in processes:
p.start()
for p in processes:
p.join()
return counter.value
if __name__ == '__main__':
print(f'with lock: {run(increment)}')
print(f'without lock: {run(increment_unsafe)}')
Four processes increment a shared integer counter 10,000 times each. The
'i' type code denotes a C int. The += operation is
a read followed by a write; without the lock, increments of different
processes overwrite each other and get lost.
$ python counter.py with lock: 40000 without lock: 14899
Message passing with queues
Message passing is the preferred way of communication among processes. It
avoids synchronization primitives such as locks, which are difficult to use
and error prone in complex situations. A Queue allows multiple
producers and consumers; a Pipe connects two processes.
from multiprocessing import Process, Queue
import random
def rand_val(queue):
num = random.random()
queue.put(num)
def main():
queue = Queue()
processes = [Process(target=rand_val, args=(queue,)) for _ in range(4)]
for p in processes:
p.start()
results = [queue.get() for _ in processes]
for p in processes:
p.join()
print(results)
if __name__ == '__main__':
main()
In the example, we create four processes. Each process generates a random
value and puts it into the queue. The get method removes and
returns an item from the queue, blocking until one is available. We read the
results before joining the processes. A process that has put items into a
queue does not terminate until all of them have been flushed to the
underlying pipe, so joining first can deadlock when the data does not fit
into the pipe buffer.
$ python simple_queue.py [0.9458185238309272, 0.7245706723075374, 0.7594288890168714, 0.4721883931975338]
Producers and consumers
In the following example, worker processes take tasks from one queue and put results into another.
import time
from multiprocessing import Process, Queue, current_process
def worker(tasks, results):
name = current_process().name
for word in iter(tasks.get, None):
time.sleep(0.1)
results.put(f'{name}: {word} -> {word.upper()}')
def main():
tasks = Queue()
results = Queue()
words = ['wood', 'sky', 'cloud', 'ocean', 'forest', 'rock']
for word in words:
tasks.put(word)
workers = [Process(target=worker, args=(tasks, results)) for _ in range(3)]
for w in workers:
tasks.put(None)
w.start()
for _ in words:
print(results.get())
for w in workers:
w.join()
if __name__ == '__main__':
main()
We put six words into the tasks queue, followed by one None
sentinel per worker. The sleep call simulates work. The
iter(tasks.get, None) call creates an
iterator that calls tasks.get until it returns the sentinel. The
main process reads one result for each word.
$ python producer_consumer.py Process-1: wood -> WOOD Process-2: sky -> SKY Process-3: cloud -> CLOUD Process-1: ocean -> OCEAN Process-2: forest -> FOREST Process-3: rock -> ROCK
Pipes
The Pipe function returns a pair of connection objects. Data
sent with send at one end is received with recv at
the other end.
from multiprocessing import Process, Pipe
def worker(conn):
while (msg := conn.recv()) is not None:
conn.send(sum(msg))
conn.close()
def main():
parent_conn, child_conn = Pipe()
p = Process(target=worker, args=(child_conn,))
p.start()
for data in ([1, 2, 3], [10, 20], [7] * 7):
parent_conn.send(data)
print(f'sum of {data}: {parent_conn.recv()}')
parent_conn.send(None)
p.join()
if __name__ == '__main__':
main()
The worker receives lists of numbers and sends back their sums until it gets
None. The walrus operator assigns the received message and tests
it in the loop condition.
$ python pipes.py sum of [1, 2, 3]: 6 sum of [10, 20]: 30 sum of [7, 7, 7, 7, 7, 7, 7]: 49
Queue order
In multiprocessing, there is no guarantee that the processes finish in a certain order. If the order of the results matters, we must restore it.
from multiprocessing import Process, Queue
import time
import random
def square(idx, x, queue):
time.sleep(random.uniform(0.1, 1))
queue.put((idx, x * x))
def main():
data = [2, 4, 6, 3, 5, 8, 9, 7]
queue = Queue()
processes = [Process(target=square, args=(idx, val, queue))
for idx, val in enumerate(data)]
for p in processes:
p.start()
unsorted_result = [queue.get() for _ in processes]
for p in processes:
p.join()
print(unsorted_result)
result = [val for _, val in sorted(unsorted_result)]
print(result)
if __name__ == '__main__':
main()
The processes calculate the squares of the input values. Each process sleeps for a random time and then puts a tuple of the input index and the square into the queue. The results arrive in random order; we sort them by the index to get the squares in the order of the input data.
$ python queue_order.py [(2, 36), (6, 81), (5, 64), (4, 25), (7, 49), (1, 16), (0, 4), (3, 9)] [4, 16, 36, 9, 25, 64, 81, 49]
Monte Carlo π sequentially
Monte Carlo methods are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. We generate random points in a unit square and count those that fall inside the quarter circle of radius 1. The following formula gives the approximation of π:
The M is the number of generated points inside the quarter circle and N is the total number of points. This method is a nice school example, but it is not very accurate; there are far better algorithms to get π.
import random
import time
def pi(n):
count = 0
for _ in range(n):
x, y = random.random(), random.random()
if x * x + y * y < 1:
count += 1
return 4 * count / n
start = time.perf_counter()
pi_est = pi(50_000_000)
print(f'elapsed time: {time.perf_counter() - start:.2f} s')
print(f'π estimate: {pi_est}')
We calculate the approximation of π from fifty million random points in a
single process. Comparing x * x + y * y with 1 avoids the square
root.
$ python monte_carlo_pi.py elapsed time: 4.64 s π estimate: 3.14142848
Monte Carlo π in parallel
Now we divide the whole task of π computation into subtasks.
import os
import random
import time
from multiprocessing import Pool
def pi_part(n):
count = 0
for _ in range(n):
x, y = random.random(), random.random()
if x * x + y * y < 1:
count += 1
return count
def main():
start = time.perf_counter()
np = os.process_cpu_count()
print(f'You have {np} cores')
n = 50_000_000
part_count = [n // np] * np
with Pool(processes=np) as pool:
count = pool.map(pi_part, part_count)
pi_est = 4 * sum(count) / sum(part_count)
print(f'elapsed time: {time.perf_counter() - start:.2f} s')
print(f'π estimate: {pi_est}')
if __name__ == '__main__':
main()
We find out the number of available cores and divide the random sampling into
equal parts. Each worker counts the hits in its part independently; the
random module reseeds itself in forked child processes, so the
workers generate different points. The partial counts are summed and used in
the final formula.
$ python monte_carlo_pi_mul.py You have 8 cores elapsed time: 1.55 s π estimate: 3.14147608
Source
multiprocessing – Process-based parallelism
concurrent.futures – Launching parallel tasks
In this article we have worked with the multiprocessing module.
See also
Python ThreadPoolExecutor,
Python asyncio,
Python subprocess, and
Python pickle.
Author
List all Python tutorials.