Skip to content
← All Python tracks

Parallel Python with Process Pools

Python

Spread CPU-heavy work across cores with a process pool. Learn what you can send to a worker and how workers start. Share large data safely, and debug workers that hang or crash.

This track uses Python. You can make it your practice mode to browse related content across the site.

17 exercises

7 articles

Explore freely. Sign in to keep your progress.

Start track

Your learning sequence

  1. 1 Speed Up CPU-Heavy Work with a Process Pool Read
  2. 2 Not solved yet. Will It Pickle? Check What a Process Pool Can Send Practice · Easy
  3. 3 Not solved yet. Find Every Unpicklable Value in a Nested Payload Practice · Medium
  4. 4 How Python Starts Worker Processes Read
  5. 5 Not solved yet. Give Each Worker Its Own Database Connection Practice · Hard
  6. 6 Write a Library That Leaves Pools and Start Methods to the Caller Read
  7. 7 Not solved yet. Audit a Library for Import-Time Side Effects Practice · Hard
  8. 8 Not solved yet. Type-Check the Arguments You Send to a Worker Practice · Medium
  9. 9 Not solved yet. Split Work into Chunks for a Process Pool Practice · Easy
  10. 10 Batch Small Jobs in Pool and ProcessPoolExecutor Read
  11. 11 Not solved yet. Size a Process Pool to a Container's CPU Limit Practice · Medium
  12. 12 Not solved yet. Collect Every Result Before You Join the Workers Practice · Hard
  13. 13 Not solved yet. Lock and Batch Updates to a Shared Counter Practice · Medium
  14. 14 Not solved yet. Share Memory Between Processes and Clean It Up Practice · Hard
  15. 15 Not solved yet. Pickle a Large Buffer Without Copying It Practice · Hard
  16. 16 Not solved yet. Choose a Start Method for Each Platform Practice · Medium
  17. 17 Not solved yet. Plan Worker Recycling for a Process Pool Practice · Medium
  18. 18 Not solved yet. Fix Multiprocessing Type Hints That Fail at Runtime Practice · Medium
  19. 19 Debug Worker Processes That Hang or Crash Read
  20. 20 Not solved yet. Simulate a Pool Shutdown with a Grace Period Practice · Hard
  21. 21 Not solved yet. Run Code in a Subinterpreter and Check the Result Practice · Medium
  22. 22 What Subinterpreters Cost and When They Help Read
  23. 23 Free-Threaded Python in Plain Words Read
  24. 24 Not solved yet. Build a Thread-Safe Cache with One Lock per Key Practice · Hard

Check yourself

5 questions · not saved while you're signed out

Use these questions to check your understanding and compare explanations. Your score does not affect track completion.

0 / 5 correct

Why does reading the results from this process pool fail?

Choose one answer to reveal the explanations. Answers are kept only while this page is open. 

from concurrent.futures import ProcessPoolExecutor
from multiprocessing import get_context

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=1, mp_context=get_context("spawn")) as pool:
        results = pool.map(lambda x: x * 2, [1, 2])
        print(list(results))
Question 1 of 5