Skip to content

← Parallel Python with Process Pools step 2 of 24

Easy Primitives

Will It Pickle? Check What a Process Pool Can Send

Before you hand work to a process pool, check that Python can send it. Python sends every argument to a worker process by pickling it, which turns the object into bytes. Some objects cannot be pickled. Write a function that tells you which ones.

can_pickle({"user": "ada"})  # True
can_pickle(lambda n: n + 1)  # False

What to write

Fill in can_pickle(obj):

  • Call pickle.dumps(obj). If it returns, return True.
  • If it raises pickle.PicklingError, TypeError or AttributeError, return False. Which one you get depends on the object and on the Python version. A function defined inside another function raises AttributeError on Python 3.12 and 3.13 and PicklingError from 3.14, so catch all three.

The samples

SAMPLES maps a name to a function that builds one sample object. Keep it and solve as they are. solve(names=...) returns a dictionary that maps each name you pass to the result of can_pickle on its sample.

Name Sample
int 42
dict {"user": "ada", "scores": [90, 85]}
date datetime.date(2026, 10, 20)
path pathlib.PurePosixPath("data/rows.csv")
math.sqrt the function math.sqrt
lambda lambda n: n + 1
nested function a function defined inside another function
lock threading.Lock()
generator (n * n for n in range(3))
list with a lock [1, 2, threading.Lock()]

What the results show

  • Plain data pickles, including numbers, strings, lists, dicts, dates and paths.
  • A function pickles when a worker can import it by name, as with math.sqrt. A lambda, or a function defined inside another function, has no name a worker can import.
  • A lock and a generator hold state that only makes sense inside one process, so they do not pickle.
  • A container pickles only if everything inside it pickles. A list that holds a lock cannot be pickled.