Cancel a Task and Clean Up After It

Your server is shutting down while a worker task is halfway through a slow download. The worker holds a database connection. You want it to stop, give the connection back, and let the server exit. In asyncio you ask a task to stop with task.cancel():

import asyncio


async def worker() -> None:
    print("worker: started")
    try:
        await asyncio.sleep(10)  # stands in for a slow download
    finally:
        print("worker: cleaning up")


async def main() -> None:
    task = asyncio.create_task(worker())
    await asyncio.sleep(0.1)
    task.cancel()
    try:
        await task
    except asyncio.CancelledError:
        print("main: worker was cancelled")


asyncio.run(main())

worker: started
worker: cleaning up
main: worker was cancelled

Cancel asks, and the task stops at an await

Cancellation means asking a task to stop. task.cancel() only marks the task. The next time the task pauses at an await, asyncio raises asyncio.CancelledError there. The error propagates like any other exception, so finally blocks and with blocks run before the task ends. Code that awaits the task then gets CancelledError too.

CancelledError inherits from BaseException directly, so an ordinary except Exception: handler does not catch it. That is what you want.

Find out why a task keeps running after cancel()

When cancel() seems to do nothing, the cause is almost always one of two things.

The first is a task stuck in blocking code. A loop doing heavy computing, or a blocking call such as time.sleep() or requests.get(), blocks the event loop until it finishes. In that time the task reaches no await, so asyncio has nowhere to raise CancelledError. Use an async version of the call, or run the blocking call in a thread, which a later article shows.

The second is a task that catches the error and keeps running:

async def worker() -> None:
    while True:
        try:
            await fetch_next()
        except BaseException:  # this also catches CancelledError
            log.exception("fetch failed")

Each cancel() raises CancelledError inside fetch_next(). The handler logs it, and the loop calls fetch_next() again. Any code that waits for this task waits forever. Catch Exception instead. If you catch CancelledError to clean up, end the handler with a bare raise.

Cleanup can await

A cancelled task can still await in its finally block. That is how you close a connection or flush a buffer:

try:
    await download(conn)
finally:
    await conn.close()

The cleanup itself can hang, for example when the server stops answering. Give it a time limit with asyncio.timeout:

try:
    async with asyncio.timeout(2):
        await conn.close()
except TimeoutError:
    log.warning("closing the connection timed out")

When the time is up, asyncio.timeout cancels the code inside the block. At the end of the block, it turns the resulting CancelledError into a TimeoutError, so you catch TimeoutError outside the async with. The next article covers timeouts in detail.

A task cancelled before it starts running skips its whole body, finally included. If you open a resource and then pass it to a new task, close it in the code that opened it.

Check your understanding A handler wraps a database call in async with asyncio.timeout(1):. Inside the block it catches asyncio.CancelledError and returns a default value. What happens when the call takes 5 seconds? Show answer

After one second the timer cancels the call. The handler catches the CancelledError and returns the default, so the block ends normally without raising TimeoutError. To the caller, the call looks like it succeeded. If the task was also being cancelled for shutdown, that request is lost too. Catch TimeoutError outside the block instead, and let CancelledError through.

Shield work that must finish

Sometimes a step, such as writing a payment record, must finish even if the caller gives up. asyncio.shield() lets the caller be cancelled while the shielded task keeps running:

save = asyncio.create_task(save_payment(order))
await asyncio.shield(save)

If the caller is cancelled, it gets CancelledError straight away, and save keeps running. Keep a reference to save and check its result later, as the article on background tasks showed. Shield protects the task only from its caller's cancellation. Code that holds save can still cancel it directly.

Going further

task.cancelling() counts the stop requests that are still pending, and task.uncancel() takes one back. asyncio.timeout and TaskGroup use them to tell their own cancellation apart from yours. Don't call them in your own code.

See the asyncio task documentation for the full cancellation, shielding and timeout API.