Skip to content

← Everyday Standard Library Tools step 47 of 55

Medium Primitives

Find Large Files Without Entering Skipped Folders

Write a top-down walk over a folder tree stored in a dictionary, then use it to find large files while skipping folders such as node_modules. It works like Path.walk(). The caller can skip or reorder subfolders by changing the list the walk yields. Nothing here touches the disk.

tree = {
    "/srv": {"dirs": ["app", "cache"], "files": {"a.txt": 10}},
    "/srv/app": {"dirs": [], "files": {"main.py": 2400}},
    "/srv/cache": {"dirs": [], "files": {"big.bin": 9000}},
}
find_large_files(tree, "/srv", 100, skip={"cache"})  # ["/srv/app/main.py"]

Each value in the tree is a Node. Node is a TypedDict, a dict type that lists its keys and their types. Its dirs key holds the subfolder names, and files maps each file name to its size in bytes. The tree's keys are absolute folder paths with no trailing slash, except / itself. There are no symlinks or loops.

What to write

child(parent, name) is already in the starter. It joins with one slash, so child("/", "app") is "/app" and child("/srv", "app") is "/srv/app".

Write walk(tree, top). It yields (dirpath, dirnames, filenames) for each folder it visits:

  • If top is not in tree, yield nothing. The same goes for a listed subfolder that is missing from tree.
  • Copy the node's folder names and file names into new lists. Yield top with those copies before you visit any subfolder.
  • When the caller asks for the next item, visit each name that is still in that same dirnames list, in its order, depth-first. If the caller cleared the list, go no deeper. If the caller reversed it, visit the subfolders in reverse order.
  • filenames starts in the order of the node's files keys.
  • Never change tree.

Fix find_large_files(tree, root, min_bytes, *, skip):

  • For every folder that walk yields, remove each name in skip from dirnames in place. The starter checks file names against skip instead. Move that check to the folder list.
  • skip matches folder names at any depth. It never matches file names or full paths, and it never skips root itself.
  • Collect every file whose size is at least min_bytes.
  • Return the files' full paths, largest file first. Break ties by path, in ascending string order.

Skip folders while you walk, not by filtering the final list, even though filtering could give the same paths.

solve is already in the starter. It turns its skip list into a set and calls find_large_files, so a repeated name has no extra effect.

Edge cases

  • A file whose size equals min_bytes counts.
  • A negative min_bytes includes every file in the folders you visit.
  • An empty tree, a missing root, or a threshold above every file size gives [].
  • With skip={"cache"}, a file named cache in a visited folder still counts.