In python a code file can do arbitrary stuff. They can connect to a server, calculate something complicated or have enormous literals to parse.

When projects get big, this is a problem because the import time compounds and most people and code dislike imports inside function or class scopes.

lazy imports appeared in python 3.15 and allows deferring the actual import while keeping them at the top of the file.

In both cases deferring the import introduces arbitrary waits that are not predictable. They also can affect the state of the program by patching, forking or performing any other side effect operation that depends on order.

Sometimes it is better to measure the import time and make the changes so the expensive code runs in the appropriate context.

The post 3.8 way

Python now includes import time in its standard library.

https://stackoverflow.com/questions/34755728/profile-python-import-times

This method requires no code changes and tuna allows to see the visualise the output without.

The custom or pre 3.7 way

Back in the day with python 3.6 patching the easiest way to get the information was to patch the import system

from dataclasses import dataclass
import inspect
import sys
import importlib.machinery
import time

@dataclass
class ImportDuration:
    name:str
    parent: str
    duration: float

    @property
    def calculated_duration(self):
        return max(0.0, self.duration - sum(x.duration for x in self.children))

    @property
    def children(self):
        return [node for node in IMPORT_DURATIONS.values() if node.parent == self.name]

IMPORT_DURATIONS = {}

def _genrate_loader_details():
    """Returns a list of file-based module loaders.
    """
    extensions = importlib.machinery.ExtensionFileLoader, importlib.machinery.EXTENSION_SUFFIXES
    source = MyLoader, importlib.machinery.SOURCE_SUFFIXES
    bytecode = importlib.machinery.SourcelessFileLoader, importlib.machinery.BYTECODE_SUFFIXES
    return [extensions, source, bytecode]

class MyLoader(importlib.machinery.SourceFileLoader):
    def exec_module(self, module):
        start = time.time()
        result = super().exec_module(module)
        duration = time.time()-start
        module_file = module.__file__
        if module_file not in IMPORT_DURATIONS:
            stack = inspect.stack()[4]
            parent_filename = stack.filename
            IMPORT_DURATIONS[module_file] = ImportDuration(module_file, parent_filename, duration)
        return result

sys.path_hooks[1] = importlib.machinery.FileFinder.path_hook(*_genrate_loader_details())


if __name__ == "__main__":
    import cogapp # any import that we want to measure
    from pprint import pprint
    pprint(sorted([(x.name, x.calculated_duration) for x
                   in IMPORT_DURATIONS.values()],
                  key=lambda y: y[1]))

The main idea is:

  1. Reimplement SourceFileLoader by adding the timing code around the actual loading. docs
  2. Tell the FileFinder to use the new loader for loading python files using .path_hook docs
  3. Run the actual code
  4. Inspect the data structure where the timing information is stored

I don’t remember where I got this code nor how much did I have to fine tune it, but I remember getting the signatures like the loader_details being hard.

This method has the advantage of being fully customisable so for example it can run for imports after forking, or raise an exception if an import goes above certain duration.