Article

Lazy Loading Heavy GIS Imports in CLI Subcommands

Import the geospatial stack inside command bodies rather than at module scope, and register sub-apps through a callable that resolves on first use. --help, --version and shell completion then cost Typer’s own start-up rather than a full GDAL initialisation, which is the difference between a tool that feels instant and one that does not. It builds on the CLI Subcommand Organization guide, part of the broader CLI Architecture & Design Patterns reference.

Prerequisites

  • Python 3.10 or later with pip install typer rasterio geopandas pyproj
  • A multi-group CLI, per the parent guide

Where the Start-up Time Goes

What each import adds before anything runs Typer and Click together start in a few tens of milliseconds. pyproj opens the PROJ database. rasterio registers every GDAL driver. geopandas pulls in pandas and shapely on top. The total is what --help pays when the imports sit at module scope. typer and click argument parsing only tens of ms + pyproj opens the PROJ database on first use adds noticeably + rasterio registers every GDAL driver the largest single step + geopandas pandas and shapely underneath it adds again

None of that is wasted when the command needs it. All of it is wasted for --help, for a completion request, and for the --version a packaging script calls in a loop.

Four invocations exercise different amounts of the import graph, and only one of them needs the geospatial stack at all.

What each invocation actually needs Printing help needs the parser and the command metadata. Printing the version needs package metadata alone. A completion request needs the parser and whatever the completion callback consults. Only a real command needs the geospatial libraries. gis --help the parser and the registered command metadata typer only gis --version package metadata, nothing else typer only a completion request the parser, plus the completion callback's data typer, maybe config gis raster warp ... the full geospatial stack for that group everything

Three of the four are on a latency budget a user notices. The fourth is doing minutes of work and will not miss a second of imports.

Complete Working Implementation

# gistools/cli.py — the root, deliberately import-light
from __future__ import annotations

import importlib
from typing import Callable

import typer

app = typer.Typer(no_args_is_help=True, add_completion=True)

# (module path, attribute, command-group name, help text)
GROUPS: list[tuple[str, str, str, str]] = [
    ("gistools.commands.raster", "app", "raster", "Raster operations."),
    ("gistools.commands.vector", "app", "vector", "Vector operations."),
    ("gistools.commands.crs", "app", "crs", "Coordinate reference tools."),
]


def _lazy_group(module_path: str, attr: str) -> Callable[[], typer.Typer]:
    def load() -> typer.Typer:
        return getattr(importlib.import_module(module_path), attr)
    return load


for module_path, attr, name, help_text in GROUPS:
    # Typer needs a Typer instance at registration time, so register a stub
    # whose callback performs the real import and re-dispatches.
    stub = typer.Typer(help=help_text, no_args_is_help=True)

    def _make(loader=_lazy_group(module_path, attr), group=name):
        @stub.callback(invoke_without_command=True)
        def _entry(ctx: typer.Context) -> None:
            real = loader()
            if ctx.invoked_subcommand is None:
                typer.echo(ctx.get_help())
                raise typer.Exit()
            # Hand the remaining arguments to the real group.
            command = typer.main.get_command(real)
            command.main(args=ctx.args, prog_name=f"gis {group}",
                         standalone_mode=True)
    _make()
    app.add_typer(stub, name=name)

And a group module that keeps its own imports inside the bodies:

# gistools/commands/raster.py
from pathlib import Path

import typer

app = typer.Typer(no_args_is_help=True)


@app.command()
def warp(src: Path, dst: Path, dst_crs: str = typer.Option(..., "--dst-crs")) -> None:
    """Reproject SRC into DST."""
    import rasterio                       # noqa: PLC0415 — deferred on purpose
    from rasterio.warp import reproject

    with rasterio.open(src) as source:
        ...

Step Annotations

  1. The stub carries the help text. A lazily-loaded group whose help is only available after the import would defeat the purpose — gis --help would load everything.
  2. invoke_without_command=True on the stub callback. It lets gis raster with no subcommand print help rather than erroring, without importing the real group.
  3. The noqa comment names the reason. A deferred import looks like a mistake to every linter and to the next reader; the comment is the difference between a decision and an accident.
  4. standalone_mode=True on re-dispatch. It keeps Click’s own exit-code handling, so the deferred path returns the same codes as a direct one.
  5. Groups are data, not code. Adding a group is one tuple, which is what stops the root file growing imports again over time.

One Named Gotcha: Deferred Imports Hide Broken Installs Until Run Time

Moving imports into command bodies means a missing or broken GDAL is not discovered at start-up. The tool starts, prints help, accepts arguments, and fails ten seconds later inside the command — which is a worse experience than failing immediately, unless the failure is handled.

The answer is a doctor command and a friendly wrapper, not a return to eager imports:

@app.command()
def doctor() -> None:
    """Check that the geospatial stack is importable and correctly configured."""
    problems = []
    for name in ("rasterio", "pyproj", "pyogrio"):
        try:
            importlib.import_module(name)
        except Exception as exc:                     # noqa: BLE001
            problems.append(f"{name}: {type(exc).__name__}: {exc}")
    if problems:
        for line in problems:
            typer.echo(line, err=True)
        raise typer.Exit(code=3)
    typer.echo("geospatial stack OK")

Combined with the error taxonomy in Handling Missing Dependencies Gracefully in Click Apps, a deferred import that fails produces a message naming the package and the fix rather than a traceback from inside the driver registry.

Verification

# Help should not import the geospatial stack at all.
python -X importtime -m gistools --help 2>&1 | grep -cE 'rasterio|geopandas|pyproj'   # expect 0

# And should be fast.
hyperfine --warmup 3 'gis --help' 'gis raster warp --help' 'gis --version'

# The real command still works.
gis raster warp in.tif out.tif --dst-crs EPSG:3857

The first check is the load-bearing one. A count above zero means something in the import path still pulls the stack in — usually a type annotation evaluated at module scope, which from __future__ import annotations defers along with everything else.

Deferring imports moves a class of failure later, so it is worth being explicit about which failures move and what to do about each.

What moves later, and how to cover it A missing package now fails inside the command rather than at start-up. A broken shared library does the same. A misconfigured PROJ database was always a run-time failure and is unchanged. A doctor command and a friendly handler cover the first two. a missing package was: crash at start-up now: fails in the command cover with a doctor command and a handler mitigated a broken shared library was: crash at start-up now: fails in the command same mitigation, and the message names the .so mitigated a missing PROJ database was: fails at first transform now: unchanged nothing moved — it was always a run-time failure unchanged

Only the first two change behaviour, and a single doctor subcommand covers both without giving up the start-up time.

Measuring Before and After

Optimising start-up without measuring it produces changes that feel faster and are not. Python’s own -X importtime flag reports the cost of every import in the tree, and it is the right tool because it attributes time to the module rather than to the call site.

python -X importtime -m gistools --help 2>&1 | sort -t'|' -k2 -rn | head -20

The output is cumulative and self-time per module, sorted here by self-time. On a typical geospatial CLI before this change, the top of that list is rasterio._base, pyproj.database, pandas.core and shapely.geometry — none of which --help needs. After the change, the top of the list is click.core and typer.main, which is what a parser costs and cannot be avoided.

Two other measurements are worth taking alongside it. The first is the wall clock for the three fast paths, using hyperfine or a shell loop, because importtime reports import cost and not process start-up as a whole. The second is the completion latency specifically: a shell completion request runs your program, and anything above roughly a hundred milliseconds is perceptible as a stall while typing.

What Not to Defer

Deferring is not free, and three categories of import are better left at module scope.

Anything used by the parser itself has to be imported before parsing, so deferring it accomplishes nothing and adds an import statement inside every function that touches it. That includes the types in option annotations — although from __future__ import annotations makes annotations strings, so a type used only in an annotation is already effectively deferred.

Anything cheap is not worth the indirection. pathlib, dataclasses, json and the rest of the standard library cost microseconds, and moving them inside functions makes the code harder to read for no measurable gain. The rule of thumb is to defer imports that cost more than about ten milliseconds, which in practice means the geospatial stack, pandas, and anything that opens a database at import time.

Anything whose absence should stop the program deserves thought rather than a rule. A tool whose entire purpose is raster processing might reasonably import rasterio eagerly, accept the start-up cost, and give users an immediate, clear failure on a broken install. A tool with several groups, only one of which needs rasterio, should not — because the vector user pays for a dependency they never touch.

The middle position is what the doctor command above provides: defer everything, and give users an explicit, fast way to check the whole stack when they want to. That keeps the fast paths fast and keeps the diagnosis available, without making every invocation pay for a check that almost always passes.

A Note on Type Annotations

Annotations are the one place a deferred import leaks back into module scope without anyone intending it. A signature written as def warp(src: rasterio.DatasetReader) -> None evaluates rasterio at definition time, which happens at import, which puts the whole stack back on the fast path.

from __future__ import annotations turns every annotation into a string and removes the problem entirely. It is worth adding to every module in the package rather than only the ones that currently need it, because the next person to add a type will not remember the rule. Where a runtime-evaluated annotation is genuinely needed — a Pydantic model, say — the import belongs at module scope in that file and the file belongs outside the fast path.

The same applies to default values. A default computed from a deferred module, such as crs: str = rasterio.crs.CRS.from_epsg(4326).to_string(), evaluates at definition time regardless of the annotations setting. Defaults should be plain literals, with any expansion happening inside the body where the import already lives.