spack.installer package

New installer that will ultimately replace installer.py. It features an event loop, non-blocking I/O, and a POSIX jobserver to limit concurrency. It also has a more advanced terminal UI. It’s mostly self-contained to avoid interfering with the rest of Spack too much while it’s being developed and tested.

The installer consists of a UI process that manages multiple build processes and handles updates to the database. It detects or creates a jobserver, and then kicks off an event loop in which it runs through a build queue, always running at least one build. Concurrent builds run as jobserver tokens are obtained. This means only one -j flag is needed to control concurrency.

The UI process has two modes: an overview mode where it shows the status of all builds, and a mode where it follows the logs of a specific build. It listens to keyboard input to switch between modes.

The build process does an ordinary install, but also spawns a “tee” thread that forwards its build output to both a log file and the UI process (if the UI process has requested it). This thread also runs an event loop to listen for control messages from the UI process (to enable/disable echoing of logs), and for output from the build process.

The parent-side orchestrator PackageInstaller lives in spack.installer.core, the build subprocess in spack.installer.build, the terminal UI in spack.installer.ui, and the build graph/scheduling in spack.installer.schedule. Shared low-level primitives are in spack.installer.base, with platform-specific implementations in spack.installer.posix and spack.installer.windows.

class spack.installer.PackageInstaller(packages: List[PackageBase], *, dirty: bool = False, explicit: Set[str] | bool = False, overwrite: List[str] | Set[str] | None = None, fail_fast: bool = False, fake: bool = False, include_build_deps: bool = False, install_deps: bool = True, install_package: bool = True, install_source: bool = False, keep_prefix: bool = False, keep_stage: bool = False, restage: bool = True, skip_patch: bool = False, stop_at: str | None = None, stop_before: str | None = None, tests: bool | List[str] | Set[str] = False, unsigned: bool | None = None, verbose: bool = False, concurrent_packages: int | None = None, root_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', create_reports: bool = False, ui: InstallerUI | None = None, launcher: Callable[[BuildRequest, JobServerBase], ChildInfo] = start_build, store: Store | None = None)

Bases: object

binary_cache_for_spec: Dict[str, List[MirrorMetadata]]

Per-spec buildcache mirrors; populated in install() from the binary index.

explicit: Set[str]

install() → None

overwrite: Set[str]

Set of DAG hashes to overwrite (if already installed)

overwrite_time: float

Time at which the overwrite install was requested; used to detect concurrent overwrites.

pending_builds

queue of packages ready to install (no children)

pending_expansions: List[str]

specs awaiting build-dep expansion (deferred until DB read lock is available)

Submodules

spack.installer.base module

Abstract base classes for new_installer: TUI terminal state, IPC channels, and job scheduling.

class spack.installer.base.BaseTerminalState(selector: BaseSelector, on_headless: Callable[[bool], None] | None = None, on_suspend: Callable[[], None] | None = None, on_resume: Callable[[], None] | None = None)

Bases: ABC

Abstract base for platform-specific terminal state management.

abstractmethod drain_sigwinch() → None

Drain the platform-specific sigwinch notification channel.

enter_background() → None

abstractmethod enter_foreground() → None

handle_continue() → None

headless

True while the process is backgrounded/suspended and the terminal is not ours

poll_foreground() → None

Switch to foreground mode if the process is no longer backgrounded.

read_input() → str

Read available keyboard input as decoded text.

abstractmethod setup() → None

classmethod stdin_is_interactive() → bool

stdin_reader: StdinReader

Non-blocking stdin reader, created by platform-specific subclasses in __init__

classmethod stdout_is_interactive() → bool

teardown() → None

abstractmethod teardown_input() → None

Restore input settings and signal handlers. Called before the final UI render.

teardown_output() → None

Restore output settings. Called after the final UI render.

wake_interval() → float | None

Extra wake-up cadence the terminal needs from the event loop, or None. While headless we poll for the background-to-foreground transition, since there is no signal for it.

class spack.installer.base.BuildChannels(state_r: Connection, state_w: Connection, output_r: Connection, output_w: Connection, control_r: Connection, control_w: Connection, tee_control_w: Connection | None)

Bases: NamedTuple

The state, output, and control channel pairs connecting the event loop to one build. Created by the platform-specific create_build_channels factories.

close_child_ends() → None

Close the ends owned by the build: in the parent after fork, or in launchers whose builds finish in-process.

control_r: Connection

Alias for field number 4

control_w: Connection

Alias for field number 5

output_r: Connection

Alias for field number 2

output_w: Connection

Alias for field number 3

state_r: Connection

Alias for field number 0

state_w: Connection

Alias for field number 1

tee_control_w: Connection | None

Control write end passed to the worker to stop its tee thread (POSIX only, None on Windows)

class spack.installer.base.ExitCode

Bases: object

BUILD_CACHE_MISS

Exit code used by the child process to signal a binary cache miss (no source fallback)

BUILD_ERROR

STOPPED_AT_PHASE

Exit code used by the child process to signal that the build was stopped at a phase boundary

SUCCESS

class spack.installer.base.FdInfo(build_id: str, name: str)

Bases: object

Associates a selector-registered file descriptor with a build and a channel name.

build_id

name

spack.installer.base.HEADLESS_WAKE_INTERVAL

How often the event loop should wake up to poll for a background-to-foreground transition while the terminal is headless (there is no signal for it).

spack.installer.base.InstallPolicy

Type for specifying installation source modes

alias of Literal[‘auto’, ‘cache_only’, ‘source_only’]

class spack.installer.base.JobServerBase(num_jobs: int, makeflags: str | None = None)

Bases: ABC

Abstract base for controlling build concurrency.

Parameters:

• num_jobs – The number of jobs to run concurrently.

• makeflags – The MAKEFLAGS value to parse for an existing jobserver to attach to. Pass an empty string to always create a fresh jobserver, or None to read the environment.

abstractmethod acquire(jobs: int) → int

Try and acquire at most ‘jobs’ tokens from the jobserver. Returns the number of tokens actually acquired (may be less than requested, or zero).

abstractmethod close() → None

Close any resources associated with the jobserver.

abstractmethod decrease_parallelism() → None

Decrease the target parallelism by one.

has_target_parallelism() → bool

abstractmethod increase_parallelism() → None

Increase the target parallelism by one.

abstractmethod makeflags_and_data(gmake: Spec | None) → JobserverInfo

Return the JobserverInfo to be passed to the child process.

num_jobs

The number of jobs to run concurrently

abstractmethod release() → None

Release a token back to the jobserver.

target_jobs

The target number of jobs to run concurrently, which may differ from num_jobs if the user has requested a decrease in parallelism, but we haven’t consumed enough tokens to reflect that yet. This value is used in the UI. The value self.target_jobs can only be modified if Spack owns the jobserver, and not when it’s attached to a parent jobserver.

abstractmethod update_selector(selector: BaseSelector, wake: bool) → None

Listen or stop listening for jobserver events on the given selector.

class spack.installer.base.JobserverInfo(makeflags: str | None, data: Any)

Bases: NamedTuple

What a build process needs to participate in the jobserver, produced by JobServerBase.makeflags_and_data().

data: Any

Implementation specific data serialized to the build process (e.g. pipe-based jobserver connections that the build process must inherit).

makeflags: str | None

MAKEFLAGS value to set in the build process environment, or None if there is no jobserver.

class spack.installer.base.NoopJobServer(num_jobs: int, makeflags: str | None = None)

Bases: JobServerBase

Dummy jobserver for platforms lacking jobserver support.

acquire(jobs: int) → int

Try and acquire at most ‘jobs’ tokens from the jobserver. Returns the number of tokens actually acquired (may be less than requested, or zero).

close() → None

Close any resources associated with the jobserver.

decrease_parallelism() → None

Decrease the target parallelism by one.

increase_parallelism() → None

Increase the target parallelism by one.

makeflags_and_data(gmake: Spec | None) → JobserverInfo

Return the JobserverInfo to be passed to the child process.

release() → None

Release a token back to the jobserver.

update_selector(selector: BaseSelector, wake: bool) → None

Listen or stop listening for jobserver events on the given selector.

spack.installer.base.OUTPUT_BUFFER_SIZE

Size of the output buffer for child processes

class spack.installer.base.ProcessExitNotifier

Bases: ABC

Selector-watchable handle that becomes readable when the child process exits.

close() → None

Release any resources. Default: nothing to release.

abstract property fileobj: int | socket

Object/fd to register with the selector to detect process exit.

spack.installer.base.STDIN_EVENT

Selector data keys registered by the terminal and dispatched by the event loop

class spack.installer.base.StdinReader(read_raw: Callable[[], bytes])

Bases: object

Non-blocking stdin reading with UTF-8 decoding, on top of a platform-specific function that reads raw bytes.

Raw bytes are read from the backing file descriptor or socket for stdin (instead of the TextIOWrapper) to avoid double buffering issues: the event loop triggers when the fd is ready to read, and if we do a partial read from the TextIOWrapper, it will likely drain the fd and buffer the remainder internally, which the event loop is not aware of, and user input doesn’t come through.

ansi_escape_re

For stripping out arrow and navigation keys

decoder

Handle multi-byte UTF-8 characters

read() → str

read_raw

Platform-specific function that reads available raw bytes from stdin

spack.installer.base.TEE_STOP

Control byte that stops the tee thread

class spack.installer.base.Tee(control_r: Connection, control_w: Connection | None, parent: Connection, log_path: str)

Bases: ABC

Emulates ./build 2>&1 | tee build.log. Output is sent to a log file and the parent process (if echoing is enabled). The control channel is used to enable/disable echoing.

close() → None

log_path

The path of the log file

abstractmethod run(log_r: int, log_file: BufferedWriter) → None

Read from log_r, write to log_file; echo to parent when enabled. Runs in a thread.

spack.installer.build module

Build subprocess (child) side of the new installer.

This module holds everything that runs in (or directly manages) a single build’s child process: the worker_function() entry point, the install steps it drives, and the loop-side ChildInfo handle the parent uses to talk to the child. See spack.installer for the overall design.

exception spack.installer.build.BinaryCacheMiss(message: str, long_message: str | None = None)

Bases: SpackError

class spack.installer.build.BuildRequest(spec: Spec, explicit: bool, mirrors: List[MirrorMetadata], unsigned: bool | None, install_policy: Literal['auto', 'cache_only', 'source_only'], dirty: bool, keep_stage: bool, restage: bool, keep_prefix: bool, skip_patch: bool, fake: bool, install_source: bool, run_tests: bool, log_path: str, stop_before: str | None, stop_at: str | None)

Bases: NamedTuple

Plain data describing a single build to be launched: the input of a build launcher.

dirty: bool

Alias for field number 5

explicit: bool

Alias for field number 1

fake: bool

Alias for field number 10

install_policy: Literal['auto', 'cache_only', 'source_only']

Alias for field number 4

install_source: bool

Alias for field number 11

keep_prefix: bool

Alias for field number 8

keep_stage: bool

Alias for field number 6

log_path: str

Alias for field number 13

mirrors: List[MirrorMetadata]

Alias for field number 2

restage: bool

Alias for field number 7

run_tests: bool

Alias for field number 12

skip_patch: bool

Alias for field number 9

spec: Spec

Alias for field number 0

stop_at: str | None

Alias for field number 15

stop_before: str | None

Alias for field number 14

unsigned: bool | None

Alias for field number 3

class spack.installer.build.ChildInfo(proc: ProcessLike, spec: Spec, output_r_conn: Connection, state_r_conn: Connection, control_w_conn: Connection, notifier: ProcessExitNotifier, log_path: str)

Bases: object

Loop-side handle to a running build: the child process and its IPC channels. Owns the prefix write lock while the build runs; on success ownership transfers to the pending AddSpecAction DB insert.

close(selector: BaseSelector) → int

Unregister and close file descriptors, and join the child process. Returns the exit code of the child process.

control_w_conn

log_path

notifier

output_r_conn

prefix_lock: Lock | None

proc

register_with_selector(selector: BaseSelector, build_id: str) → None

Register output, state, and sentinel channels with the selector.

release_prefix_lock() → None

spec

state_buffer

state_r_conn

spack.installer.build.OVERWRITE_BACKUP_SUFFIX

Suffix for temporary backup during overwrite install

spack.installer.build.OVERWRITE_GARBAGE_SUFFIX

Suffix for temporary cleanup during failed install

class spack.installer.build.PrefixPivoter(prefix: str, keep_prefix: bool = False)

Bases: object

Manages the installation prefix of a build.

keep_prefix

Whether to keep a failed installation prefix

tmp_prefix: str | None

Temporary location for the original prefix

class spack.installer.build.ProcessLike(*args, **kwargs)

Bases: Protocol

The part of the multiprocessing.Process interface the event loop relies on. Tests provide synthetic implementations to exercise the loop without forking.

property exitcode: int | None

is_alive() → bool

join(timeout: float | None = None) → None

kill() → None

property pid: int | None

terminate() → None

spack.installer.build.install_from_buildcache(mirrors: List[MirrorMetadata], spec: Spec, unsigned: bool | None, state_stream: TextIOWrapper) → bool

spack.installer.build.send_installed_from_binary_cache(state_pipe: TextIOWrapper) → None

Send a notification that the package was installed from binary cache.

spack.installer.build.send_progress(current: int, total: int, state_pipe: TextIOWrapper) → None

Send a progress update message.

spack.installer.build.send_state(state: str, state_pipe: TextIOWrapper) → None

Send a state update message.

spack.installer.build.start_build(request: BuildRequest, jobserver: JobServerBase) → ChildInfo

Start a new build in a child process.

spack.installer.build.worker_function(request: BuildRequest, state: Connection, parent: Connection, tee_control_r: Connection, tee_control_w: Connection | None, jobserver_info: JobserverInfo, global_state: GlobalStateMarshaler)

Function run in the build child process. Installs the requested spec, sending state updates and build output back to the parent process.

Parameters:

• request – Description of the build to perform

• state – Connection to send state updates to

• parent – Connection to send build output to

• tee_control_r – Read end of the control pipe; the parent sends echo on/off here

• tee_control_w – Write end of the control pipe; used to stop the tee thread on POSIX

• jobserver_info – MAKEFLAGS to set, so that the build process uses the POSIX jobserver, and opaque data only to be serialized (e.g. old style jobserver r/w pipes, only to be inherited in the build process)

• global_state – Global state to restore

spack.installer.core module

Parent side of the new installer: the PackageInstaller event loop.

Runs the selector-based event loop that starts builds, forwards their output, applies UI commands, and flushes finished builds to the database. See spack.installer for the overall design.

spack.installer.core.DATABASE_WRITE_INTERVAL

How often to flush completed builds to the database

class spack.installer.core.NullReportData

Bases: ReportData

No-op drop-in for ReportData when no reporter is configured.

Avoids creating InstallRecords and reading log files on every completed build.

finalize(reports: Dict[str, RequestRecord], build_graph: BuildGraph) → None

Finalize InstallRecords and append them to RequestRecords after all builds finish.

Parameters:

• reports – Map of root dag_hash to RequestRecord to append to.

• build_graph – The build graph containing all nodes and their states.

finish_record(spec: Spec, exitcode: int, log_path: str | None = None) → None

Mark the InstallRecord for a spec as succeeded or failed.

start_record(spec: Spec) → None

Begin an InstallRecord for a spec that is about to be built.

class spack.installer.core.PackageInstaller(packages: List[PackageBase], *, dirty: bool = False, explicit: Set[str] | bool = False, overwrite: List[str] | Set[str] | None = None, fail_fast: bool = False, fake: bool = False, include_build_deps: bool = False, install_deps: bool = True, install_package: bool = True, install_source: bool = False, keep_prefix: bool = False, keep_stage: bool = False, restage: bool = True, skip_patch: bool = False, stop_at: str | None = None, stop_before: str | None = None, tests: bool | List[str] | Set[str] = False, unsigned: bool | None = None, verbose: bool = False, concurrent_packages: int | None = None, root_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', create_reports: bool = False, ui: InstallerUI | None = None, launcher: Callable[[BuildRequest, JobServerBase], ChildInfo] = start_build, store: Store | None = None)

Bases: object

binary_cache_for_spec: Dict[str, List[MirrorMetadata]]

Per-spec buildcache mirrors; populated in install() from the binary index.

dependencies_policy: Literal['auto', 'cache_only', 'source_only']

explicit: Set[str]

install() → None

log_paths: Dict[str, str]

overwrite: Set[str]

Set of DAG hashes to overwrite (if already installed)

overwrite_time: float

Time at which the overwrite install was requested; used to detect concurrent overwrites.

pending_builds

queue of packages ready to install (no children)

pending_expansions: List[str]

specs awaiting build-dep expansion (deferred until DB read lock is available)

root_policy: Literal['auto', 'cache_only', 'source_only']

running_builds: Dict[str, ChildInfo]

tests: bool | List[str] | Set[str]

class spack.installer.core.ReportData(roots: List[Spec])

Bases: object

Data collected for reports during installation.

build_records: Dict[str, InstallRecord]

finalize(reports: Dict[str, RequestRecord], build_graph: BuildGraph) → None

Finalize InstallRecords and append them to RequestRecords after all builds finish.

Parameters:

• reports – Map of root dag_hash to RequestRecord to append to.

• build_graph – The build graph containing all nodes and their states.

finish_record(spec: Spec, exitcode: int, log_path: str | None = None) → None

Mark the InstallRecord for a spec as succeeded or failed.

start_record(spec: Spec) → None

Begin an InstallRecord for a spec that is about to be built.

spack.installer.posix module

POSIX-specific terminal state, stdin reader, IPC channels, and job scheduling.

class spack.installer.posix.PosixExitNotifier(proc: Process)

Bases: ProcessExitNotifier

Process-exit notifier for POSIX: the multiprocessing sentinel fd is selector-watchable.

property fileobj: int

Object/fd to register with the selector to detect process exit.

class spack.installer.posix.PosixJobServer(num_jobs: int, makeflags: str | None = None)

Bases: JobServerBase

Attach to an existing POSIX jobserver or create a FIFO-based one.

acquire(jobs: int) → int

Try and acquire at most ‘jobs’ tokens from the jobserver. Returns the number of tokens actually acquired (may be less than requested, or zero).

close() → None

Close any resources associated with the jobserver.

decrease_parallelism() → None

Decrease the target parallelism by one.

increase_parallelism() → None

Increase the target parallelism by one.

makeflags_and_data(gmake: Spec | None) → JobserverInfo

Return the JobserverInfo to be passed to the child process.

release() → None

Release a token back to the jobserver.

tokens_acquired

Keep track of how many tokens Spack itself has acquired, which is used to release them.

update_selector(selector: BaseSelector, wake: bool) → None

Listen or stop listening for jobserver events on the given selector.

class spack.installer.posix.PosixTee(control_r: Connection, control_w: Connection | None, parent: Connection, log_path: str)

Bases: Tee

run(log_r: int, log_file: BufferedWriter) → None

Forward log_r to log_file and parent (if echoing is enabled). Echoing is enabled and disabled by reading 1 and 0 resp. from control. Thread exit is triggered by EOF or by reading TEE_STOP from control.

class spack.installer.posix.PosixTerminalState(selector: BaseSelector, on_headless: Callable[[bool], None] | None = None, on_suspend: Callable[[], None] | None = None, on_resume: Callable[[], None] | None = None)

Bases: BaseTerminalState

Manages terminal settings, stdin selector registration, and suspend/resume signals.

Installs a SIGTSTP handler that restores the terminal before suspending and re-applies it on resume. After waking up it checks whether the process is in the foreground or background and enables or suppresses interactive output accordingly.

Optional on_suspend / on_resume hooks are called just before the process suspends and just after it wakes, allowing callers to pause and resume child processes.

drain_sigwinch() → None

Drain the platform-specific sigwinch notification channel.

enter_background() → None

Suppress output and stop reading stdin to avoid SIGTTIN/SIGTTOU.

enter_foreground() → None

Restore interactive terminal mode.

handle_continue() → None

Detect whether the process is in the foreground or background and adjust accordingly.

setup() → None

Set cbreak mode, register stdin and signal pipes in the selector.

teardown_input() → None

Restore terminal settings and signal handlers, close pipes.

spack.installer.posix.create_build_channels() → BuildChannels

Create the channel pairs of a build. The state and output read ends are non-blocking so the selector-based loop never blocks. The worker also receives the control write end, used to stop its tee thread.

spack.installer.posix.create_jobserver_fifo(num_jobs: int) → Tuple[int, int, str]

Create a new jobserver FIFO with the specified number of job tokens.

spack.installer.posix.get_jobserver_config(makeflags: str | None = None) → str | Tuple[int, int] | None

Parse MAKEFLAGS for jobserver. Either it’s a FIFO or (r, w) pair of file descriptors.

Parameters:

makeflags – MAKEFLAGS string to parse. If None, reads from os.environ.

spack.installer.posix.make_state_stream(state: Connection) → TextIOWrapper

Wrap the write end of the state Pipe as a line-buffered text stream.

spack.installer.posix.open_existing_jobserver_fifo(fifo_path: str) → Tuple[int, int] | None

Open an existing jobserver FIFO for reading and writing.

spack.installer.posix.read_connection(conn: Connection, max_size: int = 4096) → bytes

spack.installer.posix.write_connection(conn: Connection, data: bytes) → None

spack.installer.schedule module

Build graph and scheduling for the new installer.

Holds the BuildGraph (the DAG of specs to install, with lazy build-dependency expansion), schedule_builds() (per-spec lock acquisition and readiness selection), and the DatabaseAction hierarchy of pending DB writes. See spack.installer for the overall design.

class spack.installer.schedule.AddSpecAction(spec: Spec, explicit: bool, prefix_lock: Lock | None)

Bases: DatabaseAction

Action to add a successfully built spec to the database. Owns the prefix write lock from the moment the build finishes until the DB write (downgrade to read) or cleanup (release).

explicit

save_to_db(db: Database) → None

class spack.installer.schedule.BuildGraph(specs: List[Spec], root_policy: Literal['auto', 'cache_only', 'source_only'], dependencies_policy: Literal['auto', 'cache_only', 'source_only'], include_build_deps: bool, install_package: bool, install_deps: bool, store: Store, overwrite_set: Set[str] | None = None, tests: bool | List[str] | Set[str] = False, explicit_set: Set[str] | None = None, has_mirrors: bool = True)

Bases: object

Represents the dependency graph for package installation.

child_to_parent: Dict[str, Set[str]]

done: Set[str]

enqueue_parents(dag_hash: str, pending_builds: List[str]) → None

After a spec is installed, remove it from the graph and enqueue any parents that are now ready to install.

Parameters:

• dag_hash – The dag_hash of the spec that was just installed

• pending_builds – List to append parent specs that are ready to build

expand_build_deps(spec_hashes: List[str], pending_builds: List[str], database: Database, dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto') → List[str]

Expand build dependencies for a list of specs after binary cache misses.

Adds the spec’s build deps and their transitive runtime deps to the graph. When dependencies_policy is "source_only", build deps of newly added specs are included immediately. Installed deps are skipped without adding edges.

The caller must hold the database read lock and have called db._read().

Returns the list of newly added dag hashes.

force_source: Set[str]

get_unexpanded_build_deps(dag_hash: str) → List[Spec]

Returns a list of unprocessed build deps for a spec.

has_unexpanded_build_deps(dag_hash: str) → bool

parent_to_child: Dict[str, Set[str]]

pruned: Set[str]

class spack.installer.schedule.DatabaseAction

Bases: object

Base class for objects that need to be persisted to the database.

prefix_lock: Lock | None

release_prefix_lock() → None

save_to_db(db: Database) → None

spec: Spec

class spack.installer.schedule.MarkExplicitAction(spec: Spec)

Bases: DatabaseAction

Action to mark an already installed spec as explicitly installed.

save_to_db(db: Database) → None

class spack.installer.schedule.ScheduleResult(blocked: bool, to_start: List[Tuple[str, Lock]], newly_installed: List[Tuple[str, Spec, Lock]], to_mark_explicit: List[MarkExplicitAction])

Bases: NamedTuple

Return value of schedule_builds().

blocked: bool

True if any pending builds were blocked on locks held by other processes.

newly_installed: List[Tuple[str, Spec, Lock]]

(dag_hash, spec, lock) triples found already installed by another process; the read lock is held and the caller must add it to retained_read_locks.

to_mark_explicit: List[MarkExplicitAction]

Actions to mark already installed specs explicit in the DB.

to_start: List[Tuple[str, Lock]]

(dag_hash, lock) pairs where the write lock is held and the caller must start the build and eventually release the lock.

spack.installer.schedule.schedule_builds(pending: List[str], build_graph: BuildGraph, store: Store, overwrite: Set[str], overwrite_time: float, capacity: int, needs_jobserver_token: bool, jobserver: JobServerBase, explicit: Set[str]) → ScheduleResult

Try to schedule as many pending builds as possible.

For each pending spec, attempts to acquire a non-blocking per-spec write lock. If the write lock times out, a read lock is tried as a fallback: a successful read lock means the first process finished and downgraded its write lock. If the DB confirms the spec is installed, it is captured as newly_installed; if the DB says it is not installed, the concurrent process was likely killed mid-build, and the spec is retried next iteration. Under both the DB read lock and the prefix lock, checks whether another process has already installed the spec. If so, captures it as newly_installed (caller enqueues parents) and keeps a read lock on the prefix to prevent concurrent uninstall. Otherwise, acquires a jobserver token if needed and adds the (dag_hash, lock) pair to to_start (caller launches the build).

Parameters:

• pending – List of dag hashes pending installation; modified in-place.

• build_graph – The build dependency graph; used for node lookup and parent enqueueing.

• store – The local store for installs.

• overwrite – Set of dag hashes to overwrite even if already installed.

• overwrite_time – Timestamp (from time.time()) at which the overwrite install was requested. A spec in overwrite whose DB installation_time >= overwrite_time was installed by a concurrent process after our request started and should be treated as done.

• capacity – Maximum number of new builds to add to to_start in this call.

• needs_jobserver_token – True if a jobserver token is required for the first new build.

• jobserver – Jobserver for acquiring tokens.

• explicit – Set of dag hashes to mark explicit in the DB if found already installed.

Returns:

A ScheduleResult with blocked, to_start, and newly_installed fields; see ScheduleResult for field semantics.

spack.installer.ui module

Terminal UI for the new installer.

Defines the InstallerUI frontend contract (a headless no-op base) and the interactive TerminalUI implementation, plus the plain BuildInfo record and the UiCommand messages the UI produces for the event loop. See spack.installer for the overall design.

class spack.installer.ui.BuildInfo(build_id: str, *, name: str, version: str, external: bool, prefix: str, explicit: bool, log_path: str | None = None)

Bases: object

Plain data about a package being built: the payload of InstallerUI.on_build_added and the per-build record of the terminal UI.

duration: float | None

explicit: bool

external: bool

finished_time: float | None

hash: str

id: str

log_path

log_summary: str | None

name: str

prefix: str

progress_percent: int | None

start_time: float

state: str

version: str

spack.installer.ui.CLEANUP_TIMEOUT

How long to display finished packages before graying them out

class spack.installer.ui.ChangeJobs(delta: int)

Bases: NamedTuple

Command that increments/decrements job server tokens when positive/negative respectively.

delta: int

Alias for field number 0

class spack.installer.ui.InstallerUI

Bases: object

Interface between the installer event loop and a frontend. The methods are no-ops, which makes this class usable as a headless frontend.

The event loop calls methods to notify the frontend of build events, and the frontend can append to self.commands to request actions from the event loop.

commands: List[SetEcho | ChangeJobs]

The frontend appends commands to this list to request actions from the event loop.

is_tty

Whether the frontend renders interactively; determines the event loop wake interval.

on_blocked_changed(blocked: bool) → None

Whether all pending builds are blocked by another Spack process.

on_build_added(info: BuildInfo) → None

A build was started, or a spec was found installed by another process.

on_build_removed(build_id: str) → None

A build was removed after a binary cache miss; it is added back via on_build_added once it is rescheduled as a source build.

on_finished(failures: List[str]) → None

The whole installation finished. Called once at the end of the run, and is different from on_state_changed(build_id, "finished") which is called after each successful package installation. failures is a list of failed build ids (empty on success). The frontend may print a failure summary.

on_headless_changed(headless: bool) → None

The process moved to the background (True) or foreground (False); the frontend should suppress rendering while headless. Called only through the terminal path.

on_input(chars: str) → None

Keyboard input received (only called for interactive frontends).

on_jobs_changed(actual: int, target: int) → None

The actual and/or target number of concurrent jobs changed.

on_log_output(build_id: str, data: bytes) → None

Raw log output received from a build whose echoing is enabled.

on_progress(build_id: str, current: int, total: int) → None

Fetch progress of a build changed.

on_resize() → None

The terminal was resized (only called for interactive frontends).

on_state_changed(build_id: str, state: str) → None

A build transitioned to a new state (e.g. "staging", "finished").

on_total_increased(count: int) → None

The total number of scheduled builds increased (e.g. build dep expansion).

reads_terminal_input

Whether the frontend renders to and reads keyboard input from the controlling terminal. If True, the event loop manages terminal state (cbreak mode, suspend/resume signals, stdin registration) and feeds keyboard input to on_input.

refresh_interval() → float | None

How often the frontend needs a periodic render tick, or None if it never does. The event loop bounds its sleep by this together with its own database-flush cadence.

render(finalize: bool = False) → None

Periodic tick to redraw; finalize is True for the final render.

spack.installer.ui.SPINNER_INTERVAL

How often to update a spinner in seconds

class spack.installer.ui.SetEcho(build_id: str, echo: bool)

Bases: NamedTuple

Command that enables/disables log forwarding from a build.

build_id: str

Alias for field number 0

echo: bool

Alias for field number 1

class spack.installer.ui.TerminalUI(total: int, stdout: TextIOWrapper | None = None, get_terminal_size: Callable[[], terminal_size] = os.get_terminal_size, get_time: Callable[[], float] = time.monotonic, is_tty: bool | None = None, color: bool | None = None, verbose: bool = False, filter_padding: bool = False)

Bases: InstallerUI

Terminal frontend: renders an interactive build overview and follows build logs.

actual_jobs: int

blocked: bool

builds: Dict[str, BuildInfo]

enter_search() → None

finished_builds: List[BuildInfo]

headless

When True, suppress all terminal output (process is in background).

next(direction: int = 1) → None

Follow the logs of the next build in the list.

on_blocked_changed(blocked: bool) → None

Set whether all pending builds are blocked by another Spack process.

on_build_added(info: BuildInfo) → None

Add a new build to the display and mark the display as dirty.

on_build_removed(build_id: str) → None

Remove a build from the display (e.g. after a binary cache miss before retry).

on_finished(failures: List[str]) → None

Write the stored log summaries of the failed builds to stderr.

on_headless_changed(headless: bool) → None

The process moved to the background (True) or foreground (False); the frontend should suppress rendering while headless. Called only through the terminal path.

on_input(chars: str) → None

Dispatch keyboard input to search, navigation, and parallelism actions.

on_jobs_changed(actual: int, target: int) → None

Set the actual and target number of jobs to run concurrently.

on_log_output(build_id: str, data: bytes) → None

Raw log output received from a build whose echoing is enabled.

on_progress(build_id: str, current: int, total: int) → None

Update the progress of a package and mark the display as dirty.

on_resize() → None

Refresh cached terminal size and trigger a redraw.

on_state_changed(build_id: str, state: str) → None

Update the state of a package and mark the display as dirty.

on_total_increased(count: int) → None

The total number of scheduled builds increased (e.g. build dep expansion).

refresh_interval() → float | None

How often the frontend needs a periodic render tick, or None if it never does. The event loop bounds its sleep by this together with its own database-flush cadence.

render(finalize: bool = False) → None

Redraw the interactive display.

search_input(input: str) → None

Handle keyboard input when in search mode

target_jobs: int

terminal_size_changed: bool

toggle() → None

Toggle between overview mode and following a specific build.

total

Ordered dict of build ID -> info

verbose

Verbose mode only applies to non-TTY where we want to track a single build log.

spack.installer.ui.UiCommand

A command produced by the UI and executed by the event loop.