spack.environment package

This package implements Spack environments.

spack.lock format

Spack environments have existed since Spack v0.12.0, and there have been 4 different spack.lock formats since then. The formats are documented here.

The high-level format of a Spack lockfile hasn’t changed much between versions, but the contents have. Lockfiles are JSON-formatted and their top-level sections are:

_meta (object): this contains details about the file format, including:

file-type: always "spack-lockfile"

lockfile-version: an integer representing the lockfile format version

specfile-version: an integer representing the spec format version (since v0.17)

spack (object): optional, this identifies information about Spack
used to concretize the environment: * type: required, identifies form Spack version took (e.g., git, release) * commit: the commit if the version is from git * version: the Spack version

roots (list): an ordered list of records representing the roots of the Spack
environment. Each has two fields: * hash: a Spack spec hash uniquely identifying the concrete root spec * spec: a string representation of the abstract spec that was concretized

concrete_specs: a dictionary containing the specs in the environment.

Compatibility

New versions of Spack can (so far) read all old lockfile formats – they are backward-compatible. Old versions cannot read new lockfile formats, and you’ll need to upgrade Spack to use them.

Lockfile version compatibility across Spack versions
Spack version	`v1`	`v2`	`v3`	`v4`
`v0.12:0.14`	✅
`v0.15:0.16`	✅	✅
`v0.17`	✅	✅	✅
`v0.18:`	✅	✅	✅	✅

Version 1

When lockfiles were first created, there was only one hash in Spack: the DAG hash. This DAG hash (we’ll call it the old DAG hash) did not include build dependencies – it only included transitive link and run dependencies.

The spec format at this time was keyed by name. Each spec started with a key for its name, whose value was a dictionary of other spec attributes. The lockfile put these name-keyed specs into dictionaries keyed by their DAG hash, and the spec records did not actually have a “hash” field in the lockfile – you have to associate the hash from the key with the spec record after the fact.

Dependencies in original lockfiles were keyed by "hash", i.e. the old DAG hash.

{
    "_meta": {
        "file-type": "spack-lockfile",
        "lockfile-version": 1
    },
    "roots": [
        {
            "hash": "<old_dag_hash 1>",
            "spec": "<abstract spec 1>"
        },
        {
            "hash": "<old_dag_hash 2>",
            "spec": "<abstract spec 2>"
        }
    ],
    "concrete_specs": {
        "<old_dag_hash 1>": {
            "... <spec dict attributes> ...": { },
            "dependencies": {
                "depname_1": {
                    "hash": "<old_dag_hash for depname_1>",
                    "type": ["build", "link"]
                },
                "depname_2": {
                    "hash": "<old_dag_hash for depname_3>",
                    "type": ["build", "link"]
                }
            },
            "hash": "<old_dag_hash 1>"
        },
        "<old_dag_hash 2>": {
            "... <spec dict attributes> ...": { },
            "dependencies": {
                "depname_3": {
                    "hash": "<old_dag_hash for depname_3>",
                    "type": ["build", "link"]
                },
                "depname_4": {
                    "hash": "<old_dag_hash for depname_4>",
                    "type": ["build", "link"]
                },
            },
            "hash": "<old_dag_hash 2>"
        },
    }
}

Version 2

Version 2 changes one thing: specs in the lockfile are now keyed by build_hash instead of the old dag_hash. Specs have a hash attribute with their real DAG hash, so you can’t go by the dictionary key anymore to identify a spec – you have to read it in and look at "hash". Dependencies are still keyed by old DAG hash.

Even though we key lockfiles by build_hash, specs in Spack were still deployed with the old, coarser DAG hash. This means that in v2 and v3 lockfiles (which are keyed by build hash), there may be multiple versions of the same spec with different build dependencies, which means they will have different build hashes but the same DAG hash. Spack would only have been able to actually install one of these.

{
    "_meta": {
        "file-type": "spack-lockfile",
        "lockfile-version": 2
    },
    "roots": [
        {
            "hash": "<build_hash 1>",
            "spec": "<abstract spec 1>"
        },
        {
            "hash": "<build_hash 2>",
            "spec": "<abstract spec 2>"
        }
    ],
    "concrete_specs": {
        "<build_hash 1>": {
            "... <spec dict attributes> ...": { },
            "dependencies": {
                "depname_1": {
                    "hash": "<old_dag_hash for depname_1>",
                    "type": ["build", "link"]
                },
                "depname_2": {
                    "hash": "<old_dag_hash for depname_3>",
                    "type": ["build", "link"]
                }
            },
            "hash": "<old_dag_hash 1>",
        },
        "<build_hash 2>": {
            "... <spec dict attributes> ...": { },
            "dependencies": {
                "depname_3": {
                    "hash": "<old_dag_hash for depname_3>",
                    "type": ["build", "link"]
                },
                "depname_4": {
                    "hash": "<old_dag_hash for depname_4>",
                    "type": ["build", "link"]
                }
            },
            "hash": "<old_dag_hash 2>"
        }
    }
}

Version 3

Version 3 doesn’t change the top-level lockfile format, but this was when we changed the specfile format. Specs in concrete_specs are now keyed by the build hash, with no inner dictionary keyed by their package name. The package name is in a name field inside each spec dictionary. The dependencies field in the specs is a list instead of a dictionary, and each element of the list is a record with the name, dependency types, and hash of the dependency. Instead of a key called hash, dependencies are keyed by build_hash. Each spec still has a hash attribute.

Version 3 adds the specfile_version field to _meta and uses the new JSON spec format.

{
    "_meta": {
        "file-type": "spack-lockfile",
        "lockfile-version": 3,
        "specfile-version": 2
    },
    "roots": [
        {
            "hash": "<build_hash 1>",
            "spec": "<abstract spec 1>"
        },
        {
            "hash": "<build_hash 2>",
            "spec": "<abstract spec 2>"
        },
    ],
    "concrete_specs": {
        "<build_hash 1>": {
            "... <spec dict attributes> ...": { },
            "dependencies": [
                {
                    "name": "depname_1",
                    "build_hash": "<build_hash for depname_1>",
                    "type": ["build", "link"]
                },
                {
                    "name": "depname_2",
                    "build_hash": "<build_hash for depname_2>",
                    "type": ["build", "link"]
                },
            ],
            "hash": "<old_dag_hash 1>",
        },
        "<build_hash 2>": {
            "... <spec dict attributes> ...": { },
            "dependencies": [
                {
                    "name": "depname_3",
                    "build_hash": "<build_hash for depname_3>",
                    "type": ["build", "link"]
                },
                {
                    "name": "depname_4",
                    "build_hash": "<build_hash for depname_4>",
                    "type": ["build", "link"]
                },
            ],
            "hash": "<old_dag_hash 2>"
        }
    }
}

Version 4

Version 4 removes build hashes and is keyed by the new DAG hash (hash). The hash now includes build dependencies and a canonical hash of the package.py file. Dependencies are keyed by hash (DAG hash) as well. There are no more build_hash fields in the specs, and there are no more issues with lockfiles being able to store multiple specs with the same DAG hash (because the DAG hash is now finer-grained). An optional spack property may be included to track version information, such as the commit or version.

{
    "_meta": {
        "file-type": "spack-lockfile",
        "lockfile-version": 4,
        "specfile-version": 3
    },
    "roots": [
        {
            "hash": "<dag_hash 1>",
            "spec": "<abstract spec 1>"
        },
        {
            "hash": "<dag_hash 2>",
            "spec": "<abstract spec 2>"
        }
    ],
    "concrete_specs": {
        "<dag_hash 1>": {
            "... <spec dict attributes> ...": { },
            "dependencies": [
                {
                    "name": "depname_1",
                    "hash": "<dag_hash for depname_1>",
                    "type": ["build", "link"]
                },
                {
                    "name": "depname_2",
                    "hash": "<dag_hash for depname_2>",
                    "type": ["build", "link"]
                }
            ],
            "hash": "<dag_hash 1>",
        },
        "<daghash 2>": {
            "... <spec dict attributes> ...": { },
            "dependencies": [
                {
                    "name": "depname_3",
                    "hash": "<dag_hash for depname_3>",
                    "type": ["build", "link"]
                },
                {
                    "name": "depname_4",
                    "hash": "<dag_hash for depname_4>",
                    "type": ["build", "link"]
                }
            ],
            "hash": "<dag_hash 2>"
        }
    }
}

class spack.environment.Environment(manifest_dir: str | Path)[source]

Bases: object

A Spack environment, which bundles together configuration and a list of specs.

property active: True if this environment is currently active.

add(user_spec, list_name='specs')[source]

Add a single user_spec (non-concretized) to the Environment

Returns:

True if the spec was added, False if it was already: present and did not need to be added

Return type:

(bool)

add_view_to_env(env_mod: EnvironmentModifications, view: str) → EnvironmentModifications[source]

Collect the environment modifications to activate an environment using the provided view. Removes duplicate paths.

Parameters:

env_mod – the environment modifications object that is modified.
view – the name of the view to activate.

added_specs()[source]

Specs that are not yet installed.

Yields the user spec for non-concretized specs, and the concrete spec for already concretized but not yet installed specs.

all_hashes()[source]: Return hashes of all specs.

all_matching_specs(*specs: Spec) → List[Spec][source]: Returns all concretized specs in the environment satisfying any of the input specs

all_specs() → List[Spec][source]: Returns a list of all concrete specs

all_specs_generator() → Iterable[Spec][source]: Returns a generator for all concrete specs

change_existing_spec(change_spec: Spec, list_name: str = 'specs', match_spec: Spec | None = None, allow_changing_multiple_specs=False)[source]

Find the spec identified by match_spec and change it to change_spec.

Parameters:

change_spec – defines the spec properties that need to be changed. This will not change attributes of the matched spec unless they conflict with change_spec.
list_name – identifies the spec list in the environment that should be modified
match_spec – if set, this identifies the spec that should be changed. If not set, it is assumed we are looking for a spec with the same name as change_spec.

check_views()[source]: Checks if the environments default view can be activated.

clear(re_read=False)[source]

Clear the contents of the environment

Parameters:: re_read – If True, do not clear new_specs. This value cannot be read from yaml, and needs to be maintained when re-reading an existing environment.

concrete_roots()[source]: Same as concretized_specs, except it returns the list of concrete roots without associated user spec

concretize(force=False, tests=False)[source]

Concretize user_specs in this environment.

Only concretizes specs that haven’t been concretized yet unless force is True.

This only modifies the environment in memory. write() will write out a lockfile containing concretized specs.

Parameters:

force (bool) – re-concretize ALL specs, even those that were already concretized
tests (bool or list or set) – False to run no tests, True to test all packages, or a list of package names to run tests for some

Returns:

List of specs that have been concretized. Each entry is a tuple of the user spec and the corresponding concretized spec.

concretized_order: List[str]: Roots associated with the last concretization, in order

concretized_specs()[source]: Tuples of (user spec, concrete spec) for all concrete specs.

concretized_user_specs: List[Spec]: User specs from the last concretization

deconcretize(spec: Spec, concrete: bool = True)[source]

Remove specified spec from environment concretization

Parameters:

spec – Spec to deconcretize. This must be a root of the environment
concrete – If True, find all instances of spec as concrete in the environemnt. If False, find a single instance of the abstract spec as root of the environment.

property default_view

delete_default_view() → None[source]: Deletes the default view associated with this environment.

destroy()[source]: Remove this environment from Spack entirely.

property dev_specs

ensure_env_directory_exists(dot_env: bool = False) → None[source]

Ensure that the root directory of the environment exists

Parameters:: dot_env – if True also ensures that the <root>/.env directory exists

get_by_hash(dag_hash: str) → List[Spec][source]

get_one_by_hash(dag_hash)[source]: Returns the single spec from the environment which matches the provided hash. Raises an AssertionError if no specs match or if more than one spec matches.

has_view(view_name: str) → bool[source]

install_all(**install_args)[source]

Install all concretized specs in an environment.

Note: this does not regenerate the views for the environment; that needs to be done separately with a call to write().

Parameters:: install_args (dict) – keyword install arguments

install_specs(specs: List[Spec] | None = None, **install_args)[source]

invalidate_repository_cache()[source]

is_develop(spec)[source]: Returns true when the spec is built from local sources

property lock_path: Path to spack.lock file in this environment.

property manifest_path: Path to spack.yaml file in this environment.

manifest_uptodate_or_warn()[source]: Emits a warning if the manifest file is not up-to-date.

matching_spec(spec)[source]

Given a spec (likely not concretized), find a matching concretized spec in the environment.

The matching spec does not have to be installed in the environment, but must be concrete (specs added with spack add without an intervening spack concretize will not be matched).

If there is a single root spec that matches the provided spec or a single dependency spec that matches the provided spec, then the concretized instance of that spec will be returned.

If multiple root specs match the provided spec, or no root specs match and multiple dependency specs match, then this raises an error and reports all matching specs.

regenerate_views()[source]

remove(query_spec, list_name='specs', force=False)[source]: Remove specs from an environment that match a query_spec

removed_specs()[source]: Tuples of (user spec, concrete spec) for all specs that will be removed on next concretize.

property repo

property repos_path

rm_view_from_env(env_mod: EnvironmentModifications, view: str) → EnvironmentModifications[source]

Collect the environment modifications to deactivate an environment using the provided view. Reverses the action of add_view_to_env.

Parameters:

env_mod – the environment modifications object that is modified.
view – the name of the view to deactivate.

roots()[source]

Specs explicitly requested by the user in this environment.

Yields both added and installed specs that have user specs in spack.yaml.

property scope_name: Name of the config scope of this environment’s manifest file.

spec_lists: Dict[str, SpecList]: Specs from “spack.yaml”

specs_by_hash: Dict[str, Spec]: Concretized specs by hash

property unify

uninstalled_specs()[source]: Return root specs that are not installed, or are installed, but are development specs themselves or have those among their dependencies.

update_default_view(path_or_bool: str | bool) → None[source]

Updates the path of the default view.

If the argument passed as input is False the default view is deleted, if present. The manifest will have an entry “view: false”.

If the argument passed as input is True a default view is created, if not already present. The manifest will have an entry “view: true”. If a default view is already declared, it will be left untouched.

If the argument passed as input is a path a default view pointing to that path is created, if not present already. If a default view is already declared, only its “root” will be changed.

Parameters:: path_or_bool – either True, or False or a path

update_environment_repository() → None[source]: Updates the repository associated with the environment.

update_lockfile() → None[source]

update_stale_references(from_list=None)[source]: Iterate over spec lists updating references.

property user_specs

property view_path_default

write(regenerate: bool = True) → None[source]

Writes an in-memory environment to its location on disk.

Write out package files for each newly concretized spec. Also regenerate any views associated with the environment and run post-write hooks, if regenerate is True.

Parameters:: regenerate – regenerate views and run post-write hooks as well as writing if True.

write_transaction()[source]: Get a write lock context manager for use in a with block.

exception spack.environment.SpackEnvironmentConfigError(message, long_message=None)[source]

Bases: SpackEnvironmentError

Class for Spack environment-specific configuration errors.

exception spack.environment.SpackEnvironmentError(message, long_message=None)[source]

Bases: SpackError

Superclass for all errors to do with Spack environments.

exception spack.environment.SpackEnvironmentViewError(message, long_message=None)[source]

Bases: SpackEnvironmentError

Class for errors regarding view generation.

spack.environment.activate(env, use_env_repo=False)[source]

Activate an environment.

To activate an environment, we add its manifest’s configuration scope to the existing Spack configuration, and we set active to the current environment.

Parameters:

env (Environment) – the environment to activate
use_env_repo (bool) – use the packages exactly as they appear in the environment’s repository

spack.environment.active(name)[source]: True if the named environment is active.

spack.environment.active_environment() → Environment | None[source]: Returns the active environment when there is any

spack.environment.all_environment_names()[source]: List the names of environments that currently exist.

spack.environment.all_environments()[source]: Generator for all managed Environments.

Create a managed environment in Spack and returns it.

A managed environment is created in a root directory managed by this Spack instance, so that Spack can keep track of them.

Files with suffix .json or .lock are considered lockfiles. Files with any other name are considered manifest files.

Parameters:

name – name of the managed environment
init_file – either a lockfile, a manifest file, or None
with_view – whether a view should be maintained for the environment. If the value is a string, it specifies the path to the view
keep_relative – if True, develop paths are copied verbatim into the new environment file, otherwise they are made absolute

Create an environment in the directory passed as input and returns it.

Files with suffix .json or .lock are considered lockfiles. Files with any other name are considered manifest files.

Parameters:

root – directory where to create the environment.
init_file – either a lockfile, a manifest file, or None
with_view – whether a view should be maintained for the environment. If the value is a string, it specifies the path to the view
keep_relative – if True, develop paths are copied verbatim into the new environment file, otherwise they are made absolute

spack.environment.deactivate()[source]: Undo any configuration or repo settings modified by activate().

spack.environment.default_manifest_yaml()[source]: default spack.yaml file to put in new environments

spack.environment.display_specs(concretized_specs)[source]

Displays the list of specs returned by Environment.concretize().

Parameters:: concretized_specs (list) – list of specs returned by Environment.concretize()

spack.environment.environment_dir_from_name(name: str, exists_ok: bool = True) → str[source]

Returns the directory associated with a named environment.

Parameters:

name – name of the environment
exists_ok – if False, raise an error if the environment exists already

Raises:

SpackEnvironmentError – if exists_ok is False and the environment exists already

spack.environment.exists(name)[source]: Whether an environment with this name exists or not.

spack.environment.initialize_environment_dir(environment_dir: str | Path, envfile: str | Path | None) → None[source]

Initialize an environment directory starting from an envfile.

Files with suffix .json or .lock are considered lockfiles. Files with any other name are considered manifest files.

Parameters:

environment_dir – directory where the environment should be placed
envfile – manifest file or lockfile used to initialize the environment

Raises:

SpackEnvironmentError – if the directory can’t be initialized

spack.environment.installed_specs()[source]: Returns the specs of packages installed in the active environment or None if no packages are installed.

spack.environment.is_env_dir(path)[source]: Whether a directory contains a spack environment.

spack.environment.is_latest_format(manifest)[source]

Return False if the manifest file exists and is not in the latest schema format.

Parameters:: manifest (str) – manifest file to be analyzed

spack.environment.manifest_file(env_name_or_dir)[source]

Return the absolute path to a manifest file given the environment name or directory.

Parameters:: env_name_or_dir (str) – either the name of a valid environment or a directory where a manifest file resides
Raises:: AssertionError – if the environment is not found

spack.environment.no_active_environment()[source]: Deactivate the active environment for the duration of the context. Has no effect when there is no active environment.

spack.environment.read(name)[source]: Get an environment with the supplied name.

spack.environment.root(name)[source]: Get the root directory for an environment by name.

spack.environment.update_yaml(manifest, backup_file)[source]

Update a manifest file from an old format to the current one.

Parameters:

manifest (str) – path to a manifest file
backup_file (str) – file where to copy the original manifest

Returns:

True if the manifest was updated, False otherwise.

Raises:

AssertionError – in case anything goes wrong during the update

Submodules

spack.environment.depfile module

This module contains the traversal logic and models that can be used to generate depfiles from an environment.

class spack.environment.depfile.DepfileNode(target: Spec, prereqs: List[Spec], buildcache: UseBuildCache)[source]

Bases: object

Contains a spec, a subset of its dependencies, and a flag whether it should be buildcache only/never/auto.

class spack.environment.depfile.DepfileSpecVisitor(pkg_buildcache: UseBuildCache, deps_buildcache: UseBuildCache)[source]

Bases: object

This visitor produces an adjacency list of a (reduced) DAG, which is used to generate depfile targets with their prerequisites. Currently it only drops build deps when using buildcache only mode.

Note that the DAG could be reduced even more by dropping build edges of specs installed at the moment the depfile is generated, but that would produce stateful depfiles that would not fail when the database is wiped later.

accept(node)[source]

neighbors(node)[source]: Produce a list of spec to follow from node

class spack.environment.depfile.MakefileModel(env: Environment, roots: List[Spec], adjacency_list: List[DepfileNode], make_prefix: str | None, jobserver: bool)[source]

Bases: object

This class produces all data to render a makefile for specs of an environment.

property empty

static from_env(env: Environment, *, filter_specs: List[Spec] | None = None, pkg_buildcache: UseBuildCache = UseBuildCache.AUTO, dep_buildcache: UseBuildCache = UseBuildCache.AUTO, make_prefix: str | None = None, jobserver: bool = True) → MakefileModel[source]

Produces a MakefileModel from an environment and a list of specs.

Parameters:

env – the environment to use
filter_specs – if provided, only these specs will be built from the environment, otherwise the environment roots are used.
pkg_buildcache – whether to only use the buildcache for top-level specs.
dep_buildcache – whether to only use the buildcache for non-top-level specs.
make_prefix – the prefix for the makefile targets
jobserver – when enabled, make will invoke Spack with jobserver support. For dry-run this should be disabled.

to_dict()[source]

class spack.environment.depfile.MakefileSpec(spec)[source]

Bases: object

Limited interface to spec to help generate targets etc. without introducing unwanted special characters.

safe_format(format_str)[source]

safe_name()[source]

spec_hash()[source]

unsafe_format(format_str)[source]

class spack.environment.depfile.UseBuildCache(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

AUTO = 3

NEVER = 2

ONLY = 1

static from_string(s: str) → UseBuildCache[source]

spack.environment.environment module

class spack.environment.environment.Environment(manifest_dir: str | Path)[source]

Bases: object

A Spack environment, which bundles together configuration and a list of specs.

property active: True if this environment is currently active.

add(user_spec, list_name='specs')[source]

Add a single user_spec (non-concretized) to the Environment

Returns:

True if the spec was added, False if it was already: present and did not need to be added

Return type:

(bool)

add_view_to_env(env_mod: EnvironmentModifications, view: str) → EnvironmentModifications[source]

Collect the environment modifications to activate an environment using the provided view. Removes duplicate paths.

Parameters:

env_mod – the environment modifications object that is modified.
view – the name of the view to activate.

added_specs()[source]

Specs that are not yet installed.

Yields the user spec for non-concretized specs, and the concrete spec for already concretized but not yet installed specs.

all_hashes()[source]: Return hashes of all specs.

all_matching_specs(*specs: Spec) → List[Spec][source]: Returns all concretized specs in the environment satisfying any of the input specs

all_specs() → List[Spec][source]: Returns a list of all concrete specs

all_specs_generator() → Iterable[Spec][source]: Returns a generator for all concrete specs

change_existing_spec(change_spec: Spec, list_name: str = 'specs', match_spec: Spec | None = None, allow_changing_multiple_specs=False)[source]

Find the spec identified by match_spec and change it to change_spec.

Parameters:

change_spec – defines the spec properties that need to be changed. This will not change attributes of the matched spec unless they conflict with change_spec.
list_name – identifies the spec list in the environment that should be modified
match_spec – if set, this identifies the spec that should be changed. If not set, it is assumed we are looking for a spec with the same name as change_spec.

check_views()[source]: Checks if the environments default view can be activated.

clear(re_read=False)[source]

Clear the contents of the environment

Parameters:: re_read – If True, do not clear new_specs. This value cannot be read from yaml, and needs to be maintained when re-reading an existing environment.

concrete_roots()[source]: Same as concretized_specs, except it returns the list of concrete roots without associated user spec

concretize(force=False, tests=False)[source]

Concretize user_specs in this environment.

Only concretizes specs that haven’t been concretized yet unless force is True.

This only modifies the environment in memory. write() will write out a lockfile containing concretized specs.

Parameters:

force (bool) – re-concretize ALL specs, even those that were already concretized
tests (bool or list or set) – False to run no tests, True to test all packages, or a list of package names to run tests for some

Returns:

List of specs that have been concretized. Each entry is a tuple of the user spec and the corresponding concretized spec.

concretized_order: List[str]: Roots associated with the last concretization, in order

concretized_specs()[source]: Tuples of (user spec, concrete spec) for all concrete specs.

concretized_user_specs: List[Spec]: User specs from the last concretization

deconcretize(spec: Spec, concrete: bool = True)[source]

Remove specified spec from environment concretization

Parameters:

spec – Spec to deconcretize. This must be a root of the environment
concrete – If True, find all instances of spec as concrete in the environemnt. If False, find a single instance of the abstract spec as root of the environment.

property default_view

delete_default_view() → None[source]: Deletes the default view associated with this environment.

destroy()[source]: Remove this environment from Spack entirely.

property dev_specs

ensure_env_directory_exists(dot_env: bool = False) → None[source]

Ensure that the root directory of the environment exists

Parameters:: dot_env – if True also ensures that the <root>/.env directory exists

get_by_hash(dag_hash: str) → List[Spec][source]

get_one_by_hash(dag_hash)[source]: Returns the single spec from the environment which matches the provided hash. Raises an AssertionError if no specs match or if more than one spec matches.

has_view(view_name: str) → bool[source]

install_all(**install_args)[source]

Install all concretized specs in an environment.

Note: this does not regenerate the views for the environment; that needs to be done separately with a call to write().

Parameters:: install_args (dict) – keyword install arguments

install_specs(specs: List[Spec] | None = None, **install_args)[source]

invalidate_repository_cache()[source]

is_develop(spec)[source]: Returns true when the spec is built from local sources

property lock_path: Path to spack.lock file in this environment.

property manifest_path: Path to spack.yaml file in this environment.

manifest_uptodate_or_warn()[source]: Emits a warning if the manifest file is not up-to-date.

matching_spec(spec)[source]

Given a spec (likely not concretized), find a matching concretized spec in the environment.

The matching spec does not have to be installed in the environment, but must be concrete (specs added with spack add without an intervening spack concretize will not be matched).

If there is a single root spec that matches the provided spec or a single dependency spec that matches the provided spec, then the concretized instance of that spec will be returned.

If multiple root specs match the provided spec, or no root specs match and multiple dependency specs match, then this raises an error and reports all matching specs.

new_specs: List[Spec]

regenerate_views()[source]

remove(query_spec, list_name='specs', force=False)[source]: Remove specs from an environment that match a query_spec

removed_specs()[source]: Tuples of (user spec, concrete spec) for all specs that will be removed on next concretize.

property repo

property repos_path

rm_view_from_env(env_mod: EnvironmentModifications, view: str) → EnvironmentModifications[source]

Collect the environment modifications to deactivate an environment using the provided view. Reverses the action of add_view_to_env.

Parameters:

env_mod – the environment modifications object that is modified.
view – the name of the view to deactivate.

roots()[source]

Specs explicitly requested by the user in this environment.

Yields both added and installed specs that have user specs in spack.yaml.

property scope_name: Name of the config scope of this environment’s manifest file.

spec_lists: Dict[str, SpecList]: Specs from “spack.yaml”

specs_by_hash: Dict[str, Spec]: Concretized specs by hash

property unify

uninstalled_specs()[source]: Return root specs that are not installed, or are installed, but are development specs themselves or have those among their dependencies.

update_default_view(path_or_bool: str | bool) → None[source]

Updates the path of the default view.

If the argument passed as input is False the default view is deleted, if present. The manifest will have an entry “view: false”.

If the argument passed as input is True a default view is created, if not already present. The manifest will have an entry “view: true”. If a default view is already declared, it will be left untouched.

If the argument passed as input is a path a default view pointing to that path is created, if not present already. If a default view is already declared, only its “root” will be changed.

Parameters:: path_or_bool – either True, or False or a path

update_environment_repository() → None[source]: Updates the repository associated with the environment.

update_lockfile() → None[source]

update_stale_references(from_list=None)[source]: Iterate over spec lists updating references.

property user_specs

property view_path_default

views: Dict[str, ViewDescriptor]

write(regenerate: bool = True) → None[source]

Writes an in-memory environment to its location on disk.

Write out package files for each newly concretized spec. Also regenerate any views associated with the environment and run post-write hooks, if regenerate is True.

Parameters:: regenerate – regenerate views and run post-write hooks as well as writing if True.

write_transaction()[source]: Get a write lock context manager for use in a with block.

class spack.environment.environment.EnvironmentManifestFile(manifest_dir: Path | str)[source]

Bases: Mapping

Manages the in-memory representation of a manifest file, and its synchronization with the actual manifest on disk.

add_definition(user_spec: str, list_name: str) → None[source]

Appends a user spec to the first active definition matching the name passed as argument.

Parameters:

user_spec – user spec to be appended
list_name – name of the definition where to append

Raises:

SpackEnvironmentError – is no valid definition exists already

add_user_spec(user_spec: str) → None[source]

Appends the user spec passed as input to the list of root specs.

Parameters:: user_spec – user spec to be appended

property configuration: Return the dictionaries in the YAML, without the top level attribute

deactivate_config_scope() → None[source]: Remove any of the manifest’s scopes from the global config path.

property env_config_scopes: List[ConfigScope]: A list of all configuration scopes for the environment manifest. On the first call this instantiates all the scopes, on subsequent calls it returns the cached list.

flush() → None[source]: Synchronizes the object with the manifest file on disk.

static from_lockfile(manifest_dir: Path | str) → EnvironmentManifestFile[source]

Returns an environment manifest file compatible with the lockfile already present in the environment directory.

This function also writes a spack.yaml file that is consistent with the spack.lock already existing in the directory.

Parameters:: manifest_dir – directory containing the manifest and lockfile

property included_config_scopes: List[ConfigScope]

List of included configuration scopes from the manifest.

Scopes are listed in the YAML file in order from highest to lowest precedence, so configuration from earlier scope will take precedence over later ones.

This routine returns them in the order they should be pushed onto the internal scope stack (so, in reverse, from lowest to highest).

Returns: Configuration scopes associated with the environment manifest

Raises:: SpackEnvironmentError – if the manifest includes a remote file but no configuration stage directory has been identified

override_definition(user_spec: str, *, override: str, list_name: str) → None[source]

Overrides a user spec from an active definition that matches the name passed as argument.

Parameters:

user_spec – user spec to be overridden
override – new spec to be used
list_name – name of the definition where to override the spec

Raises:

SpackEnvironmentError – if the user spec cannot be overridden

override_user_spec(user_spec: str, idx: int) → None[source]

Overrides the user spec at index idx with the one passed as input.

Parameters:

user_spec – new user spec
idx – index of the spec to be overridden

Raises:

SpackEnvironmentError – when the user spec cannot be overridden

prepare_config_scope() → None[source]: Add the manifest’s scopes to the global configuration search path.

property pristine_configuration: Return the dictionaries in the pristine YAML, without the top level attribute

pristine_yaml_content: Pristine YAML content, without defaults being added

remove_default_view() → None[source]: Removes the default view from the manifest file

remove_definition(user_spec: str, list_name: str) → None[source]

Removes a user spec from an active definition that matches the name passed as argument.

Parameters:

user_spec – user spec to be removed
list_name – name of the definition where to remove the spec from

Raises:

SpackEnvironmentError – if the user spec cannot be removed from the list, or the list does not exist

remove_user_spec(user_spec: str) → None[source]

Removes the user spec passed as input from the list of root specs

Parameters:: user_spec – user spec to be removed
Raises:: SpackEnvironmentError – when the user spec is not in the list

set_default_view(view: bool | str | Path | Dict[str, str]) → None[source]

Sets the default view root in the manifest to the value passed as input.

Parameters:: view – If the value is a string or a path, it specifies the path to the view. If True the default view is used for the environment, if False there’s no view.

use_config()[source]: Ensure only the manifest’s configuration scopes are global.

yaml_content: YAML content with defaults added by Spack, if they’re missing

exception spack.environment.environment.SpackEnvironmentConfigError(message, long_message=None)[source]

Bases: SpackEnvironmentError

Class for Spack environment-specific configuration errors.

exception spack.environment.environment.SpackEnvironmentError(message, long_message=None)[source]

Bases: SpackError

Superclass for all errors to do with Spack environments.

exception spack.environment.environment.SpackEnvironmentViewError(message, long_message=None)[source]

Bases: SpackEnvironmentError

Class for errors regarding view generation.

class spack.environment.environment.ViewDescriptor(base_path, root, projections={}, select=[], exclude=[], link='all', link_type='symlink')[source]

Bases: object

content_hash(specs)[source]

exclude_fn(spec)[source]

static from_dict(base_path, d)[source]

get_projection_for_spec(spec)[source]: Get projection for spec. This function does not require the view to exist on the filesystem.

regenerate(concrete_roots: List[Spec]) → None[source]

select_fn(spec)[source]

specs_for_view(concrete_roots: List[Spec]) → List[Spec][source]: Flatten the DAGs of the concrete roots, keep only unique, selected, and installed specs in topological order from root to leaf.

to_dict()[source]

update_root(new_path)[source]

view(new: str | None = None) → SimpleFilesystemView[source]

Returns a view object for the underlying view directory. This means that the self.root symlink is followed, and that the view has to exist on the filesystem (unless new). This function is useful when writing to the view.

Raise if new is None and there is no current view

Parameters:: new – If a string, create a FilesystemView rooted at that path. Default None. This should only be used to regenerate the view, and cannot be used to access specs.

spack.environment.environment.activate(env, use_env_repo=False)[source]

Activate an environment.

To activate an environment, we add its manifest’s configuration scope to the existing Spack configuration, and we set active to the current environment.

Parameters:

env (Environment) – the environment to activate
use_env_repo (bool) – use the packages exactly as they appear in the environment’s repository

spack.environment.environment.active(name)[source]: True if the named environment is active.

spack.environment.environment.active_environment() → Environment | None[source]: Returns the active environment when there is any

spack.environment.environment.all_environment_names()[source]: List the names of environments that currently exist.

spack.environment.environment.all_environments()[source]: Generator for all managed Environments.

Create a managed environment in Spack and returns it.

A managed environment is created in a root directory managed by this Spack instance, so that Spack can keep track of them.

Files with suffix .json or .lock are considered lockfiles. Files with any other name are considered manifest files.

Parameters:

name – name of the managed environment
init_file – either a lockfile, a manifest file, or None
with_view – whether a view should be maintained for the environment. If the value is a string, it specifies the path to the view
keep_relative – if True, develop paths are copied verbatim into the new environment file, otherwise they are made absolute

Create an environment in the directory passed as input and returns it.

Files with suffix .json or .lock are considered lockfiles. Files with any other name are considered manifest files.

Parameters:

root – directory where to create the environment.
init_file – either a lockfile, a manifest file, or None
with_view – whether a view should be maintained for the environment. If the value is a string, it specifies the path to the view
keep_relative – if True, develop paths are copied verbatim into the new environment file, otherwise they are made absolute

spack.environment.environment.deactivate()[source]: Undo any configuration or repo settings modified by activate().

spack.environment.environment.default_env_path = '/home/docs/checkouts/readthedocs.org/user_builds/spack/checkouts/latest/lib/spack/docs/_spack_root/var/spack/environments': default path where environments are stored in the spack tree

spack.environment.environment.default_manifest_yaml()[source]: default spack.yaml file to put in new environments

spack.environment.environment.display_specs(concretized_specs)[source]

Displays the list of specs returned by Environment.concretize().

Parameters:: concretized_specs (list) – list of specs returned by Environment.concretize()

spack.environment.environment.ensure_env_root_path_exists()[source]

spack.environment.environment.ensure_no_disallowed_env_config_mods(scopes: List[ConfigScope]) → None[source]

spack.environment.environment.env_root_path() → str[source]: Override default root path if the user specified it

spack.environment.environment.env_subdir_name = '.spack-env': Name of the directory where environments store repos, logs, views, configs

spack.environment.environment.env_subdir_path(manifest_dir: str | Path) → str[source]

Path to where the environment stores repos, logs, views, configs.

Parameters:: manifest_dir – directory containing the environment manifest file

Returns: directory the environment uses to manage its files

spack.environment.environment.environment_dir_from_name(name: str, exists_ok: bool = True) → str[source]

Returns the directory associated with a named environment.

Parameters:

name – name of the environment
exists_ok – if False, raise an error if the environment exists already

Raises:

SpackEnvironmentError – if exists_ok is False and the environment exists already

spack.environment.environment.environment_name(path: str | Path) → str[source]

Human-readable representation of the environment.

This is the path for directory environments, and just the name for managed environments.

spack.environment.environment.exists(name)[source]: Whether an environment with this name exists or not.

spack.environment.environment.initialize_environment_dir(environment_dir: str | Path, envfile: str | Path | None) → None[source]

Initialize an environment directory starting from an envfile.

Files with suffix .json or .lock are considered lockfiles. Files with any other name are considered manifest files.

Parameters:

environment_dir – directory where the environment should be placed
envfile – manifest file or lockfile used to initialize the environment

Raises:

SpackEnvironmentError – if the directory can’t be initialized

spack.environment.environment.installed_specs()[source]: Returns the specs of packages installed in the active environment or None if no packages are installed.

spack.environment.environment.is_env_dir(path)[source]: Whether a directory contains a spack environment.

spack.environment.environment.is_latest_format(manifest)[source]

Return False if the manifest file exists and is not in the latest schema format.

Parameters:: manifest (str) – manifest file to be analyzed

spack.environment.environment.lockfile_format_version = 5: version of the lockfile format. Must increase monotonically.

spack.environment.environment.lockfile_name = 'spack.lock': Name of the input yaml file for an environment

spack.environment.environment.make_repo_path(root)[source]: Make a RepoPath from the repo subdirectories in an environment.

spack.environment.environment.manifest_file(env_name_or_dir)[source]

Return the absolute path to a manifest file given the environment name or directory.

Parameters:: env_name_or_dir (str) – either the name of a valid environment or a directory where a manifest file resides
Raises:: AssertionError – if the environment is not found

spack.environment.environment.manifest_name = 'spack.yaml': Name of the input yaml file for an environment

spack.environment.environment.no_active_environment()[source]: Deactivate the active environment for the duration of the context. Has no effect when there is no active environment.

spack.environment.environment.read(name)[source]: Get an environment with the supplied name.

spack.environment.environment.root(name)[source]: Get the root directory for an environment by name.

spack.environment.environment.spack_env_var = 'SPACK_ENV': environment variable used to indicate the active environment

spack.environment.environment.spack_env_view_var = 'SPACK_ENV_VIEW': environment variable used to indicate the active environment view

spack.environment.environment.update_yaml(manifest, backup_file)[source]

Update a manifest file from an old format to the current one.

Parameters:

manifest (str) – path to a manifest file
backup_file (str) – file where to copy the original manifest

Returns:

True if the manifest was updated, False otherwise.

Raises:

AssertionError – in case anything goes wrong during the update

spack.environment.environment.valid_env_name(name)[source]

spack.environment.environment.valid_environment_name_re = '^\\w[\\w-]*$': regex for validating enviroment names

spack.environment.environment.validate_env_name(name)[source]

spack.environment.environment.yaml_equivalent(first, second) → bool[source]: Returns whether two spack yaml items are equivalent, including overrides

spack.environment.shell module

spack.environment.shell.activate(env: Environment, use_env_repo=False, view: str | None = 'default') → EnvironmentModifications[source]

Activate an environment and append environment modifications

To activate an environment, we add its configuration scope to the existing Spack configuration, and we set active to the current environment.

Parameters:

env – the environment to activate
use_env_repo – use the packages exactly as they appear in the environment’s repository
view – generate commands to add runtime environment variables for named view

Returns:

Environment variables modifications to activate environment.

Return type:

spack.util.environment.EnvironmentModifications

spack.environment.shell.activate_header(env, shell, prompt=None, view: str | None = None)[source]

spack.environment.shell.deactivate() → EnvironmentModifications[source]

Deactivate an environment and collect corresponding environment modifications.

Note: unloads the environment in its current state, not in the state it was: loaded in, meaning that specs that were removed from the spack environment after activation are not unloaded.

Returns:: Environment variables modifications to activate environment.

spack.environment.shell.deactivate_header(shell)[source]