spack package

spack.spack_version_info = (0, 20, 0, 'dev0')

(major, minor, micro, dev release) tuple

Subpackages

Submodules

spack.abi module

class spack.abi.ABI[source]

Bases: object

This class provides methods to test ABI compatibility between specs. The current implementation is rather rough and could be improved.

architecture_compatible(target, constraint)[source]

Return true if architecture of target spec is ABI compatible to the architecture of constraint spec. If either the target or constraint specs have no architecture, target is also defined as architecture ABI compatible to constraint.

compatible(target, constraint, **kwargs)[source]

Returns true if target spec is ABI compatible to constraint spec

compiler_compatible(parent, child, **kwargs)[source]

Return true if compilers for parent and child are ABI compatible.

spack.audit module

Classes and functions to register audit checks for various parts of Spack and run them on-demand.

To register a new class of sanity checks (e.g. sanity checks for compilers.yaml), the first action required is to create a new AuditClass object:

audit_cfgcmp = AuditClass(
    tag='CFG-COMPILER',
    description='Sanity checks on compilers.yaml',
    kwargs=()
)

This object is to be used as a decorator to register functions that will perform each a single check:

@audit_cfgcmp
def _search_duplicate_compilers(error_cls):
    pass

These functions need to take as argument the keywords declared when creating the decorator object plus an error_cls argument at the end, acting as a factory to create Error objects. It should return a (possibly empty) list of errors.

Calls to each of these functions are triggered by the run method of the decorator object, that will forward the keyword arguments passed as input.

class spack.audit.AuditClass(group, tag, description, kwargs)[source]

Bases: Sequence

run(**kwargs)[source]
spack.audit.CALLBACKS = {'CFG-COMPILER': <spack.audit.AuditClass object>, 'CFG-PACKAGES': <spack.audit.AuditClass object>, 'GENERIC': <spack.audit.AuditClass object>, 'PKG-ATTRIBUTES': <spack.audit.AuditClass object>, 'PKG-DIRECTIVES': <spack.audit.AuditClass object>, 'PKG-HTTPS-DIRECTIVES': <spack.audit.AuditClass object>, 'PKG-PROPERTIES': <spack.audit.AuditClass object>}

Map an audit tag to a list of callables implementing checks

class spack.audit.Error(summary, details)[source]

Bases: object

Information on an error reported in a test.

spack.audit.GROUPS = {'configs': ['CFG-COMPILER', 'CFG-PACKAGES'], 'generic': ['GENERIC'], 'packages': ['PKG-DIRECTIVES', 'PKG-ATTRIBUTES', 'PKG-PROPERTIES'], 'packages-https': ['PKG-HTTPS-DIRECTIVES']}

Map a group of checks to the list of related audit tags

spack.audit.config_compiler = <spack.audit.AuditClass object>

Sanity checks on compilers.yaml

spack.audit.config_packages = <spack.audit.AuditClass object>

Sanity checks on packages.yaml

spack.audit.generic = <spack.audit.AuditClass object>

Generic checks relying on global state

spack.audit.package_directives = <spack.audit.AuditClass object>

Sanity checks on package directives

spack.audit.run_check(tag, **kwargs)[source]

Run the checks associated with a single tag.

Parameters

  • tag (str) – tag of the check

  • **kwargs – keyword arguments forwarded to the checks

Returns

Errors occurred during the checks

spack.audit.run_group(group, **kwargs)[source]

Run the checks that are part of the group passed as argument.

Parameters

  • group (str) – group of checks to be run

  • **kwargs – keyword arguments forwarded to the checks

Returns

List of (tag, errors) that failed.

spack.binary_distribution module

class spack.binary_distribution.BinaryCacheIndex(cache_root)[source]

Bases: object

The BinaryCacheIndex tracks what specs are available on (usually remote) binary caches.

This index is “best effort”, in the sense that whenever we don’t find what we’re looking for here, we will attempt to fetch it directly from configured mirrors anyway. Thus, it has the potential to speed things up, but cache misses shouldn’t break any spack functionality.

At the moment, everything in this class is initialized as lazily as possible, so that it avoids slowing anything in spack down until absolutely necessary.

TODO: What’s the cost if, e.g., we realize in the middle of a spack install that the cache is out of date, and we fetch directly? Does it mean we should have paid the price to update the cache earlier?

clear()[source]

For testing purposes we need to be able to empty the cache and clear associated data structures.

find_built_spec(spec, mirrors_to_check=None)[source]

Look in our cache for the built spec corresponding to spec.

If the spec can be found among the configured binary mirrors, a list is returned that contains the concrete spec and the mirror url of each mirror where it can be found. Otherwise, None is returned.

This method does not trigger reading anything from remote mirrors, but rather just checks if the concrete spec is found within the cache.

The cache can be updated by calling update() on the cache.

Parameters

  • spec (spack.spec.Spec) – Concrete spec to find

  • mirrors_to_check – Optional mapping containing mirrors to check. If None, just assumes all configured mirrors.

Returns

An list of objects containing the found specs and mirror url where

each can be found, e.g.:

[
    {
        "spec": <concrete-spec>,
        "mirror_url": <mirror-root-url>
    }
]

find_by_hash(find_hash, mirrors_to_check=None)[source]

Same as find_built_spec but uses the hash of a spec.

Parameters

  • find_hash (str) – hash of the spec to search

  • mirrors_to_check – Optional mapping containing mirrors to check. If None, just assumes all configured mirrors.

get_all_built_specs()[source]
regenerate_spec_cache(clear_existing=False)[source]

Populate the local cache of concrete specs (_mirrors_for_spec) from the locally cached buildcache index files. This is essentially a no-op if it has already been done, as we keep track of the index hashes for which we have already associated the built specs.

update(with_cooldown=False)[source]

Make sure local cache of buildcache index files is up to date. If the same mirrors are configured as the last time this was called and none of the remote buildcache indices have changed, calling this method will only result in fetching the index hash from each mirror to confirm it is the same as what is stored locally. Otherwise, the buildcache index.json and index.json.hash files are retrieved from each configured mirror and stored locally (both in memory and on disk under _index_cache_root).

update_spec(spec, found_list)[source]

Take list of {‘mirror_url’: m, ‘spec’: s} objects and update the local built_spec_cache

class spack.binary_distribution.BinaryCacheQuery(all_architectures)[source]

Bases: object

Callable object to query if a spec is in a binary cache

class spack.binary_distribution.BuildManifestVisitor[source]

Bases: BaseDirectoryVisitor

Visitor that collects a list of files and symlinks that can be checked for need of relocation. It knows how to dedupe hardlinks and deal with symlinks to files and directories.

before_visit_dir(root, rel_path, depth)[source]

Return True from this function to recurse into the directory at os.path.join(root, rel_path). Return False in order not to recurse further.

Parameters

  • root (str) – root directory

  • rel_path (str) – relative path to current directory from root

  • depth (int) – depth of current directory from the root directory

Returns

True when the directory should be recursed into. False when not

Return type

bool

before_visit_symlinked_dir(root, rel_path, depth)[source]

Return True to recurse into the symlinked directory and False in order not to. Note: rel_path is the path to the symlink itself. Following symlinked directories blindly can cause infinite recursion due to cycles.

Parameters

  • root (str) – root directory

  • rel_path (str) – relative path to current symlink from root

  • depth (int) – depth of current symlink from the root directory

Returns

True when the directory should be recursed into. False when not

Return type

bool

seen_before(root, rel_path)[source]
visit_file(root, rel_path, depth)[source]

Handle the non-symlink file at os.path.join(root, rel_path)

Parameters

  • root (str) – root directory

  • rel_path (str) – relative path to current file from root

  • depth (int) – depth of current file from the root directory

visit_symlinked_file(root, rel_path, depth)[source]

Handle the symlink to a file at os.path.join(root, rel_path). Note: rel_path is the location of the symlink, not to what it is pointing to. The symlink may be dangling.

Parameters

  • root (str) – root directory

  • rel_path (str) – relative path to current symlink from root

  • depth (int) – depth of current symlink from the root directory

class spack.binary_distribution.DefaultIndexFetcher(url, local_hash, urlopen=<function _urlopen.<locals>.dispatch_open>)[source]

Bases: object

Fetcher for index.json, using separate index.json.hash as cache invalidation strategy

conditional_fetch()[source]
get_remote_hash()[source]
class spack.binary_distribution.EtagIndexFetcher(url, etag, urlopen=<function _urlopen.<locals>.dispatch_open>)[source]

Bases: object

Fetcher for index.json, using ETags headers as cache invalidation strategy

conditional_fetch()[source]
exception spack.binary_distribution.FetchCacheError(errors)[source]

Bases: Exception

Error thrown when fetching the cache failed, usually a composite error list.

exception spack.binary_distribution.FetchIndexError[source]

Bases: Exception

class spack.binary_distribution.FetchIndexResult(etag, hash, data, fresh)

Bases: tuple

property data

Alias for field number 2

property etag

Alias for field number 0

property fresh

Alias for field number 3

property hash

Alias for field number 1

exception spack.binary_distribution.ListMirrorSpecsError(message, long_message=None)[source]

Bases: SpackError

Raised when unable to retrieve list of specs from the mirror

exception spack.binary_distribution.NewLayoutException(msg)[source]

Bases: SpackError

Raised if directory layout is different from buildcache.

exception spack.binary_distribution.NoChecksumException(message, long_message=None)[source]

Bases: SpackError

Raised if file fails checksum verification.

exception spack.binary_distribution.NoGpgException(msg)[source]

Bases: SpackError

Raised when gpg2 is not in PATH

exception spack.binary_distribution.NoKeyException(msg)[source]

Bases: SpackError

Raised when gpg has no default key added.

exception spack.binary_distribution.NoOverwriteException(file_path)[source]

Bases: SpackError

Raised when a file exists and must be overwritten.

exception spack.binary_distribution.NoVerifyException(message, long_message=None)[source]

Bases: SpackError

Raised if file fails signature verification.

exception spack.binary_distribution.PickKeyException(keys)[source]

Bases: SpackError

Raised when multiple keys can be used to sign.

exception spack.binary_distribution.UnsignedPackageException(message, long_message=None)[source]

Bases: SpackError

Raised if installation of unsigned package is attempted without the use of --no-check-signature.

spack.binary_distribution.binary_index = <spack.binary_distribution.BinaryCacheIndex object>

Singleton binary_index instance

spack.binary_distribution.binary_index_location()[source]

Set up a BinaryCacheIndex for remote buildcache