spack.util package¶
Subpackages¶
Submodules¶
spack.util.compression module¶
-
spack.util.compression.
allowed_archive
(path)¶
-
spack.util.compression.
decompressor_for
(path, extension=None)¶ Get the appropriate decompressor for a path.
-
spack.util.compression.
extension
(path)¶ Get the archive extension for a path.
-
spack.util.compression.
strip_extension
(path)¶ Get the part of a path that does not include its compressed type extension.
spack.util.crypto module¶
-
class
spack.util.crypto.
Checker
(hexdigest, **kwargs)¶ ベースクラス:
object
A checker checks files against one particular hex digest. It will automatically determine what hashing algorithm to used based on the length of the digest it's initialized with. e.g., if the digest is 32 hex characters long this will use md5.
Example: know your tarball should hash to 'abc123'. You want to check files against this. You would use this class like so:
hexdigest = 'abc123' checker = Checker(hexdigest) success = checker.check('downloaded.tar.gz')
After the call to check, the actual checksum is available in checker.sum, in case it's needed for error output.
You can trade read performance and memory usage by adjusting the block_size optional arg. By default it's a 1MB (2**20 bytes) buffer.
-
check
(filename)¶ Read the file with the specified name and check its checksum against self.hexdigest. Return True if they match, False otherwise. Actual checksum is stored in self.sum.
-
hash_name
¶ Get the name of the hash function this Checker is using.
-
-
class
spack.util.crypto.
DeprecatedHash
(hash_alg, alert_fn, disable_security_check)¶ ベースクラス:
object
-
spack.util.crypto.
bit_length
(num)¶ Number of bits required to represent an integer in binary.
-
spack.util.crypto.
checksum
(hashlib_algo, filename, **kwargs)¶ Returns a hex digest of the filename generated using an algorithm from hashlib.
-
spack.util.crypto.
hash_algo_for_digest
(hexdigest)¶ Gets name of the hash algorithm for a hex digest.
-
spack.util.crypto.
hash_fun_for_algo
(algo)¶ Get a function that can perform the specified hash algorithm.
-
spack.util.crypto.
hash_fun_for_digest
(hexdigest)¶ Gets a hash function corresponding to a hex digest.
-
spack.util.crypto.
hashes
= {'md5': 16, 'sha1': 20, 'sha224': 28, 'sha256': 32, 'sha384': 48, 'sha512': 64}¶ Set of hash algorithms that Spack can use, mapped to digest size in bytes
-
spack.util.crypto.
prefix_bits
(byte_array, bits)¶ Return the first <bits> bits of a byte array as an integer.
spack.util.debug module¶
Debug signal handler: prints a stack trace and enters interpreter.
register_interrupt_handler()
enables a ctrl-C handler that prints
a stack trace and drops the user into an interpreter.
-
spack.util.debug.
debug_handler
(sig, frame)¶ Interrupt running process, and provide a python prompt for interactive debugging.
-
spack.util.debug.
register_interrupt_handler
()¶ Print traceback and enter an interpreter on Ctrl-C
spack.util.editor module¶
Module for finding the user's preferred text editor.
Defines one function, editor(), which invokes the editor defined by the user's VISUAL environment variable if set. We fall back to the editor defined by the EDITOR environment variable if VISUAL is not set or the specified editor fails (e.g. no DISPLAY for a graphical editor). If neither variable is set, we fall back to one of several common editors, raising an EnvironmentError if we are unable to find one.
-
spack.util.editor.
editor
(*args, **kwargs)¶ Invoke the user's editor.
This will try to execute the following, in order:
$VISUAL <args> # the "visual" editor (per POSIX)
$EDITOR <args> # the regular editor (per POSIX)
some default editor (see
_default_editors
) with <args>
If an environment variable isn't defined, it is skipped. If it points to something that can't be executed, we'll print a warning. And if we can't find anything that can be executed after searching the full list above, we'll raise an error.
- パラメータ
args (list of str) -- args to pass to editor
- Optional Arguments:
_exec_func (function): invoke this function instead of
os.execv()
spack.util.environment module¶
Utilities for setting and modifying environment variables.
-
class
spack.util.environment.
AppendFlagsEnv
(name, value, **kwargs)¶ ベースクラス:
spack.util.environment.NameValueModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
AppendPath
(name, value, **kwargs)¶ ベースクラス:
spack.util.environment.NameValueModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
DeprioritizeSystemPaths
(name, **kwargs)¶ ベースクラス:
spack.util.environment.NameModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
EnvironmentModifications
(other=None)¶ ベースクラス:
object
Keeps track of requests to modify the current environment.
Each call to a method to modify the environment stores the extra information on the caller in the request:
'filename' : filename of the module where the caller is defined
'lineno': line number where the request occurred
'context' : line of code that issued the request that failed
-
append_flags
(name, value, sep=' ', **kwargs)¶ Stores in the current object a request to append to an env variable
- パラメータ
name -- name of the environment variable to be appended to
value -- value to append to the environment variable
Appends with spaces separating different additions to the variable
-
append_path
(name, path, **kwargs)¶ Stores a request to append a path to a path list.
- パラメータ
name -- name of the path list in the environment
path -- path to be appended
-
apply_modifications
()¶ Applies the modifications and clears the list.
-
clear
()¶ Clears the current list of modifications
-
deprioritize_system_paths
(name, **kwargs)¶ Stores a request to deprioritize system paths in a path list, otherwise preserving the order.
- パラメータ
name -- name of the path list in the environment.
-
extend
(other)¶
-
static
from_sourcing_file
(filename, *args, **kwargs)¶ Returns modifications that would be made by sourcing a file.
- パラメータ
filename (str) -- The file to source
*args (list of str) -- Arguments to pass on the command line
- キーワード引数
shell (str) -- The shell to use (default:
bash
)shell_options (str) -- Options passed to the shell (default:
-c
)source_command (str) -- The command to run (default:
source
)suppress_output (str) -- Redirect used to suppress output of command (default:
&> /dev/null
)concatenate_on_success (str) -- Operator used to execute a command only when the previous command succeeds (default:
&&
)blacklist ([str or re]) -- Ignore any modifications of these variables (default: [])
whitelist ([str or re]) -- Always respect modifications of these variables (default: []). Has precedence over blacklist.
clean (bool) -- In addition to removing empty entries, also remove duplicate entries (default: False).
- 戻り値
- an object that, if executed, has
the same effect on the environment as sourcing the file
- 戻り値の型
EnvironmentModifications
-
group_by_name
()¶ Returns a dict of the modifications grouped by variable name.
- 戻り値
dict mapping the environment variable name to the modifications to be done on it
-
is_unset
(var_name)¶
-
prepend_path
(name, path, **kwargs)¶ Same as append_path, but the path is pre-pended.
- パラメータ
name -- name of the path list in the environment
path -- path to be pre-pended
-
prune_duplicate_paths
(name, **kwargs)¶ Stores a request to remove duplicates from a path list, otherwise preserving the order.
- パラメータ
name -- name of the path list in the environment.
-
remove_path
(name, path, **kwargs)¶ Stores a request to remove a path from a path list.
- パラメータ
name -- name of the path list in the environment
path -- path to be removed
-
set
(name, value, **kwargs)¶ Stores a request to set an environment variable.
- パラメータ
name -- name of the environment variable to be set
value -- value of the environment variable
-
set_path
(name, elements, **kwargs)¶ Stores a request to set a path generated from a list.
- パラメータ
name -- name o the environment variable to be set.
elements -- elements of the path to set.
-
shell_modifications
(shell='sh')¶ Return shell code to apply the modifications and clears the list.
-
unset
(name, **kwargs)¶ Stores a request to unset an environment variable.
- パラメータ
name -- name of the environment variable to be set
-
class
spack.util.environment.
NameValueModifier
(name, value, **kwargs)¶ ベースクラス:
object
-
update_args
(**kwargs)¶
-
-
class
spack.util.environment.
PrependPath
(name, value, **kwargs)¶ ベースクラス:
spack.util.environment.NameValueModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
PruneDuplicatePaths
(name, **kwargs)¶ ベースクラス:
spack.util.environment.NameModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
RemovePath
(name, value, **kwargs)¶ ベースクラス:
spack.util.environment.NameValueModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
SetEnv
(name, value, **kwargs)¶ ベースクラス:
spack.util.environment.NameValueModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
SetPath
(name, value, **kwargs)¶ ベースクラス:
spack.util.environment.NameValueModifier
-
execute
(env)¶
-
-
class
spack.util.environment.
UnsetEnv
(name, **kwargs)¶ ベースクラス:
spack.util.environment.NameModifier
-
execute
(env)¶
-
-
spack.util.environment.
concatenate_paths
(paths, separator=':')¶ Concatenates an iterable of paths into a string of paths separated by separator, defaulting to colon.
- パラメータ
paths -- iterable of paths
separator -- the separator to use, default ':'
- 戻り値
string
-
spack.util.environment.
deprioritize_system_paths
(paths)¶ Put system paths at the end of paths, otherwise preserving order.
-
spack.util.environment.
dump_environment
(path, environment=None)¶ Dump an environment dictionary to a source-able file.
-
spack.util.environment.
env_flag
(name)¶
-
spack.util.environment.
env_var_to_source_line
(var, val)¶
-
spack.util.environment.
filter_environment_blacklist
(env, variables)¶ Generator that filters out any change to environment variables present in the input list.
- パラメータ
env -- list of environment modifications
variables -- list of variable names to be filtered
- 戻り値
items in env if they are not in variables
-
spack.util.environment.
filter_system_paths
(paths)¶ Return only paths that are not system paths.
-
spack.util.environment.
get_path
(name)¶
-
spack.util.environment.
inspect_path
(root, inspections, exclude=None)¶ Inspects
root
to search for the subdirectories ininspections
. Adds every path found to a list of prepend-path commands and returns it.- パラメータ
root (str) -- absolute path where to search for subdirectories
inspections (dict) -- maps relative paths to a list of environment variables that will be modified if the path exists. The modifications are not performed immediately, but stored in a command object that is returned to client
exclude (callable) -- optional callable. If present it must accept an absolute path and return True if it should be excluded from the inspection
Examples:
The following lines execute an inspection in
/usr
to search for/usr/include
and/usr/lib64
. If found we want to prepend/usr/include
toCPATH
and/usr/lib64
toMY_LIB64_PATH
.# Set up the dictionary containing the inspection inspections = { 'include': ['CPATH'], 'lib64': ['MY_LIB64_PATH'] } # Get back the list of command needed to modify the environment env = inspect_path('/usr', inspections) # Eventually execute the commands env.apply_modifications()
- 戻り値
instance of EnvironmentModifications containing the requested modifications
-
spack.util.environment.
is_system_path
(path)¶ Predicate that given a path returns True if it is a system path, False otherwise.
- パラメータ
path (str) -- path to a directory
- 戻り値
True or False
-
spack.util.environment.
path_put_first
(var_name, directories)¶ Puts the provided directories first in the path, adding them if they're not already there.
-
spack.util.environment.
path_set
(var_name, directories)¶
-
spack.util.environment.
pickle_environment
(path, environment=None)¶ Pickle an environment dictionary to a file.
-
spack.util.environment.
preserve_environment
(*variables)¶ Ensures that the value of the environment variables passed as arguments is the same before entering to the context manager and after exiting it.
Variables that are unset before entering the context manager will be explicitly unset on exit.
- パラメータ
variables (list of str) -- list of environment variables to be preserved
-
spack.util.environment.
prune_duplicate_paths
(paths)¶ Returns the paths with duplicates removed, order preserved.
-
spack.util.environment.
set_env
(**kwargs)¶ Temporarily sets and restores environment variables.
Variables can be set as keyword arguments to this function.
-
spack.util.environment.
set_or_unset_not_first
(variable, changes, errstream)¶ Check if we are going to set or unset something after other modifications have already been requested.
-
spack.util.environment.
validate
(env, errstream)¶ Validates the environment modifications to check for the presence of suspicious patterns. Prompts a warning for everything that was found.
Current checks: - set or unset variables after other changes on the same variable
- パラメータ
env -- list of environment modifications
spack.util.executable module¶
-
class
spack.util.executable.
Executable
(name)¶ ベースクラス:
object
Class representing a program that can be run on the command line.
-
add_default_arg
(arg)¶ Add a default argument to the command.
-
add_default_env
(key, value)¶ Set an environment variable when the command is run.
- パラメータ
key -- The environment variable to set
value -- The value to set it to
-
command
¶ The command-line string.
- 戻り値
The executable and default arguments
- 戻り値の型
str
-
name
¶ The executable name.
- 戻り値
The basename of the executable
- 戻り値の型
str
-
path
¶ The path to the executable.
- 戻り値
The path to the executable
- 戻り値の型
str
-
-
spack.util.executable.
which
(*args, **kwargs)¶ Finds an executable in the path like command-line which.
If given multiple executables, returns the first one that is found. If no executables are found, returns None.
- パラメータ
*args (str) -- One or more executables to search for
- キーワード引数
path (
list()
or str) -- The path to search. Defaults toPATH
required (bool) -- If set to True, raise an error if executable not found
- 戻り値
The first executable that is found in the path
- 戻り値の型
Executable
-
exception
spack.util.executable.
ProcessError
(message, long_message=None)¶ ベースクラス:
spack.error.SpackError
ProcessErrors are raised when Executables exit with an error code.
spack.util.file_cache module¶
-
exception
spack.util.file_cache.
CacheError
(message, long_message=None)¶ ベースクラス:
spack.error.SpackError
-
class
spack.util.file_cache.
FileCache
(root, timeout=120)¶ ベースクラス:
object
This class manages cached data in the filesystem.
Cache files are fetched and stored by unique keys. Keys can be relative paths, so that there can be some hierarchy in the cache.
The FileCache handles locking cache files for reading and writing, so client code need not manage locks for cache entries.
-
cache_path
(key)¶ Path to the file in the cache for a particular key.
-
destroy
()¶ Remove all files under the cache root.
-
init_entry
(key)¶ Ensure we can access a cache file. Create a lock for it if needed.
Return whether the cache file exists yet or not.
-
mtime
(key)¶ Return modification time of cache file, or 0 if it does not exist.
Time is in units returned by os.stat in the mtime field, which is platform-dependent.
-
read_transaction
(key)¶ Get a read transaction on a file cache item.
Returns a ReadTransaction context manager and opens the cache file for reading. You can use it like this:
- with file_cache_object.read_transaction(key) as cache_file:
cache_file.read()
-
remove
(key)¶
-
write_transaction
(key)¶ Get a write transaction on a file cache item.
Returns a WriteTransaction context manager that opens a temporary file for writing. Once the context manager finishes, if nothing went wrong, moves the file into place on top of the old file atomically.
spack.util.gpg module¶
-
class
spack.util.gpg.
Gpg
¶ ベースクラス:
object
-
classmethod
create
(**kwargs)¶
-
classmethod
export_keys
(location, *keys)¶
-
static
gpg
()¶
-
classmethod
list
(trusted, signing)¶
-
classmethod
sign
(key, file, output, clearsign=False)¶
-
classmethod
signing_keys
()¶
-
classmethod
trust
(keyfile)¶
-
classmethod
untrust
(signing, *keys)¶
-
classmethod
verify
(signature, file)¶
-
classmethod
spack.util.lock module¶
Wrapper for llnl.util.lock
allows locking to be enabled/disabled.
-
class
spack.util.lock.
Lock
(*args, **kwargs)¶ ベースクラス:
llnl.util.lock.Lock
Lock that can be disabled.
This overrides the
_lock()
and_unlock()
methods fromllnl.util.lock
so that all the lock API calls will succeed, but the actual locking mechanism can be disabled via_enable_locks
.
-
spack.util.lock.
check_lock_safety
(path)¶ Do some extra checks to ensure disabling locks is safe.
This will raise an error if
path
can is group- or world-writable AND the current user can write to the directory (i.e., if this user AND others could write to the path).This is intended to run on the Spack prefix, but can be run on any path for testing.
spack.util.log_parse module¶
-
spack.util.log_parse.
parse_log_events
(stream, context=6, jobs=None, profile=False)¶ Extract interesting events from a log file as a list of LogEvent.
- パラメータ
stream (str or fileobject) -- build log name or file object
context (int) -- lines of context to extract around each log event
jobs (int) -- number of jobs to parse with; default ncpus
profile (bool) -- print out profile information for parsing
- 戻り値
- two lists containig
BuildError
and BuildWarning
objects.
- two lists containig
- 戻り値の型
(tuple)
This is a wrapper around
ctest_log_parser.CTestLogParser
that lazily constructs a singleCTestLogParser
object. This ensures that all the regex compilation is only done once.
-
spack.util.log_parse.
make_log_context
(log_events, width=None)¶ Get error context from a log file.
- パラメータ
log_events (list of LogEvent) -- list of events created by
ctest_log_parser.parse()
width (int or None) -- wrap width;
0
for no limit;None
to auto-size for terminal
- 戻り値
context from the build log with errors highlighted
- 戻り値の型
str
Parses the log file for lines containing errors, and prints them out with line numbers and context. Errors are highlighted with '>>' and with red highlighting (if color is enabled).
Events are sorted by line number before they are displayed.
spack.util.module_cmd module¶
This module contains routines related to the module command for accessing and parsing environment modules.
-
spack.util.module_cmd.
get_path_arg_from_module_line
(line)¶
-
spack.util.module_cmd.
get_path_from_module
(mod)¶ Inspects a TCL module for entries that indicate the absolute path at which the library supported by said module can be found.
-
spack.util.module_cmd.
get_path_from_module_contents
(text, module_name)¶
-
spack.util.module_cmd.
load_module
(mod)¶ Takes a module name and removes modules until it is possible to load that module. It then loads the provided module. Depends on the modulecmd implementation of modules used in cray and lmod.
-
spack.util.module_cmd.
module
(*args)¶
spack.util.naming module¶
-
spack.util.naming.
mod_to_class
(mod_name)¶ Convert a name from module style to class name style. Spack mostly follows PEP-8:
Module and package names use lowercase_with_underscores.
Class names use the CapWords convention.
Regular source code follows these convetions. Spack is a bit more liberal with its Package names and Compiler names:
They can contain '-' as well as '_', but cannot start with '-'.
They can start with numbers, e.g. "3proxy".
This function converts from the module convention to the class convention by removing _ and - and converting surrounding lowercase text to CapWords. If mod_name starts with a number, the class name returned will be prepended with '_' to make a valid Python identifier.
-
spack.util.naming.
spack_module_to_python_module
(mod_name)¶ Given a Spack module name, returns the name by which it can be imported in Python.
-
spack.util.naming.
valid_module_name
(mod_name)¶ Return whether mod_name is valid for use in Spack.
-
spack.util.naming.
valid_fully_qualified_module_name
(mod_name)¶ Return whether mod_name is a valid namespaced module name.
-
spack.util.naming.
validate_fully_qualified_module_name
(mod_name)¶ Raise an exception if mod_name is not a valid namespaced module name.
-
spack.util.naming.
validate_module_name
(mod_name)¶ Raise an exception if mod_name is not valid.
-
spack.util.naming.
possible_spack_module_names
(python_mod_name)¶ Given a Python module name, return a list of all possible spack module names that could correspond to it.
-
spack.util.naming.
simplify_name
(name)¶ Simplify package name to only lowercase, digits, and dashes.
Simplifies a name which may include uppercase letters, periods, underscores, and pluses. In general, we want our package names to only contain lowercase letters, digits, and dashes.
- パラメータ
name (str) -- The original name of the package
- 戻り値
The new name of the package
- 戻り値の型
str
-
class
spack.util.naming.
NamespaceTrie
(separator='.')¶ ベースクラス:
object
-
class
Element
(value)¶ ベースクラス:
object
-
has_value
(namespace)¶ True if there is a value set for the given namespace.
-
is_leaf
(namespace)¶ True if this namespace has no children in the trie.
-
is_prefix
(namespace)¶ True if the namespace has a value, or if it's the prefix of one that does.
-
class
spack.util.package_hash module¶
-
exception
spack.util.package_hash.
PackageHashError
(message, long_message=None)¶ ベースクラス:
spack.error.SpackError
Raised for all errors encountered during package hashing.
-
class
spack.util.package_hash.
RemoveDirectives
(spec)¶ ベースクラス:
ast.NodeTransformer
Remove Spack directives from a package AST.
-
is_directive
(node)¶
-
is_spack_attr
(node)¶
-
visit_ClassDef
(node)¶
-
-
class
spack.util.package_hash.
RemoveDocstrings
¶ ベースクラス:
ast.NodeTransformer
Transformer that removes docstrings from a Python AST.
-
remove_docstring
(node)¶
-
visit_ClassDef
(node)¶
-
visit_FunctionDef
(node)¶
-
visit_Module
(node)¶
-
-
class
spack.util.package_hash.
ResolveMultiMethods
(methods)¶ ベースクラス:
ast.NodeTransformer
Remove methods which do not exist if their @when is not satisfied.
-
resolve
(node)¶
-
visit_FunctionDef
(node)¶
-
-
class
spack.util.package_hash.
TagMultiMethods
(spec)¶ ベースクラス:
ast.NodeVisitor
Tag @when-decorated methods in a spec.
-
visit_FunctionDef
(node)¶
-
-
spack.util.package_hash.
package_ast
(spec)¶
-
spack.util.package_hash.
package_content
(spec)¶
-
spack.util.package_hash.
package_hash
(spec, content=None)¶
spack.util.path module¶
Utilities for managing paths in Spack.
TODO: this is really part of spack.config. Consolidate it.
-
spack.util.path.
substitute_config_variables
(path)¶ Substitute placeholders into paths.
Spack allows paths in configs to have some placeholders, as follows:
$spack The Spack instance's prefix
$user The current user's username
$tempdir Default temporary directory returned by tempfile.gettempdir()
These are substituted case-insensitively into the path, and users can use either
$var
or${var}
syntax for the variables.
-
spack.util.path.
canonicalize_path
(path)¶ Substitute config vars, expand environment vars, expand user home, take abspath.
spack.util.pattern module¶
-
class
spack.util.pattern.
Args
(*flags, **kwargs)¶ ベースクラス:
spack.util.pattern.Bunch
Subclass of Bunch to write argparse args more naturally.
-
class
spack.util.pattern.
Bunch
(**kwargs)¶ ベースクラス:
object
Carries a bunch of named attributes (from Alex Martelli bunch)
-
spack.util.pattern.
composite
(interface=None, method_list=None, container=<class 'list'>)¶ Decorator implementing the GoF composite pattern.
- パラメータ
interface (type) -- class exposing the interface to which the composite object must conform. Only non-private and non-special methods will be taken into account
method_list (list of str) -- names of methods that should be part of the composite
container (MutableSequence) -- container for the composite object (default = list). Must fulfill the MutableSequence contract. The composite class will expose the container API to manage object composition
- 戻り値
a class decorator that patches a class adding all the methods it needs to be a composite for a given interface.
spack.util.prefix module¶
This file contains utilities for managing the installation prefix of a package.
-
class
spack.util.prefix.
Prefix
¶ ベースクラス:
str
This class represents an installation prefix, but provides useful attributes for referring to directories inside the prefix.
Attributes of this object are created on the fly when you request them, so any of the following is valid:
>>> prefix = Prefix('/usr') >>> prefix.bin /usr/bin >>> prefix.lib64 /usr/lib64 >>> prefix.share.man /usr/share/man >>> prefix.foo.bar.baz /usr/foo/bar/baz >>> prefix.join('dashed-directory').bin64 /usr/dashed-directory/bin64
Prefix objects behave identically to strings. In fact, they subclass
str
. So operators like+
are legal:print('foobar ' + prefix)
This prints
foobar /usr
. All of this is meant to make custom installs easy.-
join
(string)¶ Concatenates a string to a prefix.
- パラメータ
string (str) -- the string to append to the prefix
- 戻り値
the newly created installation prefix
- 戻り値の型
Prefix
-
spack.util.spack_json module¶
Simple wrapper around JSON to guarantee consistent use of load/dump.
-
spack.util.spack_json.
load
(stream)¶ Spack JSON needs to be ordered to support specs.
-
spack.util.spack_json.
dump
(data, stream=None)¶ Dump JSON with a reasonable amount of indentation and separation.
-
exception
spack.util.spack_json.
SpackJSONError
(msg, json_error)¶ ベースクラス:
spack.error.SpackError
Raised when there are issues with JSON parsing.
spack.util.spack_yaml module¶
Enhanced YAML parsing for Spack.
load()
preserves YAML Marks on returned objects -- this allows us to access file and line information later.Our load methods use ``OrderedDict
class instead of YAML's default unorderd dict.
-
spack.util.spack_yaml.
load
(*args, **kwargs)¶ Load but modify the loader instance so that it will add __line__ atrributes to the returned object.
-
spack.util.spack_yaml.
dump
(*args, **kwargs)¶
-
exception
spack.util.spack_yaml.
SpackYAMLError
(msg, yaml_error)¶ ベースクラス:
spack.error.SpackError
Raised when there are issues with YAML parsing.
spack.util.string module¶
-
spack.util.string.
comma_and
(sequence)¶
-
spack.util.string.
comma_list
(sequence, article='')¶
-
spack.util.string.
comma_or
(sequence)¶
-
spack.util.string.
plural
(n, singular, plural=None, show_n=True)¶ Pluralize <singular> word by adding an s if n != 1.
- パラメータ
n (int) -- number of things there are
singular (str) -- singular form of word
plural (str, optional) -- optional plural form, for when it's not just singular + 's'
show_n (bool) -- whether to include n in the result string (default True)
- 戻り値
"1 thing" if n == 1 or "n things" if n != 1
- 戻り値の型
(str)
-
spack.util.string.
quote
(sequence, q="'")¶
spack.util.web module¶
-
exception
spack.util.web.
HTMLParseError
¶ ベースクラス:
Exception
-
class
spack.util.web.
LinkParser
¶ ベースクラス:
html.parser.HTMLParser
This parser just takes an HTML page and strips out the hrefs on the links. Good enough for a really simple spider.
-
handle_starttag
(tag, attrs)¶
-
-
exception
spack.util.web.
NoNetworkConnectionError
(message, url)¶ ベースクラス:
spack.util.web.SpackWebError
Raised when an operation can't get an internet connection.
-
class
spack.util.web.
NonDaemonContext
¶ ベースクラス:
multiprocessing.context.ForkContext
-
Process
¶ NonDaemonProcess
のエイリアス
-
-
class
spack.util.web.
NonDaemonPool
(*args, **kwargs)¶ ベースクラス:
multiprocessing.pool.Pool
Pool that uses non-daemon processes
-
class
spack.util.web.
NonDaemonProcess
(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)¶ ベースクラス:
multiprocessing.context.Process
Process tha allows sub-processes, so pools can have sub-pools.
-
daemon
¶ Return whether process is a daemon
-
-
exception
spack.util.web.
SpackWebError
(message, long_message=None)¶ ベースクラス:
spack.error.SpackError
Superclass for Spack web spidering errors.
-
exception
spack.util.web.
VersionFetchError
(message, long_message=None)¶ ベースクラス:
spack.util.web.SpackWebError
Raised when we can't determine a URL to fetch a package.
-
spack.util.web.
find_versions_of_archive
(archive_urls, list_url=None, list_depth=0)¶ Scrape web pages for new versions of a tarball.
- パラメータ
archive_urls -- URL or sequence of URLs for different versions of a package. Typically these are just the tarballs from the package file itself. By default, this searches the parent directories of archives.
- キーワード引数
list_url -- URL for a listing of archives. Spack wills scrape these pages for download links that look like the archive URL.
list_depth -- Max depth to follow links on list_url pages. Default 0.
-
spack.util.web.
get_checksums_for_versions
(url_dict, name, first_stage_function=None, keep_stage=False)¶ Fetches and checksums archives from URLs.
This function is called by both
spack checksum
andspack create
. Thefirst_stage_function
argument allows the caller to inspect the first downloaded archive, e.g., to determine the build system.- パラメータ
url_dict (dict) -- A dictionary of the form: version -> URL
name (str) -- The name of the package
first_stage_function (callable) -- function that takes a Stage and a URL; this is run on the stage of the first URL downloaded
keep_stage (bool) -- whether to keep staging area when command completes
- 戻り値
A multi-line string containing versions and corresponding hashes
- 戻り値の型
(str)
-
spack.util.web.
read_from_url
(url, accept_content_type=None)¶
-
spack.util.web.
spider
(root_url, depth=0)¶ Gets web pages from a root URL.
If depth is specified (e.g., depth=2), then this will also follow up to <depth> levels of links from the root.
This will spawn processes to fetch the children, for much improved performance over a sequential fetch.