spack package

spack.get_short_version() → str[source]

Short Spack version.

spack.get_spack_commit() → str | None[source]

Get the Spack git commit sha.

Returns:

(str or None) the commit sha if available, otherwise None

spack.get_version() → str[source]

Get a descriptive version of this instance of Spack.

Outputs "<PEP440 version> (<git commit sha>)".

The commit sha is only added when available.

spack.min_package_api_version

The minimum Package API version that this version of Spack is compatible with. This should always be a tuple of the form (major, 0), since compatibility with vX.Y implies compatibility with vX.0.

spack.package_api_version

The current Package API version implemented by this version of Spack. The Package API defines the Python interface for packages as well as the layout of package repositories. The minor version is incremented when the package API is extended in a backwards-compatible way. The major version is incremented upon breaking changes. This version is changed independently from the Spack version.

spack.spack_version_info

(major, minor, micro, dev release) tuple

Subpackages

Submodules

spack.aliases module

Alias names to convert legacy compilers to builtin packages and vice-versa

spack.archspec module

Adapter for the archspec library.

spack.archspec.HOST_TARGET_FAMILY

The host target family, like x86_64 or aarch64

spack.archspec.microarchitecture_flags(spec: Spec, language: str) → str[source]

Get compiler flags for the spec’s microarchitecture.

Parameters:

  • spec – The spec defining the target microarchitecture and compiler.

  • language – The language ("c", "cxx", "fortran") used to select the appropriate compiler from the spec.

Example:

>>> spec.format("{target}")
'm1'
>>> spec["c"].format("{name}{@version}")
'apple-clang@17.0.0'
>>> microarchitecture_flags(spec, language="c")
'-mcpu=apple-m1'
spack.archspec.microarchitecture_flags_from_target(target: Microarchitecture, compiler: Spec) → str[source]

Get compiler flags for the spec’s microarchitecture. Similar to microarchitecture_flags(), but takes a target and compiler directly instead of a spec.

Parameters:

  • target – The target microarchitecture.

  • compiler – The spec defining the compiler.

spack.audit module

Classes and functions to register audit checks for various parts of Spack and run them on-demand.

To register a new class of sanity checks (e.g. sanity checks for compilers.yaml), the first action required is to create a new AuditClass object:

audit_cfgcmp = AuditClass(
    tag="CFG-COMPILER",
    description="Sanity checks on compilers.yaml",
    kwargs=()
)

This object is to be used as a decorator to register functions that will perform each a single check:

@audit_cfgcmp
def _search_duplicate_compilers(error_cls):
    pass

These functions need to take as argument the keywords declared when creating the decorator object plus an error_cls argument at the end, acting as a factory to create Error objects. It should return a (possibly empty) list of errors.

Calls to each of these functions are triggered by the run method of the decorator object, that will forward the keyword arguments passed as input.

class spack.audit.AuditClass(group, tag, description, kwargs)[source]

Bases: Sequence

run(**kwargs)[source]
spack.audit.CALLBACKS

Map an audit tag to a list of callables implementing checks

class spack.audit.DeprecatedMagicGlobals(magic_globals: Iterable[str])[source]

Bases: NodeVisitor

depth: int
descend_in_function_def(node: AST) → None[source]
generic_visit(node: AST) → None[source]

Called if no explicit visitor function exists for a node.

in_function: bool
locals: Set[str]
magic_globals: Set[str]
references_to_globals: List[Tuple[str, int]]
class spack.audit.Error(summary, details)[source]

Bases: object

Information on an error reported in a test.

spack.audit.GROUPS

Map a group of checks to the list of related audit tags

spack.audit.config_compiler

Sanity checks on compilers.yaml

spack.audit.config_packages

Sanity checks on packages.yaml

spack.audit.config_repos

Sanity checks on packages.yaml

spack.audit.external_detection

Sanity checks on package directives

spack.audit.generic

Generic checks relying on global state

spack.audit.package_directives

Sanity checks on package directives

spack.audit.packages_with_detection_tests()[source]

Return the list of packages with a corresponding detection_test.yaml file.

spack.audit.run_check(tag, **kwargs)[source]

Run the checks associated with a single tag.

Parameters:

  • tag (str) – tag of the check

  • **kwargs – keyword arguments forwarded to the checks

Returns:

Errors occurred during the checks

spack.audit.run_group(group, **kwargs)[source]

Run the checks that are part of the group passed as argument.

Parameters:

  • group (str) – group of checks to be run

  • **kwargs – keyword arguments forwarded to the checks

Returns:

List of (tag, errors) that failed.

spack.binary_distribution module

spack.binary_distribution.BINARY_INDEX

Default binary cache index instance

class spack.binary_distribution.BinaryCacheQuery(all_architectures)[source]

Bases: object

Callable object to query if a spec is in a binary cache

class spack.binary_distribution.BinaryIndexCache(cache_root: str | None = None)[source]

Bases: object

The BinaryIndexCache tracks what specs are available on (usually remote) binary caches.

This index is “best effort”, in the sense that whenever we don’t find what we’re looking for here, we will attempt to fetch it directly from configured mirrors anyway. Thus, it has the potential to speed things up, but cache misses shouldn’t break any spack functionality.

At the moment, everything in this class is initialized as lazily as possible, so that it avoids slowing anything in spack down until absolutely necessary.

find_built_spec(spec: Spec) → List[MirrorMetadata][source]

Returns a list of MirrorMetadata objects indicating which mirrors have the given concrete spec.

This method does not trigger reading anything from remote mirrors, but rather just checks if the concrete spec is found within the cache.

The cache can be updated by calling update() on the cache.

Parameters:

spec – Concrete spec to find

find_by_hash(dag_hash: str) → List[MirrorMetadata][source]

Same as find_built_spec but uses the hash of a spec.

Parameters:

dag_hash – hash of the spec to search

get_all_built_specs() → List[Spec][source]

Returns a list of all concrete specs known to be available in a binary cache.

mirrors_without_index: Set[str]

URLs of binary mirrors that had no buildcache index during the last update()

regenerate_spec_cache(clear_existing=False)[source]

Populate the local cache of concrete specs (_mirrors_for_spec) from the locally cached buildcache index files. This is essentially a no-op if it has already been done, as we keep track of the index hashes for which we have already associated the built specs.

update(with_cooldown: bool = False) → None[source]

Make sure local cache of buildcache index files is up to date. If the same mirrors are configured as the last time this was called and none of the remote buildcache indices have changed, calling this method will only result in fetching the index hash from each mirror to confirm it is the same as what is stored locally. Otherwise, the buildcache index.json and index.json.hash files are retrieved from each configured mirror and stored locally (both in memory and on disk under _index_cache_root).

update_spec(spec: Spec, found_list: List[MirrorMetadata]) → None[source]

Update the cache with a new list of mirrors for a given spec.

class spack.binary_distribution.BuildCacheDatabase(root)[source]

Bases: Database

A database for binary buildcaches.

A database supports writing buildcache index files, in which case certain fields are not needed in each install record, and no locking is required. To use this feature, it provides lock_cfg=NO_LOCK, and override the list of record_fields.

record_fields: Tuple[str, ...]

Fields written for each install record

exception spack.binary_distribution.BuildcacheIndexError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when a buildcache cannot be read for any reason

exception spack.binary_distribution.BuildcacheIndexNotExists[source]

Bases: Exception

Buildcache does not contain an index

exception spack.binary_distribution.CannotListKeys(message: str, long_message: str | None = None)[source]

Bases: GenerateIndexError

Raised when unable to list keys when generating key index

class spack.binary_distribution.DefaultIndexHandler(mirror_metadata: MirrorMetadata, local_hash, urlopen=web_util.urlopen)[source]

Bases: IndexHandler

Fetcher for buildcache index, cache invalidation via manifest contents

conditional_fetch() → FetchIndexResult[source]
class spack.binary_distribution.DefaultIndexHandlerV2(mirror_metadata, local_hash, urlopen=web_util.urlopen)[source]

Bases: IndexHandler

Fetcher for index.json, using separate index.json.hash as cache invalidation strategy

conditional_fetch() → FetchIndexResult[source]
get_remote_hash()[source]
class spack.binary_distribution.EtagIndexHandler(mirror_metadata: MirrorMetadata, etag, urlopen=web_util.urlopen)[source]

Bases: IndexHandler

Fetcher for buildcache index, cache invalidation via ETags headers

This class differs from the DefaultIndexHandler in the following ways:

1. It is provided with an etag value on creation, rather than an index checksum value. Note that since we never start out with an etag, the default fetcher must have been used initially and determined that the etag approach is valid. 2. It provides this etag value in the If-None-Match request header for the index manifest. 3. It checks for special exception type and response code indicating the index manifest is not modified, exiting early and returning Fresh, if encountered. 4. If it needs to actually read the manifest, it does not need to do any checks of the url scheme to determine whether an etag should be included in the return value.

conditional_fetch() → FetchIndexResult[source]
class spack.binary_distribution.EtagIndexHandlerV2(mirror_metadata, etag, urlopen=web_util.urlopen)[source]

Bases: IndexHandler

Fetcher for index.json, using ETags headers as cache invalidation strategy

conditional_fetch() → FetchIndexResult[source]
class spack.binary_distribution.FancyProgress(total: int)[source]

Bases: object

fail() → None[source]
ok(msg: str | None = None) → None[source]
pretty_spec: str
start(spec: Spec, running: bool) → None[source]
exception spack.binary_distribution.FetchCacheError(errors)[source]

Bases: Exception

Error thrown when fetching the cache failed, usually a composite error list.

exception spack.binary_distribution.FetchIndexError[source]

Bases: Exception

class spack.binary_distribution.FetchIndexResult(etag, hash, data, fresh)

Bases: tuple

data

Alias for field number 2

etag

Alias for field number 0

fresh

Alias for field number 3

hash

Alias for field number 1

class spack.binary_distribution.FileTypes[source]

Bases: object

BINARY
TEXT
UNKNOWN
exception spack.binary_distribution.GenerateIndexError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when unable to generate key or package index for mirror

class spack.binary_distribution.IndexHandler[source]

Bases: object

conditional_fetch() → FetchIndexResult[source]
fetch_index_blob(cache_entry: URLBuildcacheEntry, blob_record: BlobRecord) → Tuple[str, str][source]

Fetch the index blob indicated by the BlobRecord, and return the (checksum, contents) of the blob

get_index_manifest(manifest_response) → BlobRecord[source]

Read the response of the manifest request and return a BlobRecord

exception spack.binary_distribution.NewLayoutException(msg)[source]

Bases: SpackError

Raised if directory layout is different from buildcache.

exception spack.binary_distribution.NoConfiguredBinaryMirrors[source]

Bases: SpackError

Raised when no binary mirrors are configured but an operation requires them

exception spack.binary_distribution.NoGpgException(msg)[source]

Bases: SpackError

Raised when gpg2 is not in PATH

exception spack.binary_distribution.NoKeyException(msg)[source]

Bases: SpackError

Raised when gpg has no default key added.

exception spack.binary_distribution.NoOverwriteException(file_path)[source]

Bases: SpackError

Raised when a file would be overwritten

class spack.binary_distribution.OCIIndexHandler(mirror_metadata: MirrorMetadata, local_hash, urlopen=None)[source]

Bases: IndexHandler

conditional_fetch() → FetchIndexResult[source]

Download an index from an OCI registry type mirror.

class spack.binary_distribution.OCIUploader(mirror: Mirror, force: bool, update_index: bool, base_image: str | None)[source]

Bases: Uploader

push(specs: List[Spec]) → Tuple[List[Spec], List[Tuple[Spec, BaseException]]][source]
tag(tag: str, roots: List[Spec])[source]

Make a list of selected specs together available under the given tag

exception spack.binary_distribution.PickKeyException(keys)[source]

Bases: SpackError

Raised when multiple keys can be used to sign.

exception spack.binary_distribution.PushToBuildCacheError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when unable to push objects to binary mirror

class spack.binary_distribution.URLUploader(mirror: Mirror, force: bool, update_index: bool, signing_key: str | None)[source]

Bases: Uploader

push(specs: List[Spec]) → Tuple[List[Spec], List[Tuple[Spec, BaseException]]][source]
exception spack.binary_distribution.UnsignedPackageException(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised if installation of unsigned package is attempted without the use of --no-check-signature.

class spack.binary_distribution.Uploader(mirror: Mirror, force: bool, update_index: bool)[source]

Bases: object

executor: Executor
push(specs: List[Spec]) → Tuple[List[Spec], List[Tuple[Spec, BaseException]]][source]
push_or_raise(specs: List[Spec]) → List[Spec][source]
tag(tag: str, roots: List[Spec])[source]

Make a list of selected specs together available under the given tag

tmpdir: str
spack.binary_distribution.binary_index_location()[source]

Set up a BinaryIndexCache for remote buildcache dbs in the user’s homedir.

spack.binary_distribution.buildcache_relative_blobs_path(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_blobs_url(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_index_path(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_index_url(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_keys_path(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_keys_url(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_specs_path(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildcache_relative_specs_url(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]
spack.binary_distribution.buildinfo_file_name(prefix)[source]

Filename of the binary package meta-data file

spack.binary_distribution.check_specs_against_mirrors(mirrors, specs, output_file=None)[source]

Check all the given specs against buildcaches on the given mirrors and determine if any of the specs need to be rebuilt. Specs need to be rebuilt when their hash doesn’t exist in the mirror.

Parameters:

  • mirrors (dict) – Mirrors to check against

  • specs (Iterable) – Specs to check against mirrors

  • output_file (str) – Path to output file to be written. If provided, mirrors with missing or out-of-date specs will be formatted as a JSON object and written to this file.

Returns: 1 if any spec was out-of-date on any mirror, 0 otherwise.

spack.binary_distribution.compute_hash(data)[source]
spack.binary_distribution.create_tarball(spec: Spec, tarfile_path: str) → Tuple[str, str][source]

Create a tarball of a spec and return the checksums of the compressed tarfile and the uncompressed tarfile.

Updates a buildinfo dict for old archives that did not dedupe hardlinks. De-duping hardlinks is necessary when relocating files in parallel and in-place. This means we must preserve inodes when relocating.

spack.binary_distribution.default_index_tag

Default OCI index tag

spack.binary_distribution.download_single_spec(concrete_spec, destination, mirror_url=None, layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION)[source]

Download the buildcache files for a single concrete spec.

Parameters:

  • concrete_spec – concrete spec to be downloaded

  • destination (str) – path where to put the downloaded buildcache

  • mirror_url (str) – url of the mirror from which to download

spack.binary_distribution.download_tarball(spec: Spec, unsigned: bool | None = False, mirrors_for_spec: List[MirrorMetadata] | None = None) → Stage | None[source]

Download binary tarball for given package

Parameters:

  • spec – a concrete spec

  • unsigned – if True or False override the mirror signature verification defaults

  • mirrors_for_spec – Optional list of mirrors known to have the spec. These will be checked in order first before looking in other configured mirrors.

Returns:

None if the tarball could not be downloaded, the signature verified (if required), and its checksum validated. Otherwise, return the stage containing the downloaded tarball.

spack.binary_distribution.extract_buildcache_tarball(tarfile_path: str, destination: str) → None[source]
spack.binary_distribution.extract_tarball(spec, tarball_stage: Stage, force=False, timer=timer.NULL_TIMER)[source]

extract binary tarball for given package into install area

spack.binary_distribution.file_matches(f: IO[bytes], regex: Pattern[bytes]) → bool[source]
spack.binary_distribution.file_type(f: IO[bytes]) → int[source]
spack.binary_distribution.generate_key_index(mirror_url: str, tmpdir: str) → None[source]

Create the key index page.

Creates (or replaces) the index.json page at the location given in mirror_url. This page contains an entry for each key under mirror_url.

spack.binary_distribution.get_buildinfo_dict(spec)[source]

Create metadata for a tarball

spack.binary_distribution.get_index_fetcher(scheme: str, mirror_metadata: MirrorMetadata, cache_entry: Dict[str, str]) → IndexHandler[source]
spack.binary_distribution.get_keys(install: bool = False, trust: bool = False, force: bool = False, mirrors: Mapping[str, Mirror] | None = None)[source]

Get pgp public keys available on mirror with suffix .pub

spack.binary_distribution.get_mirrors_for_spec(spec: Spec, index_only: bool = False) → List[MirrorMetadata][source]

Check if concrete spec exists on mirrors and return a list indicating the mirrors on which it can be found

Parameters:

  • spec – The spec to look for in binary mirrors

  • index_only – When index_only is set to True, only the local cache is checked, no requests are made.

spack.binary_distribution.install_root_node(spec: Spec, unsigned=False, force: bool = False, sha256: str | None = None, allow_missing: bool = False) → None[source]

Install the root node of a concrete spec from a buildcache.

Checking the sha256 sum of a node before installation is usually needed only for software installed during Spack’s bootstrapping (since we might not have a proper signature verification mechanism available).

Parameters:

  • spec – spec to be installed (note that only the root node will be installed)

  • unsigned – if True allows installing unsigned binaries

  • force – force installation if the spec is already present in the local store

  • sha256 – optional sha256 of the binary package, to be checked before installation

  • allow_missing – when true, allows installing a node with missing dependencies

spack.binary_distribution.install_single_spec(spec, unsigned=False, force=False)[source]

Install a single concrete spec from a buildcache.

Parameters:

  • spec (spack.spec.Spec) – spec to be installed

  • unsigned (bool) – if True allows installing unsigned binaries

  • force (bool) – force installation if the spec is already present in the local store

spack.binary_distribution.make_uploader(mirror: Mirror, force: bool = False, update_index: bool = False, signing_key: str | None = None, base_image: str | None = None) → Uploader[source]

Builder for the appropriate uploader based on the mirror type

spack.binary_distribution.needs_rebuild(spec, mirror_url)[source]
spack.binary_distribution.prefixes_to_relocate(spec)[source]
spack.binary_distribution.read_buildinfo_file(prefix)[source]

Read buildinfo file

spack.binary_distribution.relocate_package(spec: Spec) → None[source]

Relocate binaries and text files in the given spec prefix, based on its buildinfo file.

spack.binary_distribution.select_signing_key() → str[source]
spack.binary_distribution.specs_to_relocate(spec: Spec) → List[Spec][source]

Return the set of specs that may be referenced in the install prefix of the provided spec. We currently include non-external transitive link and direct run dependencies.

spack.binary_distribution.tag_is_spec(tag: str) → bool[source]

Check if a tag is likely a Spec

spack.binary_distribution.tarfile_of_spec_prefix(tar: TarFile, prefix: str, prefixes_to_relocate: List[str]) → dict[source]

Create a tarfile of an install prefix of a spec. Skips existing buildinfo file.

Parameters:

  • tar – tarfile object to add files to

  • prefix – absolute install prefix of spec

spack.binary_distribution.try_direct_fetch(spec: Spec) → List[MirrorMetadata][source]

Try to find the spec directly on the configured mirrors

spack.binary_distribution.try_fetch(url_to_fetch)[source]

Utility function to try and fetch a file from a url, stage it locally, and return the path to the staged file.

Parameters:

url_to_fetch (str) – Url pointing to remote resource to fetch

Returns:

Path to locally staged resource or None if it could not be fetched.

spack.binary_distribution.update_cache_and_get_specs()[source]

Get all concrete specs for build caches available on configured mirrors. Initialization of internal cache data structures is done as lazily as possible, so this method will also attempt to initialize and update the local index cache (essentially a no-op if it has been done already and nothing has changed on the configured mirrors.)

Raises:

FetchCacheError –

spack.binary_distribution.warn_v2_layout(mirror_url: str, action: str) → bool[source]

spack.build_environment module

This module contains all routines related to setting up the package build environment. All of this is set up by package.py just before install() is called.

There are two parts to the build environment:

  1. Python build environment (i.e. install() method)

    This is how things are set up when install() is called. Spack takes advantage of each package being in its own module by adding a bunch of command-like functions (like configure(), make(), etc.) in the package’s module scope. This allows package writers to call them all directly in Package.install() without writing ‘self.’ everywhere. No, this isn’t Pythonic. Yes, it makes the code more readable and more like the shell script from which someone is likely porting.

  2. Build execution environment

    This is the set of environment variables, like PATH, CC, CXX, etc. that control the build. There are also a number of environment variables used to pass information (like RPATHs and other information about dependencies) to Spack’s compiler wrappers. All of these env vars are also set up here.

Skimming this module is a nice way to get acquainted with the types of calls you can make from within the install() function.

class spack.build_environment.BuildProcess(*, target: Callable, args: Tuple[Any, ...], pkg: PackageBase, read_pipe: Connection, timeout: int | None)[source]

Bases: object

Class used to manage builds launched by Spack.

Each build is launched in its own child process, and the main Spack process tracks each child with a BuildProcess object. BuildProcess is used to: - Start and monitor an active child process. - Clean up its processes and resources when the child process completes. - Kill the child process if needed.

See also start_build_process() and complete_build_process().

complete()[source]

Wait (if needed) for child process to complete and return its exit status.

See complete_build_process().

property exitcode
is_alive() → bool[source]
join(*, timeout: int | None = None)[source]
property pid
poll() → bool[source]

Check if there is data available to receive from the read pipe.

start() → None[source]
terminate()[source]
exception spack.build_environment.ChildError(msg, module, classname, traceback_string, log_name, log_type, context)[source]

Bases: InstallError

Special exception class for wrapping exceptions from child processes in Spack’s build environment.

The main features of a ChildError are:

  1. They’re serializable, so when a child build fails, we can send one of these to the parent and let the parent report what happened.

  2. They have a traceback field containing a traceback generated on the child immediately after failure. Spack will print this on failure in lieu of trying to run sys.excepthook on the parent process, so users will see the correct stack trace from a child.

  3. They also contain context, which shows context in the Package implementation where the error happened. This helps people debug Python code in their packages. To get it, Spack searches the stack trace for the deepest frame where self is in scope and is an instance of PackageBase. This will generally find a useful spot in the package.py file.

The long_message of a ChildError displays one of two things:

  1. If the original error was a ProcessError, indicating a command died during the build, we’ll show context from the build log.

  2. If the original error was any other type of error, we’ll show context from the Python code.

SpackError handles displaying the special traceback if we’re in debug mode with spack -d.

build_errors
property long_message
class spack.build_environment.DeprecatedExecutable(pkg: str, exe: str, exe_pkg: str)[source]

Bases: object

__call__(*args, **kwargs)[source]

Call self as a function.

add_default_env(key: str, value: str)[source]
class spack.build_environment.EnvironmentVisitor(*roots: Spec, context: Context)[source]

Bases: object

accept(item)[source]
neighbors(item)[source]
class spack.build_environment.MakeExecutable(name: str, *, jobs: int, supports_jobserver: bool = True)[source]

Bases: Executable

Special callable executable object for make so the user can specify parallelism options on a per-invocation basis.

__call__(*args: str, parallel: bool = True, jobs_env: str | None = None, jobs_env_supports_jobserver: bool = False, fail_on_error: bool = ..., ignore_errors: int | Sequence[int] = ..., ignore_quotes: bool | None = ..., timeout: int | None = ..., env: Dict[str, str] | EnvironmentModifications | None = ..., extra_env: Dict[str, str] | EnvironmentModifications | None = ..., input: BinaryIO | None = ..., output: BinaryIO | None | str = ..., error: BinaryIO | None | str = ..., _dump_env: Dict[str, str] | None = ...) → None[source]
__call__(*args: str, parallel: bool = True, jobs_env: str | None = None, jobs_env_supports_jobserver: bool = False, fail_on_error: bool = ..., ignore_errors: int | Sequence[int] = ..., ignore_quotes: bool | None = ..., timeout: int | None = ..., env: Dict[str, str] | EnvironmentModifications | None = ..., extra_env: Dict[str, str] | EnvironmentModifications | None = ..., input: BinaryIO | None = ..., output: Type[str] | Callable = ..., error: BinaryIO | None | str | Type[str] | Callable = ..., _dump_env: Dict[str, str] | None = ...) → str
__call__(*args: str, parallel: bool = True, jobs_env: str | None = None, jobs_env_supports_jobserver: bool = False, fail_on_error: bool = ..., ignore_errors: int | Sequence[int] = ..., ignore_quotes: bool | None = ..., timeout: int | None = ..., env: Dict[str, str] | EnvironmentModifications | None = ..., extra_env: Dict[str, str] | EnvironmentModifications | None = ..., input: BinaryIO | None = ..., output: BinaryIO | None | str | Type[str] | Callable = ..., error: Type[str] | Callable = ..., _dump_env: Dict[str, str] | None = ...) → str

Runs this make executable in a subprocess.

Parameters:

  • parallel – if False, parallelism is disabled

  • jobs_env – environment variable that will be set to the current level of parallelism

  • jobs_env_supports_jobserver – whether the jobs env supports a job server

For all the other **kwargs, refer to spack.util.executable.Executable.__call__().

class spack.build_environment.ModuleChangePropagator(package: PackageBase)[source]

Bases: object

The function spack.package_base.PackageBase.setup_dependent_package() receives an instance of this class for the module argument. It’s used to set global variables in the module of a package, and propagate those globals to the modules of all classes in the inheritance hierarchy of the package. It’s reminiscent of spack.util.environment.EnvironmentModifications, but sets Python variables instead of environment variables. This class should typically not be instantiated in packages directly.

propagate_changes_to_mro()[source]
class spack.build_environment.SetupContext(*specs: Spec, context: Context)[source]

Bases: object

This class encapsulates the logic to determine environment modifications, and is used as well to set globals in modules of package.py.

external: List[Tuple[Spec, UseMode]]
get_env_modifications() → EnvironmentModifications[source]

Returns the environment variable modifications for the given input specs and context. Environment modifications include: - Updating PATH for packages that are required at runtime - Updating CMAKE_PREFIX_PATH and PKG_CONFIG_PATH so that their respective tools can find Spack-built dependencies (when context=build) - Running custom package environment modifications: setup_run_environment, setup_dependent_run_environment, setup_build_environment, setup_dependent_build_environment.

The (partial) order imposed on the specs is externals first, then topological from leaf to root. That way externals cannot contribute search paths that would shadow Spack’s prefixes, and dependents override variables set by dependencies.

nonexternal: List[Tuple[Spec, UseMode]]
set_all_package_py_globals()[source]

Set the globals in modules of package.py files.

exception spack.build_environment.UndeclaredDependencyError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised if a dependency is invoking an executable through a module global, without declaring a dependency on it.

class spack.build_environment.UseMode(*values)[source]

Bases: Flag

ADDED

Flag is set when the (node, mode) is finalized

BUILDTIME

A spec that should be visible in search paths in a build env.

BUILDTIME_DIRECT

A spec that’s a direct build or test dep

ROOT

Entrypoint spec (a spec to be built; an env root, etc)

RUNTIME

A spec used at runtime, but no executables in PATH

RUNTIME_EXECUTABLE

A spec used at runtime, with executables in PATH

spack.build_environment.clean_environment()[source]
spack.build_environment.complete_build_process(process: BuildProcess)[source]

Wait for the child process to complete and handles its exit status.

If something goes wrong, the child process catches the error and passes it to the parent wrapped in a ChildError. The parent is expected to handle (or re-raise) the ChildError.

spack.build_environment.effective_deptypes(*specs: Spec, context: Context = Context.BUILD) → List[Tuple[Spec, UseMode]][source]

Given a list of input specs and a context, return a list of tuples of all specs that contribute to (environment) modifications, together with a flag specifying in what way they do so. The list is ordered topologically from root to leaf, meaning that environment modifications should be applied in reverse so that dependents override dependencies, not the other way around.

spack.build_environment.get_cmake_prefix_path(pkg: PackageBase) → List[str][source]

Obtain the CMAKE_PREFIX_PATH entries for a package, based on the cmake_prefix_paths package attribute of direct build/test and transitive link dependencies.

spack.build_environment.get_effective_jobs(jobs, parallel: bool = True, supports_jobserver: bool = False) → int | None[source]

Return the number of jobs, or None if supports_jobserver and a jobserver is detected.

spack.build_environment.get_package_context(traceback, context=3)[source]

Return some context for an error message when the build fails.

Parameters:

  • traceback – A traceback from some exception raised during install

  • context (int) – Lines of context to show before and after the line where the error happened

This function inspects the stack to find where we failed in the package file, and it adds detailed context to the long_message from there.

spack.build_environment.get_rpath_deps(pkg: PackageBase) → List[Spec][source]

Return immediate or transitive dependencies (depending on the package) that need to be rpath’ed. If a package occurs multiple times, the newest version is kept.

spack.build_environment.jobserver_enabled()[source]

Returns true if a posix jobserver (make) is detected.

spack.build_environment.load_external_modules(context: SetupContext) → None[source]

Traverse a package’s spec DAG and load any external modules.

Traverse a package’s dependencies and load any external modules associated with them.

Parameters:

context – A populated SetupContext object

spack.build_environment.optimization_flags(compiler, target)[source]
spack.build_environment.set_package_py_globals(pkg, context: Context = Context.BUILD)[source]

Populate the Python module of a package with some useful global names. This makes things easier for package writers.

spack.build_environment.set_wrapper_environment_variables_for_flags(pkg, env)[source]
spack.build_environment.set_wrapper_variables(pkg, env)[source]

Set environment variables used by the Spack compiler wrapper (which have the prefix SPACK_) and also add the compiler wrappers to PATH.

This determines the injected -L/-I/-rpath options; each of these specifies a search order and this function computes these options in a manner that is intended to match the DAG traversal order in SetupContext. TODO: this is not the case yet, we’re using post order, SetupContext is using topo order.

spack.build_environment.setup_package(pkg, dirty, context: Context = Context.BUILD)[source]

Execute all environment setup routines.

spack.build_environment.shared_library_suffix(spec: Spec) → str[source]

Return the shared library suffix for the given spec.

spack.build_environment.start_build_process(pkg: PackageBase, function: Callable, kwargs: Dict[str, Any], *, timeout: int | None = None) → BuildProcess[source]

Create a child process to do part of a spack build.

Parameters:

  • pkg – package whose environment we should set up the child process for.

  • function – argless function to run in the child process.

  • kwargs – additional keyword arguments to pass to function()

  • timeout – maximum time allowed to finish the execution of function

Usage:

def child_fun():
    # do stuff
process = build_env.start_build_process(pkg, child_fun)
complete_build_process(process)

The child process is run with the build environment set up by spack.build_environment. This allows package authors to have full control over the environment, etc. without affecting other builds that might be executed in the same spack call.

spack.build_environment.static_library_suffix(spec: Spec) → str[source]

Return the static library suffix for the given spec.

spack.build_environment.write_log_summary(out, log_type, log, last=None)[source]

spack.buildcache_migrate module

class spack.buildcache_migrate.MigrateSpecResult(success, message)[source]

Bases: NamedTuple

message: str

Alias for field number 1

success: bool

Alias for field number 0

exception spack.buildcache_migrate.MigrationException(msg)[source]

Bases: SpackError

Raised when migration fails irrevocably

spack.buildcache_migrate.migrate(mirror: Mirror, unsigned: bool = False, delete_existing: bool = False) → None[source]

Perform migration of the given mirror

If unsigned is True, signatures on signed specs will be ignored, and specs will not be re-signed before pushing to the new location. Otherwise, spack will attempt to verify signatures and re-sign specs, and will fail if not able to do so. If delete_existing is True, spack will delete the original contents of the mirror once the migration is complete.

spack.buildcache_migrate.v2_tarball_directory_name(spec)[source]

Return name of the tarball directory according to the convention <os>-<architecture>/<compiler>/<package>-<version>/

spack.buildcache_migrate.v2_tarball_name(spec, ext)[source]

Return the name of the tarfile according to the convention <os>-<architecture>-<package>-<dag_hash><ext>

spack.buildcache_migrate.v2_tarball_path_name(spec, ext)[source]

Return the full path+name for a given spec according to the convention <tarball_directory_name>/<tarball_name>

spack.buildcache_prune module

exception spack.buildcache_prune.BuildcachePruningException(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when pruning fails irrevocably

exception spack.buildcache_prune.MalformedKeepListException(message: str, long_message: str | None = None)[source]

Bases: BuildcachePruningException

Raised when the keeplist passed to the direct pruner is invalid or malformed in some way

spack.buildcache_prune.get_buildcache_normalized_time(mirror: Mirror) → float[source]

Get the current time as reported by the buildcache.

This is necessary because different buildcache implementations may use different time formats/time zones. This function creates a temporary file, calls stat_url on it, and then deletes it. This guarantees that the time used for the beginning of the pruning is consistent across all buildcache implementations.

spack.buildcache_prune.prune_buildcache(mirror: Mirror, keeplist: str | None = None, dry_run: bool = False)[source]

Runs buildcache pruning for a given mirror.

Parameters:

  • mirror – Mirror to prune

  • keeplist_file – Path to file containing newline-delimited hashes to keep

  • dry_run – Whether to perform a dry run without actually deleting

spack.buildcache_prune.prune_direct(mirror: Mirror, keeplist_file: Path, manifest_to_mtime_mapping: Dict[str, float], read_fn: Callable[[str], URLBuildcacheEntry], blob_list: List[str], tmpspecsdir: str, pruning_started_at: float, dry_run: bool) → None[source]

Execute direct pruning for a given mirror using a keeplist file.

This function reads a file containing spec hashes to keep, then deletes all other spec manifests from the buildcache. Note that this function does not prune the blobs associated with the manifests; to do that, prune_orphan must be invoked to clean up the now-orphaned blobs.

Parameters:

  • mirror – Mirror to prune

  • keeplist_file – Path to file containing newline-delimited hashes to keep

  • pruning_started_at – Timestamp of when the pruning started

  • dry_run – Whether to perform a dry run without actually deleting

spack.buildcache_prune.prune_orphan(mirror: Mirror, manifest_to_mtime_mapping: Dict[str, float], read_fn: Callable[[str], URLBuildcacheEntry], blob_list: List[str], tmpspecsdir: str, pruning_started_at: float, dry_run: bool) → None[source]

Execute the pruning process for a given mirror.

Currently, this function only performs the pruning of orphaned manifests and blobs.

spack.builder module

spack.builder.BUILDER_CLS: Dict[str, Type[Builder]]

Builder classes, as registered by the builder decorator

class spack.builder.BaseBuilder(pkg: PackageBase)[source]

Bases: object

An interface for builders, without any phases defined. This class is exposed in the package API, so that packagers can create a single class to define setup_build_environment() and spack.phase_callbacks.run_before() and spack.phase_callbacks.run_after() callbacks that can be shared among different builders.

Example:

class AnyBuilder(BaseBuilder):
    @run_after("install")
    def fixup_install(self):
        # do something after the package is installed
        pass

    def setup_build_environment(self, env: EnvironmentModifications) -> None:
        env.set("MY_ENV_VAR", "my_value")


class CMakeBuilder(cmake.CMakeBuilder, AnyBuilder):
    pass


class AutotoolsBuilder(autotools.AutotoolsBuilder, AnyBuilder):
    pass
property prefix
setup_build_environment(env: EnvironmentModifications) → None[source]

Sets up the build environment for a package.

This method will be called before the current package prefix exists in Spack’s store.

Parameters:

env – environment modifications to be applied when the package is built. Package authors can call methods on it to alter the build environment.

setup_dependent_build_environment(env: EnvironmentModifications, dependent_spec: Spec) → None[source]

Sets up the build environment of a package that depends on this one.

This is similar to setup_build_environment, but it is used to modify the build environment of a package that depends on this one.

This gives packages the ability to set environment variables for the build of the dependent, which can be useful to provide search hints for headers or libraries if they are not in standard locations.

This method will be called before the dependent package prefix exists in Spack’s store.

Parameters:

  • env – environment modifications to be applied when the dependent package is built. Package authors can call methods on it to alter the build environment.

  • dependent_spec – the spec of the dependent package about to be built. This allows the extendee (self) to query the dependent’s state. Note that this package’s spec is available as self.spec

property spec: Spec
property stage
class spack.builder.Builder(pkg: PackageBase)[source]

Bases: BaseBuilder, Sequence

A builder is a class that, given a package object (i.e. associated with concrete spec), knows how to install it.

The builder behaves like a sequence, and when iterated over return the phases of the installation in the correct order.

property archive_files: List[str]
build_system: str | None

Build system name. Must also be defined in derived classes.

build_time_test_callbacks: List[str]
install_time_test_callbacks: List[str]
legacy_attributes: Tuple[str, ...]
legacy_long_methods: Tuple[str, ...]
legacy_methods: Tuple[str, ...]
package_attributes: Tuple[str, ...]

Attributes that the adapter can find in Package classes, if a builder is not defined

package_long_methods: Tuple[str, ...]

Methods with the same signature as phases, that the adapter can find in Package classes, if a builder is not defined.

package_methods: Tuple[str, ...]

Methods, with no arguments, that the adapter can find in Package classes, if a builder is not defined.

phases: Tuple[str, ...]

Sequence of phases. Must be defined in derived classes

class spack.builder.BuilderMeta(name, bases, attr_dict)[source]

Bases: PhaseCallbacksMeta, MultiMethodMeta, ABCMeta

class spack.builder.BuilderWithDefaults(pkg: PackageBase)[source]

Bases: Builder

Base class for all specific builders with common callbacks registered.

build_time_test_callbacks: List[str]
install_time_test_callbacks: List[str]
legacy_long_methods: Tuple[str, ...]
package_attributes: Tuple[str, ...]

Attributes that the adapter can find in Package classes, if a builder is not defined

package_long_methods: Tuple[str, ...]

Methods with the same signature as phases, that the adapter can find in Package classes, if a builder is not defined.

package_methods: Tuple[str, ...]

Methods, with no arguments, that the adapter can find in Package classes, if a builder is not defined.

class spack.builder.GenericBuilder(pkg: PackageBase)[source]

Bases: BuilderWithDefaults

The associated builder for the Package base class. This class is typically only used in package.py files when a package has multiple build systems. Packagers need to implement the install() phase to define how the package is installed.

This is the only builder that is defined in the Spack core, all other builders are defined in the builtin package repository spack_repo.builtin.build_systems.

Example:

from spack.package import *

class MyPackage(Package):
    """A package that does not use a specific build system."""
    homepage = "https://example.com/mypackage"
    url = "https://example.com/mypackage-1.0.tar.gz"

    version("1.0", sha256="...")

class GenericBuilder(GenericBuilder):
    def install(self, pkg: Package, spec: Spec, prefix: Prefix) -> None:
        pass
build_system: str | None

Build system name. Must also be defined in derived classes.

install(pkg: Package, spec: Spec, prefix: Prefix) → None[source]

Install phase for the generic builder, to be implemented by packagers.

install_time_test_callbacks: List[str]

Callback names for post-install phase tests

package_attributes: Tuple[str, ...]

Names associated with package attributes in the old build-system format

package_methods: Tuple[str, ...]

Names associated with package methods in the old build-system format

phases: Tuple[str, ...]

A generic package has only the install phase

class spack.builder.InstallationPhase(name, builder)[source]

Bases: object

Manages a single phase of the installation.

This descriptor stores at creation time the name of the method it should search for execution. The method is retrieved at __get__ time, so that it can be overridden by subclasses of whatever class declared the phases.

It also provides hooks to execute arbitrary callbacks before and after the phase.

copy()[source]
execute()[source]
class spack.builder.Package(spec: Spec)[source]

Bases: PackageBase

Build system base class for packages that do not use a specific build system. It adds the build_system=generic variant to the package.

This is the only build system base class defined in Spack core. All other build systems are defined in the builtin package repository spack_repo.builtin.build_systems.

The associated builder is GenericBuilder, which is only necessary when the package has multiple build systems.

Example:

from spack.package import *

class MyPackage(Package):
    """A package that does not use a specific build system."""

    homepage = "https://example.com/mypackage"
    url = "https://example.com/mypackage-1.0.tar.gz"

    version("1.0", sha256="...")

    def install(self, spec: Spec, prefix: Prefix) -> None:
        # Custom installation logic here
        pass

Note

The difference between Package and PackageBase is that PackageBase is the universal base class for all package classes, no matter their build system.

The Package class is a build system base class, similar to CMakePackage, and AutotoolsPackage. It is called Package and not GenericPackage for legacy reasons.

build_system_class: str

This attribute is used in UI queries that require to know which build-system class we are using

conflicts: Dict[Spec, List[Tuple[Spec, str | None]]]

Class level dictionary populated by conflicts() directives

default_buildsystem: str

Legacy buildsystem attribute used to deserialize and install old specs

dependencies: Dict[Spec, Dict[str, Dependency]]

Class level dictionary populated by depends_on() and extends() directives

disable_redistribute: Dict[Spec, DisableRedistribute]

Class level dictionary populated by redistribute() directives

extendees: Dict[str, Tuple[Spec, Spec]]

Class level dictionary populated by extends() directives

legacy_buildsystem: str

Use default_buildsystem instead of this attribute, which is deprecated

licenses: Dict[Spec, str]

Class level dictionary populated by license() directives

patches: Dict[Spec, List[Patch]]

Class level dictionary populated by patch() directives

provided: Dict[Spec, Set[Spec]]

Class level dictionary populated by provides() directives

provided_together: Dict[Spec, List[Set[str]]]

Class level dictionary populated by provides() directives

requirements: Dict[Spec, List[Tuple[Tuple[Spec, ...], str, str | None]]]

Class level dictionary populated by requires() directives

resources: Dict[Spec, List[Resource]]

Class level dictionary populated by resource() directives

splice_specs: Dict[Spec, Tuple[Spec, None | str | List[str]]]

Class level dictionary populated by can_splice() directives

variants: Dict[Spec, Dict[str, Variant]]

Class level dictionary populated by variant() directives

versions: Dict[StandardVersion, Dict[str, Any]]

Class level dictionary populated by version() directives

spack.builder.apply_macos_rpath_fixups(builder: Builder)[source]

On Darwin, make installed libraries more easily relocatable.

Some build systems (handrolled, autotools, makefiles) can set their own rpaths that are duplicated by spack’s compiler wrapper. This fixup interrogates, and postprocesses if necessary, all libraries installed by the code.

It should be added as a run_after() to packaging systems (or individual packages) that do not install relocatable libraries by default.

Example:

run_after("install", when="platform=darwin")(apply_macos_rpath_fixups)
Parameters:

builder – builder that installed the package

spack.builder.buildsystem_name(pkg: PackageBase) → str[source]

Given a package object with an associated concrete spec, return the name of its build system.

spack.builder.create(pkg: PackageBase) → Builder[source]

Given a package object with an associated concrete spec, return the builder object that can install it.

spack.builder.execute_install_time_tests(builder: Builder)[source]

Execute the install-time tests prescribed by builder.

Parameters:

builder – builder prescribing the test callbacks. The name of the callbacks is stored as a list of strings in the install_time_test_callbacks attribute.

spack.builder.get_builder_class(pkg, name: str) → Type[Builder] | None[source]

Return the builder class if a package module defines it.

spack.builder.package_attributes(builder: Type[Builder]) → Tuple[str, ...][source]

Returns the list of attributes that are defined in the package class and are associated with the builder.

spack.builder.package_long_methods(builder: Type[Builder]) → Tuple[str, ...][source]

Returns the list of methods, with the same signature as phases, that are defined in the package class and are associated with the builder.

spack.builder.package_methods(builder: Type[Builder]) → Tuple[str, ...][source]

Returns the list of methods, taking no arguments, that are defined in the package class and are associated with the builder.

spack.builder.register_builder(build_system_name: str)[source]

Class decorator used to register the default builder for a given build system. The name corresponds to the build_system variant value of the package.

Example:

@register_builder("cmake")
class CMakeBuilder(BuilderWithDefaults):
    pass
Parameters:

build_system_name – name of the build system

spack.builder.sanity_check_prefix(builder: Builder)[source]

Check that specific directories and files are created after installation.

The files to be checked are in the sanity_check_is_file attribute of the package object, while the directories are in the sanity_check_is_dir.

Parameters:

builder – builder that installed the package

spack.caches module

Caches used by Spack to store data

spack.caches.FETCH_CACHE

Spack’s local cache for downloaded source archives

spack.caches.MISC_CACHE

Spack’s cache for small data

class spack.caches.MirrorCache(root, skip_unstable_versions)[source]

Bases: FsCacheBase

store(fetcher, relative_dest)[source]

Fetch and relocate the fetcher’s target into our mirror cache.

Note: archives package sources even if not normally cached (e.g. tip of hg/git branch).

spack.caches.fetch_cache_location()[source]

Filesystem cache of downloaded archives.

This prevents Spack from repeatedly fetch the same files when building the same package different ways or multiple times.

spack.caches.misc_cache_location()[source]

The MISC_CACHE is Spack’s cache for small data.

Currently the MISC_CACHE stores indexes for virtual dependency providers and for which packages provide which tags.

spack.concretize module

High-level functions to concretize list of specs

exception spack.concretize.UnavailableCompilerVersionError(compiler_spec: CompilerSpec, arch: ArchSpec | None = None)[source]

Bases: SpackError

Raised when there is no available compiler that satisfies a compiler spec.

spack.concretize.concretize_one(spec: str | Spec, *, tests: bool | Iterable[str] = False, factory: SpecFiltersFactory | None = None) → Spec[source]

Return a concretized copy of the given spec.

Parameters:

tests – if False disregard test dependencies, if a list of names activate them for the packages in the list, if True activate test dependencies for all packages.

spack.concretize.concretize_separately(spec_list: Sequence[Tuple[Spec, Spec | None]], *, tests: bool | Iterable[str] = False, factory: SpecFiltersFactory | None = None) → List[Tuple[Spec, Spec]][source]

Concretizes the input specs separately from each other.

Parameters:

  • spec_list – list of tuples to concretize. First entry is abstract spec, second entry is already concrete spec or None if not yet concretized

  • tests – list of package names for which to consider tests dependencies. If True, all nodes will have test dependencies. If False, test dependencies will be disregarded.

  • factory – optional factory to produce a list of specs to be reused

spack.concretize.concretize_together(spec_list: Sequence[Tuple[Spec, Spec | None]], *, tests: bool | Iterable[str] = False, factory: SpecFiltersFactory | None = None) → List[Tuple[Spec, Spec]][source]

Given a number of specs as input, tries to concretize them together.

Parameters:

  • spec_list – list of tuples to concretize. First entry is abstract spec, second entry is already concrete spec or None if not yet concretized

  • tests – list of package names for which to consider tests dependencies. If True, all nodes will have test dependencies. If False, test dependencies will be disregarded.

  • factory – optional factory to produce a list of specs to be reused

spack.concretize.concretize_together_when_possible(spec_list: Sequence[Tuple[Spec, Spec | None]], *, tests: bool | Iterable[str] = False, factory: SpecFiltersFactory | None = None) → List[Tuple[Spec, Spec]][source]

Given a number of specs as input, tries to concretize them together to the extent possible.

See documentation for unify: when_possible concretization for the precise definition of “to the extent possible”.

Parameters:

  • spec_list – list of tuples to concretize. First entry is abstract spec, second entry is already concrete spec or None if not yet concretized

  • tests – list of package names for which to consider tests dependencies. If True, all nodes will have test dependencies. If False, test dependencies will be disregarded.

  • factory – optional factory to produce a list of specs to be reused

spack.config module

This module implements Spack’s configuration file handling.

This implements Spack’s configuration system, which handles merging multiple scopes with different levels of precedence. See the documentation on Configuration Scopes for details on how Spack’s configuration system behaves. The scopes set up here are:

  1. spack in $spack/etc/spack - controls all built-in spack scopes, except default

  2. defaults in $spack/etc/spack/defaults - defaults that Spack needs to function

Important functions in this module are:

get_config reads in YAML data for a particular scope and returns it. Callers can then modify the data and write it back with update_config.

When read in, Spack validates configurations with jsonschemas. The schemas are in submodules of spack.schema.

spack.config.CONFIG

This is the singleton configuration instance for Spack.

spack.config.CONFIGURATION_DEFAULTS_PATH

Path to the main configuration scope

spack.config.CONFIG_DEFAULTS

Hard-coded default values for some key configuration options. This ensures that Spack will still work even if config.yaml in the defaults scope is removed.

exception spack.config.ConfigFileError(message: str, long_message: str | None = None)[source]

Bases: ConfigError

Issue reading or accessing a configuration file.

exception spack.config.ConfigFormatError(validation_error, data: Dict[str, Any], filename: str | None = None, line: int | None = None)[source]

Bases: ConfigError

Raised when a configuration format does not match its schema.

class spack.config.ConfigPath[source]

Bases: object

element
next_key_pattern
static process(path)[source]
quoted_string
unquoted_string
class spack.config.ConfigScope(name: str, included: bool = False)[source]

Bases: object

clear() → None[source]

Empty cached config information.

property exists: bool

Whether the config object indicated by the scope can be read

get_section(section: str) → Dict[str, Any] | None[source]
get_section_filename(section: str) → str[source]
property included_scopes: List[ConfigScope]

Memoized list of included scopes, in the order they appear in this scope.

override_include()[source]

Whether the include:: section of this scope should override lower scopes.

transitive_includes(_names: Set[str] | None = None) → Set[str][source]

Get name of this scope and names of its transitively included scopes.

exception spack.config.ConfigSectionError(message: str, long_message: str | None = None)[source]

Bases: ConfigError

Error for referring to a bad config section name in a configuration.

class spack.config.Configuration[source]

Bases: object

A hierarchical configuration, merging a number of scopes at different priorities.

property active_include_section_scopes: List[ConfigScope]

Return a list of all scopes whose includes have not been overridden by include::.

This is different from the active scopes because the spack scope can be active while its includes are overwritten, as can the transitive includes from the overriding scope.

property active_scopes: List[ConfigScope]

Return a list of scopes that have not been overridden by include::.

clear_caches() → None[source]

Clears the caches for configuration files,

This will cause files to be re-read upon the next request.

deepcopy_as_builtin(section: str, scope: str | None = None, *, line_info: bool = False) → Dict[str, Any][source]

Get a deep copy of a section with native Python types, excluding YAML metadata.

ensure_unwrapped() → Configuration[source]

Ensure we unwrap this object from any dynamic wrapper (like Singleton)

property existing_scopes: Generator[ConfigScope, None, None]

Generator of existing scopes. These are self.scopes where the scope has a representation on the filesystem or is internal

get(path: str, default: Any | None = None, scope: str | None = None) → Any[source]

Get a config section or a single value from one.

Accepts a path syntax that allows us to grab nested config map entries. Getting the config section would look like:

spack.config.get("config")

and the dirty section in the config scope would be:

spack.config.get("config:dirty")

We use : as the separator, like YAML objects.

get_config(section: str, scope: str | None = None, _merged_scope: str | None = None) → Dict[str, Any][source]

Get configuration settings for a section.

If scope is None or not provided, return the merged contents of all of Spack’s configuration scopes. If scope is provided, return only the configuration as specified in that scope.

This off the top-level name from the YAML section. That is, for a YAML config file that looks like this:

config:
  install_tree:
    root: $spack/opt/spack
  build_stage:
  - $tmpdir/$user/spack-stage

get_config('config') will return:

{ 'install_tree': {
      'root': '$spack/opt/spack',
  }
  'build_stage': ['$tmpdir/$user/spack-stage']
}
get_config_filename(scope: str, section: str) → str[source]

For some scope and section, get the name of the configuration file.

highest() → ConfigScope[source]

Scope with the highest precedence

highest_precedence_scope() → ConfigScope[source]

Writable scope with the highest precedence.

matching_scopes(reg_expr) → List[ConfigScope][source]

List of all scopes whose names match the provided regular expression.

For example, matching_scopes(r'^command') will return all scopes whose names begin with command.

print_section(section: str, yaml: bool = True, blame: bool = False, *, scope: str | None = None) → None[source]

Print a configuration to stdout.

Parameters:

  • section – The configuration section to print.

  • yaml – If True, output in YAML format, otherwise JSON (ignored when blame is True).

  • blame – Whether to include source locations for each entry.

  • scope – The configuration scope to use.

push_scope(scope: ConfigScope, priority: int | None = None, _depth: int = 0) → None[source]

Add a scope to the Configuration, at a given priority.

If a priority is not given, it is assumed to be the current highest priority.

Parameters:

  • scope – scope to be added

  • priority – priority of the scope

push_scope_incremental(scope: ConfigScope, priority: int | None = None, _depth: int = 0) → Generator[Configuration, None, None][source]

Adds a scope to the Configuration, at a given priority.

push_scope_incremental yields included scopes incrementally, so that their data can be used by higher priority scopes during config initialization. If you push a scope that includes other, low-priority scopes, they will be pushed on first, before the scope that included them.

If a priority is not given, it is assumed to be the current highest priority.

Parameters:

  • scope – scope to be added

  • priority – priority of the scope

remove_scope(scope_name: str) → ConfigScope | None[source]

Removes a scope by name, and returns it. If the scope does not exist, returns None.

scopes: PriorityOrderedMapping[str, ConfigScope]
set(path: str, value: Any, scope: str | None = None) → None[source]

Convenience function for setting single values in config files.

Accepts the path syntax described in get().

update_config(section: str, update_data: Dict, scope: str | None = None, force: bool = False) → None[source]

Update the configuration file for a particular scope.

Overwrites contents of a section in a scope with update_data, then writes out the config file.

update_data should have the top-level section name stripped off (it will be re-added). Data itself can be a list, dict, or any other yaml-ish structure.

Configuration scopes that are still written in an old schema format will fail to update unless force is True.

Parameters:

  • section – section of the configuration to be updated

  • update_data – data to be used for the update

  • scope – scope to be updated

  • force – force the update

updated_scopes_by_section: Dict[str, List[ConfigScope]]
property writable_scopes: Generator[ConfigScope, None, None]

Generator of writable scopes with an associated file.

class spack.config.DirectoryConfigScope(name: str, path: str, *, writable: bool = True, prefer_modify: bool = True, included: bool = False)[source]

Bases: ConfigScope

Config scope backed by a directory containing one file per section.

property exists: bool

Whether the config object indicated by the scope can be read

get_section(section: str) → Dict[str, Any] | None[source]

Returns the data associated with a given section if the scope exists

get_section_filename(section: str) → str[source]

Returns the filename associated with a given section

class spack.config.GitIncludePaths(entry: dict)[source]

Bases: OptionalInclude

branch: str
commit: str
destination: str | None
fetched() → bool[source]
git: str
property paths: List[str]

Path(s) associated with the include.

scopes(parent_scope: ConfigScope) → List[ConfigScope][source]

Instantiate configuration scopes for the included paths.

Parameters:

parent_scope – including scope

Returns: configuration scopes IF the when condition is satisfied;

otherwise, an empty list.

Raises:

  • ConfigFileError – unable to access remote configuration file(s)

  • ValueError – included path has an unsupported URL scheme, is required but does not exist; configuration stage directory argument is missing

tag: str
class spack.config.IncludePath(entry: dict)[source]

Bases: OptionalInclude

destination: str | None
path: str
property paths: List[str]

Path(s) associated with the include.

scopes(parent_scope: ConfigScope) → List[ConfigScope][source]

Instantiate a configuration scope for the included path.

Parameters:

parent_scope – including scope

Returns: configuration scopes IF the when condition is satisfied;

otherwise, an empty list.

Raises:

  • ConfigFileError – unable to access remote configuration file

  • ValueError – included path has an unsupported URL scheme, is required but does not exist; configuration stage directory argument is missing

sha256: str
class spack.config.InternalConfigScope(name: str, data: Dict[str, Any] | None = None)[source]

Bases: ConfigScope

An internal configuration scope that is not persisted to a file.

This is for spack internal use so that command-line options and config file settings are accessed the same way, and Spack can easily override settings from files.

clear() → None[source]

Empty cached config information.

get_section(section: str) → Dict[str, Any] | None[source]

Just reads from an internal dictionary.

spack.config.MAX_RECURSIVE_INCLUDES

safeguard for recursive includes – maximum include depth

class spack.config.OptionalInclude(entry: dict)[source]

Bases: object

Base properties for all includes.

base_directory(path_or_url: str, parent_scope: ConfigScope | None = None) → str | None[source]

Return the local directory to use for this include.

For remote includes this is the cache destination directory. For local relative includes this is the working directory from which to resolve the path.

Parameters:

  • path_or_url – path or URL of the include

  • parent_scope – including scope

Returns: None for a local include without an enclosing parent scope;

an appropriate subdirectory of the enclosing (parent) scope’s writable directory (when available); otherwise a stable temporary directory.

evaluate_condition() → bool[source]

Evaluate the include condition:

Returns: True if the include condition is satisfied; else False.

name: str
optional: bool
property paths: List[str]

Path(s) associated with the include.

prefer_modify: bool
remote: bool
scopes(parent_scope: ConfigScope) → List[ConfigScope][source]

Instantiate configuration scopes.

Parameters:

parent_scope – including scope

Returns: configuration scopes for configuration files IF the when

condition is satisfied; otherwise, an empty list.

Raises:

ValueError – the required configuration path does not exist

when: str
exception spack.config.RecursiveIncludeError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Too many levels of recursive includes.

spack.config.SCOPES_METAVAR

metavar to use for commands that accept scopes this is shorter and more readable than listing all choices

spack.config.SECTION_SCHEMAS: Dict[str, Any]

Dict from section names -> schema for that section

class spack.config.SingleFileScope(name: str, path: str, schema: Dict[str, Any], *, yaml_path: List[str] | None = None, writable: bool = True, prefer_modify: bool = True, included: bool = False)[source]

Bases: ConfigScope

This class represents a configuration scope in a single YAML file.

property exists: bool

Whether the config object indicated by the scope can be read

get_section(section: str) → Dict[str, Any] | None[source]
get_section_filename(section) → str[source]
spack.config.YamlConfigDict

Type used for raw YAML configuration

alias of Dict[str, Any]

spack.config.add(fullpath: str, scope: str | None = None) → None[source]

Add the given configuration to the specified config scope. Add accepts a path. If you want to add from a filename, use add_from_file

spack.config.add_from_file(filename: str, scope: str | None = None) → None[source]

Add updates to a config from a filename

spack.config.change_or_add(section_name: str, find_fn: Callable[[str], bool], update_fn: Callable[[str], None]) → None[source]

Change or add a subsection of config, with additional logic to select a reasonable scope where the change is applied.

Search through config scopes starting with the highest priority: the first matching a criteria (determined by find_fn) is updated; if no such config exists, find the first config scope that defines any config for the named section; if no scopes define any related config, then update the highest-priority config scope.

spack.config.config_paths_from_entry_points() → List[Tuple[str, str]][source]

Load configuration paths from entry points

A python package can register entry point metadata so that Spack can find its configuration by adding the following to the project’s pyproject.toml:

[project.entry-points."spack.config"]
baz = "baz:get_spack_config_path"

The function get_spack_config_path returns the path to the package’s spack configuration scope

spack.config.create() → Configuration[source]

Create a configuration using create_incremental(), return the last yielded result.

spack.config.create_from(*scopes_or_paths: ConfigScope | Tuple[int, ConfigScope] | str) → Configuration[source]

Creates a configuration object from the scopes passed in input.

Parameters:

*scopes_or_paths – either a tuple of (priority, ConfigScope), or a ConfigScope, or a string If priority is not given, it is assumed to be ConfigScopePriority.CONFIG_FILES. If a string is given, a DirectoryConfigScope is created from it.

Examples

>>> builtin_scope = InternalConfigScope("_builtin", {"config": {"build_jobs": 1}})
>>> cl_scope = InternalConfigScope("command_line", {"config": {"build_jobs": 10}})
>>> cfg = create_from(
...     (ConfigScopePriority.COMMAND_LINE, cl_scope),
...     (ConfigScopePriority.BUILTIN, builtin_scope)
... )
spack.config.create_incremental() → Generator[Configuration, None, None][source]

Singleton Configuration instance.

This constructs one instance associated with this module and returns it. It is bundled inside a function so that configuration can be initialized lazily.

spack.config.default_modify_scope(section: str = 'config') → str[source]

Return the config scope that commands should modify by default.

Commands that modify configuration by default modify the highest priority scope.

Parameters:

section (bool) – Section for which to get the default scope.

spack.config.determine_number_of_jobs(*, parallel: bool = False, max_cpus: int = cpus_available(), config: Configuration | None = None) → int[source]

Packages that require sequential builds need 1 job. Otherwise we use the number of jobs set on the command line. If not set, then we use the config defaults (which is usually set through the builtin config scope), but we cap to the number of CPUs available to avoid oversubscription.

Parameters:

  • parallel – true when package supports parallel builds

  • max_cpus – maximum number of CPUs to use (defaults to cpus_available())

  • config – configuration object (defaults to global config)

spack.config.ensure_latest_format_fn(section: str) → Callable[[Dict[str, Any]], bool][source]

Return a function that takes a config dictionary and update it to the latest format.

The function returns True iff there was any update.

Parameters:

section – section of the configuration e.g. “packages”, “config”, etc.

spack.config.existing_scope_names() → List[str][source]
spack.config.existing_scopes() → List[ConfigScope][source]

Return list of existing scopes. Scopes where Spack is aware of said scope, and the scope has a representation on the filesystem or are internal scopes. Higher-priority scopes come first in the list.

spack.config.get(path: str, default: Any | None = None, scope: str | None = None) → Any[source]

Module-level wrapper for Configuration.get().

spack.config.get_valid_type(path)[source]

Returns an instance of a type that will pass validation for path.

The instance is created by calling the constructor with no arguments. If multiple types will satisfy validation for data at the configuration path given, the priority order is list, dict, str, bool, int, float.

spack.config.included_path(entry: str | Path | dict) → IncludePath | GitIncludePaths[source]

Convert the included paths entry into the appropriate optional include.

Parameters:

entry – include configuration entry

Returns: converted entry, where an empty when means the path is not conditionally included

spack.config.matched_config(cfg_path: str) → List[Tuple[str, Any]][source]
spack.config.override(path_or_scope: ConfigScope | str, value: Any | None = None) → Generator[Configuration, None, None][source]

Simple way to override config settings within a context.

Parameters:

  • path_or_scope (ConfigScope or str) – scope or single option to override

  • value (object or None) – value for the single option

Temporarily push a scope on the current configuration, then remove it after the context completes. If a single option is provided, create an internal config scope for it and push/pop that scope.

spack.config.paths_from_includes(includes: List[str | dict]) → List[str][source]

The path(s) from the configured includes.

Parameters:

includes – include configuration information

Returns: list of path or an empty list if there are none

spack.config.process_config_path(path: str) → List[str][source]

Process a path argument to config.set() that may contain overrides (:: or trailing :)

Colons will be treated as static strings if inside of quotes, e.g. this:is:a:path:'value:with:colon' will yield:

[this, is, a, path, value:with:colon]

The path may consist only of keys (e.g. for a get) or may end in a value. Keys are always strings: if a user encloses a key in quotes, the quotes should be removed. Values with quotes should be treated as strings, but without quotes, may be parsed as a different yaml object (e.g. '{}' is a dict, but '"{}"' is a string).

This function does not know whether the final element of the path is a key or value, so:

  • It must strip the quotes, in case it is a key (so we look for key and not "key")

  • It must indicate somehow that the quotes were stripped, in case it is a value (so that we don’t process "{}" as a YAML dict)

Therefore, all elements with quotes are stripped, and then also converted to syaml_str (if treating the final element as a value, the caller should not parse it in this case).

spack.config.read_config_file(path: str, schema: Dict[str, Any] | None = None) → Dict[str, Any] | None[source]

Read a YAML configuration file.

User can provide a schema for validation. If no schema is provided, we will infer the schema from the top-level key.

spack.config.remove_yaml(dest, source)[source]

UnMerges source from dest; entries in source take precedence over dest.

This routine may modify dest and should be assigned to dest, in case dest was None to begin with, e.g.:

dest = remove_yaml(dest, source)

In the result, elements from lists from source will not appear as elements of lists from dest. Likewise, when iterating over keys or items in merged OrderedDict objects, keys from source will not appear as keys in dest.

Config file authors can optionally end any attribute in a dict with :: instead of :, and the key will remove the entire section from dest

spack.config.scopes() → PriorityOrderedMapping[str, ConfigScope][source]

Convenience function to get list of configuration scopes.

spack.config.set(path: str, value: Any, scope: str | None = None) → None[source]

Convenience function for setting single values in config files.

Accepts the path syntax described in get().

spack.config.update_all(section_name: str, change_fn: Callable[[str], bool]) → None[source]

Change a config section, which may have details duplicated across multiple scopes.

spack.config.use_configuration(*scopes_or_paths: ConfigScope | Tuple[int, ConfigScope] | str) → Generator[Configuration, None, None][source]

Use the configuration scopes passed as arguments within the context manager.

This function invalidates caches, and is therefore very slow.

Parameters:

*scopes_or_paths – scope objects or paths to be used

Returns:

Configuration object associated with the scopes passed as arguments

spack.config.validate(data: Dict[str, Any], schema: Dict[str, Any], filename: str | None = None) → Dict[str, Any][source]

Validate data read in from a Spack YAML file.

Parameters:

  • data – data read from a Spack YAML file

  • schema – jsonschema to validate data

This leverages the line information (start_mark, end_mark) stored on Spack YAML structures.

spack.config.writable_scope_names() → List[str][source]
spack.config.writable_scopes() → List[ConfigScope][source]

Return list of writable scopes. Higher-priority scopes come first in the list.

spack.context module

This module provides classes used in user and build environment

class spack.context.Context(*values)[source]

Bases: Enum

Enum used to indicate the context in which an environment has to be setup: build, run or test.

BUILD
RUN
TEST
classmethod from_string(s: str)[source]

spack.cray_manifest module

exception spack.cray_manifest.CrayCompilerDetectionError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised if a compiler, listed in the Cray manifest, cannot be detected correctly based on the paths provided.

exception spack.cray_manifest.ManifestValidationError(msg, long_msg=None)[source]

Bases: SpackError

spack.cray_manifest.compiler_from_entry(entry: dict, *, manifest_path: str) → Spec | None[source]
spack.cray_manifest.compiler_spec_from_paths(*, pkg_name: str, compiler_paths: Iterable[str]) → Spec[source]

Returns the external spec associated with a series of compilers, if any.

spack.cray_manifest.default_path

Cray systems can store a Spack-compatible description of system packages here.

spack.cray_manifest.entries_to_specs(entries)[source]
spack.cray_manifest.extract_compiler_paths(entry: Dict[str, Any]) → List[str][source]

Returns the paths to compiler executables, from a dictionary entry in the Cray manifest.

spack.cray_manifest.read(path, apply_updates)[source]
spack.cray_manifest.spec_from_entry(entry)[source]
spack.cray_manifest.translated_compiler_name(manifest_compiler_name)[source]

When creating a Compiler object, Spack expects a name matching one of the classes in spack.compilers.config. Names in the Cray manifest may differ; for cases where we know the name refers to a compiler in Spack, this function translates it automatically.

This function will raise an error if there is no recorded translation and the name doesn’t match a known compiler name.

spack.database module

Spack’s installation tracking database.

The database serves two purposes:

  1. It implements a cache on top of a potentially very large Spack directory hierarchy, speeding up many operations that would otherwise require filesystem access.

  2. It will allow us to track external installations as well as lost packages and their dependencies.

Prior to the implementation of this store, a directory layout served as the authoritative database of packages in Spack. This module provides a cache and a sanity checking mechanism for what is in the filesystem.

exception spack.database.CorruptDatabaseError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when errors are found while reading the database.

spack.database.DEFAULT_INSTALL_RECORD_FIELDS

Default list of fields written for each install record

spack.database.DEFAULT_LOCK_CFG: LockConfiguration

Default configuration for database locks

class spack.database.Database(root: str, *, upstream_dbs: List[Database] | None = None, is_upstream: bool = False, lock_cfg: LockConfiguration = DEFAULT_LOCK_CFG, layout: DirectoryLayout | None = None)[source]

Bases: object

add(spec_like, *args, **kwargs)[source]
all_hashes()[source]

Return dag hash of every spec in the database.

db_for_spec_hash(hash_key)[source]
property db_version: ConcreteVersion
deprecate(spec_like, *args, **kwargs)[source]
deprecator(spec: Spec) → Spec | None[source]

Return the spec that the given spec is deprecated for, or None

get_by_hash(dag_hash: str, default: List[Spec] | None = None, installed: bool | InstallRecordStatus = InstallRecordStatus.ANY) → List[Spec] | None[source]

Look up a spec by DAG hash, or by a DAG hash prefix.

Parameters:

  • dag_hash – hash (or hash prefix) to look up

  • default – default value to return if dag_hash is not in the DB

  • installed – if True, includes only installed specs in the search; if False only missing specs. Otherwise, a InstallRecordStatus flag.

installed defaults to InstallRecordStatus.ANY so we can refer to any known hash. query() and query_one() differ in that they only return installed specs by default.

get_by_hash_local(dag_hash: str, default: List[Spec] | None = None, installed: bool | InstallRecordStatus = InstallRecordStatus.ANY) → List[Spec] | None[source]

Look up a spec in this DB by DAG hash, or by a DAG hash prefix.

Parameters:

  • dag_hash – hash (or hash prefix) to look up

  • default – default value to return if dag_hash is not in the DB

  • installed – if True, includes only installed specs in the search; if False only missing specs. Otherwise, a InstallRecordStatus flag.

installed defaults to InstallRecordStatus.ANY so we can refer to any known hash.

query() and query_one() differ in that they only return installed specs by default.

get_record(spec_like, *args, **kwargs)[source]
installed_extensions_for(spec_like, *args, **kwargs)[source]
installed_relatives(spec_like, *args, **kwargs)[source]
is_occupied_install_prefix(path)[source]
is_readable() → bool[source]

Returns true if this DB can be read without reindexing

lock: ForbiddenLock | Lock
mark(spec_like, *args, **kwargs)[source]
missing(spec)[source]
query(query_spec: str | Spec | None = None, *, predicate_fn: Callable[[InstallRecord], bool] | None = None, installed: bool | InstallRecordStatus = True, explicit: bool | None = None, start_date: datetime | None = None, end_date: datetime | None = None, in_buildcache: bool | None = None, hashes: List[str] | None = None, origin: str | None = None, install_tree: str = 'all') → List[Spec][source]

Queries the Spack database including all upstream databases.

Parameters:

  • query_spec – if query_spec is None, match all specs in the database. If it is a spec, return all specs matching spec.satisfies(query_spec).

  • predicate_fn – optional predicate taking an InstallRecord as argument, and returning whether that record is selected for the query. It can be used to craft criteria that need some data for selection not provided by the Database itself.

  • installed – if True, includes only installed specs in the search. If False only missing specs, and if any, all specs in database. If an InstallStatus or iterable of InstallStatus, returns specs whose install status matches at least one of the InstallStatus.

  • explicit – a spec that was installed following a specific user request is marked as explicit. If instead it was pulled-in as a dependency of a user requested spec it’s considered implicit.

  • start_date – if set considers only specs installed from the starting date.

  • end_date – if set considers only specs installed until the ending date.

  • in_buildcache – specs that are marked in this database as part of an associated binary cache are in_buildcache. All other specs are not. This field is used for querying mirror indices. By default, it does not check this status.

  • hashes – list of hashes used to restrict the search

  • install_tree – query "all" (default), "local", "upstream", or upstream path

  • origin – origin of the spec

query_by_spec_hash(hash_key: str, data: Dict[str, InstallRecord] | None = None) → Tuple[bool, InstallRecord | None][source]

Get a spec for hash, and whether it’s installed upstream.

Returns:

Tuple of bool and optional InstallRecord. The bool tells us whether the record is from an upstream. Its InstallRecord is also returned if available (the record must be checked to know whether the hash is installed).

If the record is available locally, this function will always have a preference for returning that, even if it is not installed locally and is installed upstream.

query_local(query_spec: str | Spec | None = None, *, predicate_fn: Callable[[InstallRecord], bool] | None = None, installed: bool | InstallRecordStatus = True, explicit: bool | None = None, start_date: datetime | None = None, end_date: datetime | None = None, hashes: List[str] | None = None, in_buildcache: bool | None = None, origin: str | None = None) → List[Spec][source]

Queries the local Spack database.

This function doesn’t guarantee any sorting of the returned data for performance reason, since comparing specs for __lt__ may be an expensive operation.

Parameters:

  • query_spec – if query_spec is None, match all specs in the database. If it is a spec, return all specs matching spec.satisfies(query_spec).

  • predicate_fn – optional predicate taking an InstallRecord as argument, and returning whether that record is selected for the query. It can be used to craft criteria that need some data for selection not provided by the Database itself.

  • installed – if True, includes only installed specs in the search. If False only missing specs, and if any, all specs in database. If an InstallStatus or iterable of InstallStatus, returns specs whose install status matches at least one of the InstallStatus.

  • explicit – a spec that was installed following a specific user request is marked as explicit. If instead it was pulled-in as a dependency of a user requested spec it’s considered implicit.

  • start_date – if set considers only specs installed from the starting date.

  • end_date – if set considers only specs installed until the ending date.

  • in_buildcache – specs that are marked in this database as part of an associated binary cache are in_buildcache. All other specs are not. This field is used for querying mirror indices. By default, it does not check this status.

  • hashes – list of hashes used to restrict the search

  • origin – origin of the spec

query_local_by_spec_hash(hash_key: str) → InstallRecord | None[source]

Get a spec by hash in the local database

Returns:

InstallRecord when installed locally, otherwise None.

query_one(query_spec: str | Spec | None, predicate_fn: Callable[[InstallRecord], bool] | None = None, installed: bool | InstallRecordStatus = True) → Spec | None[source]

Query for exactly one spec that matches the query spec.

Returns None if no installed package matches.

Raises:

AssertionError – if more than one spec matches the query.

raise_explicit_database_upgrade_error()[source]

Raises an ExplicitDatabaseUpgradeError with an appropriate message

read_transaction()[source]

Get a read lock context manager for use in a with block.

record_fields: Tuple[str, ...]

Fields written for each install record

reindex()[source]

Build database index from scratch based on a directory layout.

Locks the DB if it isn’t locked already.

remove(spec_like, *args, **kwargs)[source]
specs_deprecated_by(spec: Spec) → List[Spec][source]

Return all specs deprecated in favor of the given spec

unused_specs(root_hashes: Container[str] | None = None, deptype: int | str | List[str] | Tuple[str, ...] = dt.LINK | dt.RUN) → List[Spec][source]

Return all specs that are currently installed but not needed by root specs.

By default, roots are all explicit specs in the database. If a set of root hashes are passed in, they are instead used as the roots.

Parameters:

  • root_hashes – optional list of roots to consider when evaluating needed installations.

  • deptype – if a spec is reachable from a root via these dependency types, it is considered needed. By default only link and run dependency types are considered.

write_transaction()[source]

Get a write lock context manager for use in a with block.

exception spack.database.DatabaseNotReadableError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised to signal Database.reindex that the reindex should happen via spec.json

exception spack.database.ExplicitDatabaseUpgradeError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised to request an explicit DB upgrade to the user

class spack.database.FailureTracker(root_dir: str | Path, default_timeout: float | None)[source]

Bases: object

Tracks installation failures.

Prefix failure marking takes the form of a byte range lock on the nth byte of a file for coordinating between concurrent parallel build processes and a persistent file, named with the full hash and containing the spec, in a subdirectory of the database to enable persistence across overlapping but separate related build processes.

The failure lock file lives alongside the install DB.

n is the sys.maxsize-bit prefix of the associated DAG hash to make the likelihood of collision very low with no cleanup required.

clear(spec: Spec, force: bool = False) → None[source]

Removes any persistent and cached failure tracking for the spec.

see mark().

Parameters:

  • spec – the spec whose failure indicators are being removed

  • force – True if the failure information should be cleared when a failure lock exists for the file, or False if the failure should not be cleared (e.g., it may be associated with a concurrent build)

clear_all() → None[source]

Force remove install failure tracking files.

dir: Path

Ensure a persistent location for dealing with parallel installation failures (e.g., across near-concurrent processes).

has_failed(spec: Spec) → bool[source]

Return True if the spec is marked as failed.

lock_taken(spec: Spec) → bool[source]

Return True if another process has a failure lock on the spec.

locker: SpecLocker

File for locking particular concrete spec hashes

mark(spec: Spec) → Lock[source]

Marks a spec as failing to install.

Parameters:

spec – spec that failed to install

persistent_mark(spec: Spec) → bool[source]

Determine if the spec has a persistent failure marking.

class spack.database.ForbiddenLock[source]

Bases: object

exception spack.database.ForbiddenLockError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when an upstream DB attempts to acquire a lock

spack.database.INDEX_JSON_FILE

File where the database is written

class spack.database.InstallRecord(spec: Spec, path: str | None, installed: bool, ref_count: int = 0, explicit: bool = False, installation_time: float | None = None, deprecated_for: str | None = None, in_buildcache: bool = False, origin: str | None = None)[source]

Bases: object

A record represents one installation in the DB.

The record keeps track of the spec for the installation, its install path, AND whether or not it is installed. We need the installed flag in case a user either:

  1. blew away a directory, or

  2. used spack uninstall -f to get rid of it

If, in either case, the package was removed but others still depend on it, we still need to track its spec, so we don’t actually remove from the database until a spec has no installed dependents left.

Parameters:

  • spec – spec tracked by the install record

  • path – path where the spec has been installed

  • installed – whether or not the spec is currently installed

  • ref_count (int) – number of specs that depend on this one

  • explicit (bool or None) – whether or not this spec was explicitly installed, or pulled-in as a dependency of something else

  • installation_time (datetime.datetime or None) – time of the installation

classmethod from_dict(spec, dictionary)[source]
install_type_matches(installed: InstallRecordStatus) → bool[source]
to_dict(include_fields=DEFAULT_INSTALL_RECORD_FIELDS)[source]
exception spack.database.InvalidDatabaseVersionError(database, expected, found)[source]

Bases: SpackError

Exception raised when the database metadata is newer than current Spack.

property database_version_message
class spack.database.LockConfiguration(enable: bool, database_timeout: int | None, package_timeout: int | None)[source]

Bases: NamedTuple

Data class to configure locks in Database objects

Parameters:

  • enable – whether to enable locks or not.

  • database_timeout – timeout for the database lock

  • package_timeout – timeout for the package lock

database_timeout: int | None

Alias for field number 1

enable: bool

Alias for field number 0

package_timeout: int | None

Alias for field number 2

exception spack.database.MissingDependenciesError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when DB cannot find records for dependencies

spack.database.NO_LOCK: LockConfiguration

Configure a database to avoid using locks

spack.database.NO_TIMEOUT: LockConfiguration

Configure the database to use locks without a timeout

exception spack.database.NoSuchSpecError(spec)[source]

Bases: KeyError

Raised when a spec is not found in the database.

class spack.database.NoUpstreamVisitor(upstream_hashes: Set[str], on_visit: Callable[[DependencySpec, bool], None])[source]

Bases: object

Gives edges to upstream specs, but does follow edges from upstream specs.

accept(item: EdgeAndDepth) → bool[source]
is_upstream(item: EdgeAndDepth) → bool[source]
neighbors(item: EdgeAndDepth)[source]
exception spack.database.NonConcreteSpecAddError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when attempting to add non-concrete spec to DB.

class spack.database.SpecLocker(lock_path: str | Path, default_timeout: float | None)[source]

Bases: object

Manages acquiring and releasing read or write locks on concrete specs.

clear(spec: Spec) → Tuple[bool, Lock | None][source]
clear_all(clear_fn: Callable[[Lock], Any] | None = None) → None[source]
has_lock(spec: Spec) → bool[source]

Returns True if the spec is already managed by this spec locker

lock(spec: Spec, timeout: float | None = None) → Lock[source]

Returns a lock on a concrete spec.

The lock is a byte range lock on the nth byte of a file.

The lock file is self.lock_path.

n is the sys.maxsize-bit prefix of the DAG hash. This makes likelihood of collision is very low AND it gives us readers-writer lock semantics with just a single lockfile, so no cleanup required.

locks: Dict[Tuple[str, str], Lock]
raw_lock(spec: Spec, timeout: float | None = None) → Lock[source]

Returns a raw lock for a Spec, but doesn’t keep track of it.

write_lock(spec: Spec) → Generator[SpecLocker, None, None][source]
exception spack.database.UpstreamDatabaseLockingError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when an operation would need to lock an upstream database

spack.database.failures_lock_path(root_dir: str | Path) → Path[source]

Returns the path of the failures lock file, given the root directory.

Parameters:

root_dir – root directory containing the database directory

spack.database.lock_configuration(configuration)[source]

Return a LockConfiguration from a spack.config.Configuration object.

spack.database.normalize_query(installed: bool | InstallRecordStatus) → InstallRecordStatus[source]
spack.database.prefix_lock_path(root_dir: str | Path) → Path[source]

Returns the path of the prefix lock file, given the root directory.

Parameters:

root_dir – root directory containing the database directory

spack.database.reader(version: StandardVersion) → Type[SpecfileReaderBase][source]

spack.dependency module

Data structures that represent Spack’s dependency relationships.

class spack.dependency.Dependency(pkg: Type[PackageBase], spec: Spec, depflag: int = dt.DEFAULT)[source]

Bases: object

Class representing metadata for a dependency on a package.

This class differs from spack.spec.DependencySpec because it represents metadata at the Package level. spack.spec.DependencySpec is a descriptor for an actual package configuration, while Dependency is a descriptor for a package’s dependency requirements.

A dependency is a requirement for a configuration of another package that satisfies a particular spec. The dependency can have types, which determine how that package configuration is required, e.g. whether it is required for building the package, whether it needs to be linked to, or whether it is needed at runtime so that Spack can call commands from it.

A package can also depend on another package with patches. This is for cases where the maintainers of one package also maintain special patches for their dependencies. If one package depends on another with patches, a special version of that dependency with patches applied will be built for use by the dependent package. The patches are included in the new version’s spec hash to differentiate it from unpatched versions of the same package, so that unpatched versions of the dependency package can coexist with the patched version.

depflag
property name: str

Get the name of the dependency package.

patches: Dict[Spec, List[Patch]]
pkg
spec

spack.deptypes module

Data structures that represent Spack’s edge types.

spack.deptypes.ALL: int

A flag with all dependency types set

spack.deptypes.ALL_FLAGS: Tuple[int, int, int, int]

An iterator of all flag components

spack.deptypes.ALL_TYPES: Tuple[Literal['build', 'link', 'run', 'test'], ...]

The types of dependency relationships that Spack understands.

spack.deptypes.DEFAULT: int

Default dependency type if none is specified

spack.deptypes.DEFAULT_TYPES: Tuple[Literal['build', 'link', 'run', 'test'], ...]

Default dependency type if none is specified

spack.deptypes.DepFlag

Type hint for the low-level dependency input (enum.Flag is too slow)

spack.deptypes.DepType

Individual dependency types

alias of Literal[‘build’, ‘link’, ‘run’, ‘test’]

spack.deptypes.DepTypes

Type hint for the high-level dependency input

alias of str | List[str] | Tuple[str, …]

spack.deptypes.NONE: int

A flag with no dependency types set

spack.deptypes.canonicalize(deptype: str | List[str] | Tuple[str, ...]) → int[source]

Convert deptype user input to a DepFlag, or raise ValueError.

Parameters:

deptype – string representing dependency type, or a list/tuple of such strings. Can also be the builtin function all or the string ‘all’, which result in a tuple of all dependency types known to Spack.

spack.deptypes.compatible(flag1: int, flag2: int) → bool[source]

Returns True if two depflags can be dependencies from a Spec to deps of the same name.

The only allowable separated dependencies are a build-only dependency, combined with a non-build dependency. This separates our two process spaces, build time and run time.

These dependency combinations are allowed:

  • single dep on name: [b], [l], [r], [bl], [br], [blr]

  • two deps on name: [b, l], [b, r], [b, lr]

but none of these make any sense:

  • two build deps: [b, b], [b, br], [b, bl], [b, blr]

  • any two deps that both have an l or an r, i.e. [l, l], [r, r], [l, r], [bl, l], [bl, r]

spack.deptypes.flag_from_string(s: str) → int[source]
spack.deptypes.flag_from_strings(deptype: Iterable[str]) → int[source]

Transform an iterable of deptype strings into a flag.

spack.deptypes.flag_to_chars(depflag: int) → str[source]

Create a string representing deptypes for many dependencies.

The string will be some subset of blrt, like bl ``, ``b t, or `` lr `` where each letter in blrt stands for build, link, run, and test (the dependency types).

For a single dependency, this just indicates that the dependency has the indicated deptypes. For a list of dependnecies, this shows whether ANY dependency in the list has the deptypes (so the deptypes are merged).

spack.deptypes.flag_to_string(x: int) → Literal['build', 'link', 'run', 'test'][source]
spack.deptypes.flag_to_tuple(x: int) → Tuple[Literal['build', 'link', 'run', 'test'], ...][source]

spack.directives module

This package contains directives that can be used within a package.

Directives are functions that can be called inside a package definition to modify the package, for example:

class OpenMpi(Package):
    depends_on("hwloc")
    provides("mpi")
    ...

provides and depends_on are spack directives.

The available directives are:

  • build_system

  • conflicts

  • depends_on

  • extends

  • license

  • patch

  • provides

  • resource

  • variant

  • version

  • requires

  • redistribute

They’re implemented as functions that return partial functions that are later executed with a package class as first argument:

@directive("example")
def example_directive(arg1, arg2):
    return partial(_execute_example_directive, arg1=arg1, arg2=arg2)

def _execute_example_directive(pkg, arg1, arg2):
    # modify pkg.example based on arg1 and arg2
exception spack.directives.DirectiveError(message: str, long_message: str | None = None)[source]

Bases: SpackError

This is raised when something is wrong with a package directive.

spack.directives.build_system(*values, **kwargs)[source]

Define the build system used by the package. This defines the build_system variant.

Example:

build_system("cmake", "autotools", "meson", default="cmake")
spack.directives.can_splice(target: str, *, when: str, match_variants: None | str | List[str] = None)[source]

Declare whether the package is ABI-compatible with another package and thus can be spliced into concrete versions of that package.

Parameters:

  • target – The spec that the current package is ABI-compatible with.

  • when – An anonymous spec constraining current package for when it is ABI-compatible with target.

  • match_variants – A list of variants that must match between target spec and current package, with special value * which matches all variants. Example: a json variant is defined on two packages, and they are ABI-compatible whenever they agree on the json variant (regardless of whether it is turned on or off). Note that this cannot be applied to multi-valued variants and multi-valued variants will be skipped by *.

spack.directives.conditional(*values: str | bool, when: Spec | str | bool | None = None)[source]

Conditional values that can be used in variant declarations.

spack.directives.conflicts(conflict_spec: str, when: Spec | str | bool | None = None, msg: str | None = None)[source]

Declare a conflict for a package.

A conflict is a spec that is known to be invalid. For example, a package that cannot build with GCC 14 and above can declare:

conflicts("%gcc@14:")

To express the same constraint only when the foo variant is activated:

conflicts("%gcc@14:", when="+foo")
Parameters:

  • conflict_spec – constraint defining the known conflict

  • when – optional condition that triggers the conflict

  • msg – optional user defined message

spack.directives.depends_on(spec: str, when: Spec | str | bool | None = None, type: Tuple[str, ...] | str = dt.DEFAULT_TYPES, *, patches: Callable[[Type[PackageBase] | Dependency], None] | str | List[Callable[[Type[PackageBase] | Dependency], None] | str] | None = None)[source]

Declare a dependency on another package.

Example:

depends_on("hwloc@2:", when="@1:", type="link")
Parameters:

  • spec – dependency spec

  • when – condition when this dependency applies

  • type – One or more of "build", "run", "test", or "link" (either a string or tuple). Defaults to ("build", "link").

  • patches – single result of patch() directive, a str to be passed to patch, or a list of these

spack.directives.extends(spec: str, when: Spec | str | bool | None = None, type: Tuple[str, ...] | str = ('build', 'run'), *, patches: Callable[[Type[PackageBase] | Dependency], None] | str | List[Callable[[Type[PackageBase] | Dependency], None] | str] | None = None)[source]

Same as depends_on(), but also adds this package to the extendee list. In case of Python, also adds a dependency on python-venv.

Note

Notice that the default type is ("build", "run"), which is different from depends_on() where the default is ("build", "link").

spack.directives.license(license_identifier: str, checked_by: str | List[str] | None = None, when: str | bool | None = None)[source]

Declare the license(s) the software is distributed under.

Parameters:

  • license_identifiers – SPDX identifier specifying the license(s) the software is distributed under.

  • checked_by – string or list of strings indicating which github user checked the license (if any).

  • when – A spec specifying when the license applies.

spack.directives.maintainers(*names: str)[source]

Declare the maintainers of a package.

Parameters:

names – GitHub username for the maintainer

spack.directives.patch(url_or_filename: str, level: int = 1, when: Spec | str | bool | None = None, working_dir: str = '.', reverse: bool = False, sha256: str | None = None, archive_sha256: str | None = None) → Callable[[Type[PackageBase] | Dependency], None][source]

Declare a patch to apply to package sources. A when spec can be provided to indicate that a particular patch should only be applied when the package’s spec meets certain conditions.

Example:

patch("foo.patch", when="@1.0.0:")
patch("https://example.com/foo.patch", sha256="...")
Parameters:

  • url_or_filename – url or relative filename of the patch

  • level – patch level (as in the patch shell command)

  • when – optional anonymous spec that specifies when to apply the patch

  • working_dir – dir to change to before applying

  • reverse – reverse the patch

  • sha256 – sha256 sum of the patch, used to verify the patch (only required for URL patches)

  • archive_sha256 – sha256 sum of the archive, if the patch is compressed (only required for compressed URL patches)

spack.directives.provides(*specs: str, when: Spec | str | bool | None = None)[source]

Declare that this package provides a virtual dependency.

If a package provides mpi, other packages can declare that they depend on mpi, and spack can use the providing package to satisfy the dependency.

Parameters:

  • *specs – virtual specs provided by this package

  • when – condition when this provides clause needs to be considered

spack.directives.redistribute(source: bool | None = None, binary: bool | None = None, when: Spec | str | bool | None = None)[source]

Declare that the package source and/or compiled binaries should not be redistributed.

By default, packages allow source/binary distribution (in mirrors/build caches resp.). This directive allows users to explicitly disable redistribution for specs.

spack.directives.requires(*requirement_specs: str, policy: str = 'one_of', when: str | None = None, msg: str | None = None)[source]

Declare that a spec must be satisfied for a package.

For instance, a package whose Fortran code can only be compiled with GCC can declare:

requires("%fortran=gcc")

A package that requires Apple-Clang on Darwin can declare instead:

requires("%apple-clang", when="platform=darwin", msg="Apple Clang is required on Darwin")
Parameters:

  • requirement_specs – spec expressing the requirement

  • policy – either "one_of" or "any_of". If "one_of", exactly one of the requirements must be satisfied. If "any_of", at least one of the requirements must be satisfied. Defaults to "one_of".

  • when – optional constraint that triggers the requirement. If None the requirement is applied unconditionally.

  • msg – optional user defined message

spack.directives.resource(*, name: str | None = None, destination: str = '', placement: str | None = None, when: Spec | str | bool | None = None, **kwargs)[source]

Declare an external resource to be fetched and staged when building the package. Based on the keywords present in the dictionary the appropriate FetchStrategy will be used for the resource. Resources are fetched and staged in their own folder inside spack stage area, and then moved into the stage area of the package that needs them.

Keyword Arguments:

  • name – name for the resource

  • when – condition defining when the resource is needed

  • destination – path, relative to the package stage area, to which resource should be moved

  • placement – optionally rename the expanded resource inside the destination directory

spack.directives.variant(name: str, default: bool | str | Tuple[str, ...] | None = None, description: str = '', values: Sequence | Callable[[Any], bool] | None = None, multi: bool | None = None, validator: Callable[[str, str, Tuple[Any, ...]], None] | None = None, when: str | bool | None = None, sticky: bool = False)[source]

Declare a variant for a package.

Packager can specify a default value as well as a text description.

Parameters:

  • name – Name of the variant

  • default – Default value for the variant, if not specified otherwise the default will be False for a boolean variant and ‘nothing’ for a multi-valued variant

  • description – Description of the purpose of the variant

  • values – Either a tuple of strings containing the allowed values, or a callable accepting one value and returning True if it is valid

  • multi – If False only one value per spec is allowed for this variant

  • validator – Optional group validator to enforce additional logic. It receives the package name, the variant name and a tuple of values and should raise an instance of SpackError if the group doesn’t meet the additional constraints

  • when – Optional condition on which the variant applies

  • sticky – The variant should not be changed by the concretizer to find a valid concrete spec

Raises:

spack.directives_meta.DirectiveError – If arguments passed to the directive are invalid

spack.directives.version(ver: str | int, checksum: str | None = None, *, preferred: bool | None = None, deprecated: bool | None = None, no_cache: bool | None = None, url: str | None = None, extension: str | None = None, expand: bool | None = None, fetch_options: dict | None = None, md5: str | None = None, sha1: str | None = None, sha224: str | None = None, sha256: str | None = None, sha384: str | None = None, sha512: str | None = None, git: str | None = None, commit: str | None = None, tag: str | None = None, branch: str | None = None, get_full_repo: bool | None = None, git_sparse_paths: List[str] | Callable[[PackageBase], List[str]] | None = None, submodules: Callable[[PackageBase], str | List[str] | bool] | bool | None = None, submodules_delete: bool | None = None, svn: str | None = None, hg: str | None = None, cvs: str | None = None, revision: str | None = None, date: str | None = None)[source]

Declare a version for a package with optional metadata for fetching its code.

Example:

version("2.1", sha256="...")
version("2.0", sha256="...", preferred=True)

Changed in version v2.3: The git_sparse_paths parameter was added.

spack.directives_meta module

class spack.directives_meta.DirectiveDictDescriptor(name: str)[source]

Bases: object

A descriptor that lazily executes directives on first access.

exception spack.directives_meta.DirectiveError(message: str, long_message: str | None = None)[source]

Bases: SpackError

This is raised when something is wrong with a package directive.

class spack.directives_meta.DirectiveMeta(name: str, bases: tuple, attr_dict: dict)[source]

Bases: type

Flushes the directives that were temporarily stored in the staging area into the package.

static pop_default_args() → dict[source]

Pop default arguments

static pop_when_constraint() → str[source]

Pop the last constraint from the context

static push_default_args(default_args: Dict[str, Any]) → None[source]

Push default arguments

static push_when_constraint(when_spec: str) → None[source]

Add a spec to the context constraints.

static register_directive(name: str, dicts: Tuple[str, ...]) → None[source]

Called by @directive to register relationships.

class spack.directives_meta.directive(dicts: Tuple[str, ...] | str = (), supports_when: bool = True, can_patch_dependencies: bool = False)[source]

Bases: object

spack.directives_meta.directive_names

Names of possible directives. This list is mostly populated using the @directive decorator. Some directives leverage others and in that case are not automatically added.

spack.directives_meta.get_spec(spec_str: str) → Spec[source]

Get a spec from the cache, or create it if not present.

spack.directory_layout module

class spack.directory_layout.DirectoryLayout(root: str, *, projections: Dict[str, str] | None = None, hash_length: int | None = None)[source]

Bases: object

A directory layout is used to associate unique paths with specs. Different installations are going to want different layouts for their install, and they can use this to customize the nesting structure of spack installs. The default layout is:

  • <install root>/

    • <platform-os-target>/

      • <compiler>-<compiler version>/

        • <name>-<version>-<hash>

The installation directory projections can be modified with the projections argument.

all_specs() → List[Spec][source]

Returns a list of all specs detected in self.root, detected by .spack directories. Their prefix is set to the directory containing the .spack directory. Note that these specs may follow a different layout than the current layout if it was changed after installation.

build_packages_path(spec: Spec) → str[source]
create_install_directory(spec: Spec) → None[source]
deprecated_file_path(deprecated_spec: Spec, deprecator_spec: Spec | None = None) → str[source]

Gets full path to spec file for deprecated spec

If the deprecator_spec is provided, use that. Otherwise, assume deprecated_spec is already deprecated and its prefix links to the prefix of its deprecator.

deprecated_for(specs: List[Spec]) → List[Tuple[Spec, Spec]][source]

Returns a list of tuples of specs (new, old) where new is deprecated for old

ensure_installed(spec: Spec) → None[source]

Throws InconsistentInstallDirectoryError if: 1. spec prefix does not exist 2. spec prefix does not contain a spec file, or 3. We read a spec with the wrong DAG hash out of an existing install directory.

env_metadata_path(spec: Spec) → str[source]
property hidden_file_regexes: Tuple[str]
metadata_path(spec: Spec) → str[source]
path_for_spec(spec: Spec) → str[source]

Return absolute path from the root to a directory for the spec.

read_spec(path: str) → Spec[source]

Read the contents of a file and parse them as a spec

relative_path_for_spec(spec: Spec) → str[source]
remove_install_directory(spec: Spec, deprecated: bool = False) → None[source]

Removes a prefix and any empty parent directories from the root. Raised RemoveFailedError if something goes wrong.

spec_file_path(spec: Spec) → str[source]

Gets full path to spec file

write_host_environment(spec: Spec) → None[source]

The host environment is a json file with os, kernel, and spack versioning. We use it in the case that an analysis later needs to easily access this information.

write_spec(spec: Spec, path: str) → None[source]

Write a spec out to a file.

exception spack.directory_layout.DirectoryLayoutError(message, long_msg=None)[source]

Bases: SpackError

Superclass for directory layout errors.

exception spack.directory_layout.ExtensionAlreadyInstalledError(spec, ext_spec)[source]

Bases: DirectoryLayoutError

Raised when an extension is added to a package that already has it.

exception spack.directory_layout.ExtensionConflictError(spec, ext_spec, conflict)[source]

Bases: DirectoryLayoutError

Raised when an extension is added to a package that already has it.

exception spack.directory_layout.InconsistentInstallDirectoryError(message, long_msg=None)[source]

Bases: DirectoryLayoutError

Raised when a package seems to be installed to the wrong place.

exception spack.directory_layout.InvalidDirectoryLayoutParametersError(message, long_msg=None)[source]

Bases: DirectoryLayoutError

Raised when a invalid directory layout parameters are supplied

exception spack.directory_layout.InvalidExtensionSpecError(message, long_msg=None)[source]

Bases: DirectoryLayoutError

Raised when an extension file has a bad spec in it.

exception spack.directory_layout.RemoveFailedError(installed_spec, prefix, error)[source]

Bases: DirectoryLayoutError

Raised when a DirectoryLayout cannot remove an install prefix.

exception spack.directory_layout.SpecReadError(message, long_msg=None)[source]

Bases: DirectoryLayoutError

Raised when directory layout can’t read a spec.

spack.directory_layout.specs_from_metadata_dirs(root: str) → List[Spec][source]

spack.enums module

Enumerations used throughout Spack

class spack.enums.ConfigScopePriority(*values)[source]

Bases: IntEnum

Priorities of the different kind of config scopes used by Spack

COMMAND_LINE
CONFIG_FILES
CUSTOM
DEFAULTS
ENVIRONMENT
ENVIRONMENT_SPEC_GROUPS
class spack.enums.InstallRecordStatus(*values)[source]

Bases: Flag

Enum flag to facilitate querying status from the DB

ANY
DEPRECATED
INSTALLED
MISSING
class spack.enums.PropagationPolicy(*values)[source]

Bases: Enum

Enum to specify the behavior of a propagated dependency

NONE
PREFERENCE

spack.error module

exception spack.error.CompilerError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised if something goes wrong when probing or querying a compiler.

exception spack.error.ConfigError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Superclass for all Spack config related errors.

exception spack.error.FetchError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Superclass for fetch-related errors.

exception spack.error.InstallError(message, long_msg=None, pkg=None)[source]

Bases: SpackError

Raised when something goes wrong during install or uninstall.

The error can be annotated with a pkg attribute to allow the caller to get the package for which the exception was raised.

exception spack.error.InvalidVirtualOnEdgeError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when an edge requires a virtual that does not exist in the repository.

exception spack.error.MirrorError(msg, long_msg=None)[source]

Bases: SpackError

Superclass of all mirror-creation related errors.

exception spack.error.NoChecksumException(path, size, contents, algorithm, expected, computed)[source]

Bases: SpackError

Raised if file fails checksum verification.

exception spack.error.NoHeadersError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when package headers are requested but cannot be found

exception spack.error.NoLibrariesError(message_or_name, prefix=None)[source]

Bases: SpackError

Raised when package libraries are requested but cannot be found

exception spack.error.NoSuchPatchError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when a patch file doesn’t exist.

exception spack.error.NoSuchSpecFileError(message: str, long_message: str | None = None)[source]

Bases: SpecFilenameError

Raised when a spec file doesn’t exist.

exception spack.error.NoURLError(cls)[source]

Bases: PackageError

Raised when someone tries to build a URL for a package with no URLs.

exception spack.error.PackageError(message, long_msg=None)[source]

Bases: SpackError

Raised when something is wrong with a package definition.

exception spack.error.PatchDirectiveError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when the wrong arguments are suppled to the patch directive.

exception spack.error.PatchLookupError(message: str, long_message: str | None = None)[source]

Bases: NoSuchPatchError

Raised when a patch file cannot be located from sha256.

spack.error.SHOW_BACKTRACE

whether to show a backtrace when an error is printed, enabled with --backtrace.

exception spack.error.SpackAPIWarning[source]

Bases: UserWarning

Warning that formats with file and line number.

exception spack.error.SpackError(message: str, long_message: str | None = None)[source]

Bases: Exception

This is the superclass for all Spack errors. Subclasses can be found in the modules they have to do with.

die()[source]
property long_message
print_context()[source]

Print extended debug information about this exception.

This is usually printed when the top-level Spack error handler calls die(), but it can be called separately beforehand if a lower-level error handler needs to print error context and continue without raising the exception to the top level.

exception spack.error.SpecError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Superclass for all errors that occur while constructing specs.

exception spack.error.SpecFilenameError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when a spec file name is invalid.

exception spack.error.SpecSyntaxError[source]

Bases: Exception

Base class for Spec syntax errors

exception spack.error.StopPhase(message: str, long_message: str | None = None)[source]

Bases: SpackError

Pickle-able exception to control stopped builds.

exception spack.error.UnsatisfiableSpecError(provided, required, constraint_type)[source]

Bases: SpecError

Raised when a spec conflicts with package constraints.

For original concretizer, provide the requirement that was violated when raising.

exception spack.error.UnsupportedPlatformError(message)[source]

Bases: SpackError

Raised by packages when a platform is not supported

spack.error.debug

at what level we should write stack traces or short error messages this is module-scoped because it needs to be set very early

spack.extensions module

Service functions and classes to implement the hooks for Spack’s command extensions.

exception spack.extensions.ExtensionNamingError(path)[source]

Bases: SpackError

Exception class thrown when a configured extension does not follow the expected naming convention.

spack.extensions.ensure_extension_loaded(extension, *, path)[source]
spack.extensions.extension_name(path)[source]

Returns the name of the extension in the path passed as argument.

Parameters:

path (str) – path where the extension resides

Returns:

The extension name.

Raises:

ExtensionNamingError – if path does not match the expected format for a Spack command extension.

spack.extensions.extension_paths_from_entry_points() → List[str][source]

Load extensions from a Python package’s entry points.

A python package can register entry point metadata so that Spack can find its extensions by adding the following to the project’s pyproject.toml:

[project.entry-points."spack.extensions"]
baz = "baz:get_spack_extensions"

The function get_spack_extensions returns paths to the package’s spack extensions

This function assumes that the state of entry points doesn’t change from the first time it’s called. E.g., it doesn’t support any new installation of packages between two calls.

spack.extensions.get_command_paths()[source]

Return the list of paths where to search for command files.

spack.extensions.get_extension_paths()[source]

Return the list of canonicalized extension paths from config:extensions.

spack.extensions.get_module(cmd_name)[source]

Imports the extension module for a particular command name and returns it.

Parameters:

cmd_name (str) – name of the command for which to get a module (contains -, not _).

spack.extensions.get_template_dirs()[source]

Returns the list of directories where to search for templates in extensions.

spack.extensions.load_command_extension(command, path)[source]

Loads a command extension from the path passed as argument.

Parameters:

  • command (str) – name of the command (contains -, not _).

  • path (str) – base path of the command extension

Returns:

A valid module if found and loadable; None if not found. Module

loading exceptions are passed through.

spack.extensions.load_extension(name: str) → str[source]

Loads a single extension into the spack.extensions package.

Parameters:

name – name of the extension

spack.extensions.path_for_extension(target_name: str, *, paths: List[str]) → str[source]

Return the test root dir for a given extension.

Parameters:

  • target_name (str) – name of the extension to test

  • *paths – paths where the extensions reside

Returns:

Root directory where tests should reside or None

spack.externals module

This module turns the configuration data in the packages section into a list of concrete specs.

This is mainly done by the ExternalSpecsParser class, which is responsible for:

  1. Transforming an intermediate representation of the YAML configuration into a set of nodes

  2. Ensuring the dependency specifications are not ambiguous

  3. Inferring missing information about the external specs (e.g. architecture, deptypes)

  4. Wiring up the external specs to their dependencies

The helper function extract_dicts_from_configuration is used to transform the configuration into the intermediate representation.

class spack.externals.DependencyDict[source]

Bases: TypedDict

deptypes: str | List[str] | Tuple[str, ...]
id: str
spec: str
virtuals: str
exception spack.externals.DuplicateExternalError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when a duplicate external is detected.

exception spack.externals.ExternalDependencyError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when a dependency on an external package is specified wrongly.

class spack.externals.ExternalDict[source]

Bases: TypedDict

Dictionary representation of an external spec.

This representation mostly follows the one used in the configuration files, with a few exceptions needed to support specific features.

dependencies: List[DependencyDict]
extra_attributes: Dict[str, Any]
id: str
modules: List[str]
prefix: str
required_target: str
spec: str
class spack.externals.ExternalSpecAndConfig(spec, config)[source]

Bases: NamedTuple

config: ExternalDict

Alias for field number 1

spec: Spec

Alias for field number 0

exception spack.externals.ExternalSpecError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when a dependency on an external package is specified wrongly.

class spack.externals.ExternalSpecsParser(external_dicts: List[ExternalDict], *, complete_node: Callable[[Spec], None] = complete_variants_and_architecture, allow_nonexisting: bool = True)[source]

Bases: object

Transforms a list of external dicts into a list of specs.

all_specs() → List[Spec][source]

Returns all the external specs.

get_specs_for_package(package_name: str) → List[Spec][source]

Returns the external specs for a given package name.

nodes: List[Spec]
query(query: str | Spec) → List[Spec][source]

Returns the external specs matching a query spec.

specs_by_external_id: Dict[str, ExternalSpecAndConfig]
specs_by_name: Dict[str, List[ExternalSpecAndConfig]]
spack.externals.complete_architecture(node: Spec) → None[source]

Completes a node with architecture information.

Undefined targets are set to the default host target family (e.g. x86_64). The operating system and platform are set based on the current host.

spack.externals.complete_variants_and_architecture(node: Spec) → None[source]

Completes a node with variants and architecture information.

Architecture is completed first, delegating to complete_architecture. Variants are then added to the node, using their default value.

spack.externals.external_spec(config: ExternalDict) → Spec[source]

Returns an external spec from a dictionary representation.

spack.externals.extract_dicts_from_configuration(packages_yaml) → List[ExternalDict][source]

Transforms the packages.yaml configuration into a list of external dictionaries.

The default required target is extracted from packages:all:require, if present. Any package-specific required target overrides the default.

spack.externals.node_from_dict(external_dict: ExternalDict) → Spec[source]

Returns an external spec node from a dictionary representation.

spack.fetch_strategy module

Fetch strategies are used to download source code into a staging area in order to build it. They need to define the following methods:

fetch()

This should attempt to download/check out source from somewhere.

check()

Apply a checksum to the downloaded source code, e.g. for an archive. May not do anything if the fetch method was safe to begin with.

expand()

Expand (e.g., an archive) downloaded file to source, with the standard stage source path as the destination directory.

reset()

Restore original state of downloaded code. Used by clean commands. This may just remove the expanded source and re-expand an archive, or it may run something like git reset --hard.

archive()

Archive a source directory, e.g. for creating a mirror.

class spack.fetch_strategy.BundleFetchStrategy(**kwargs)[source]

Bases: FetchStrategy

Fetch strategy associated with bundle, or no-code, packages.

Having a basic fetch strategy is a requirement for executing post-install hooks. Consequently, this class provides the API but does little more than log messages.

TODO: Remove this class by refactoring resource handling and the link between composite stages and composite fetch strategies (see #11981).

property cachable

Report False as there is no code to cache.

fetch()[source]

Simply report success – there is no code to fetch.

mirror_id()[source]

BundlePackages don’t have a mirror id.

source_id()[source]

BundlePackages don’t have a source id.

url_attr: str | None

There is no associated URL keyword in version() for no-code packages but this property is required for some strategy-related functions (e.g., check_pkg_attributes).

class spack.fetch_strategy.CacheURLFetchStrategy(*, url: str, checksum: str | None = None, **kwargs)[source]

Bases: URLFetchStrategy

The resource associated with a cache URL may be out of date.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

exception spack.fetch_strategy.ChecksumError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised when archive fails to checksum.

class spack.fetch_strategy.CvsFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a CVS repository. Use like this in a package:

version("name", cvs=":pserver:anonymous@www.example.com:/cvsroot%module=modulename")

Optionally, you can provide a branch and/or a date for the URL:

version(
    "name",
    cvs=":pserver:anonymous@www.example.com:/cvsroot%module=modulename",
    branch="branchname", date="date"
)

Repositories are checked out into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

property cvs
fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str]
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str.

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

exception spack.fetch_strategy.ExtrapolationError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised when we can’t extrapolate a version for a package.

exception spack.fetch_strategy.FailedDownloadError(*exceptions: Exception)[source]

Bases: FetchError

Raised when a download fails.

class spack.fetch_strategy.FetchAndVerifyExpandedFile(url, archive_sha256: str, expanded_sha256: str)[source]

Bases: URLFetchStrategy

Fetch strategy that verifies the content digest during fetching, as well as after expanding it.

expand()[source]

Verify checksum after expanding the archive.

class spack.fetch_strategy.FetchProgress(total_bytes: int | None = None, enabled: bool = True, get_time: Callable[[], float] = time.time)[source]

Bases: object

advance(num_bytes: int, out=sys.stdout) → None[source]
current_bytes

Number of bytes downloaded so far.

delta

Delta time between progress prints

enabled

Whether to print progress information.

classmethod from_headers(headers: Mapping[str, str], enabled: bool = True, get_time: Callable[[], float] = time.time) → FetchProgress[source]

Create a FetchProgress instance from HTTP headers.

get_time

Function to get the current time.

index

Index of spinner character to print (used if total bytes is unknown)

last_printed

Time of last progress print to limit output

print(final: bool = False, out=sys.stdout) → None[source]
spinner

Characters to rotate in the spinner.

start_time

Time of start of download

total_bytes

Total number of bytes to download, if known.

class spack.fetch_strategy.FetchStrategy(**kwargs)[source]

Bases: object

Superclass of all fetch strategies.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

check()[source]

Checksum the archive fetched by this FetchStrategy.

expand()[source]

Expand the downloaded archive into the stage source path.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

classmethod matches(args)[source]

Predicate that matches fetch strategies to arguments of the version directive.

Parameters:

args – arguments of the version directive

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str]
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

set_package(package)[source]
source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str.

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

exception spack.fetch_strategy.FetcherConflict(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised for packages with invalid fetch attributes.

class spack.fetch_strategy.FsCache(root)[source]

Bases: FsCacheBase

destroy()[source]
fetcher(target_path: str, digest: str | None, **kwargs) → CacheURLFetchStrategy[source]
store(fetcher, relative_dest)[source]
class spack.fetch_strategy.FsCacheBase(root)[source]

Bases: object

store(fetcher, relative_dest)[source]
class spack.fetch_strategy.GCSFetchStrategy(*, url: str, checksum: str | None = None, **kwargs)[source]

Bases: URLFetchStrategy

FetchStrategy that pulls from a GCS bucket.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class spack.fetch_strategy.GitFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a git repository. Use like this in a package:

version("name", git="https://github.com/project/repo.git")

Optionally, you can provide a branch, or commit to check out, e.g.:

version("1.1", git="https://github.com/project/repo.git", tag="v1.1")

You can use these three optional attributes in addition to git:

  • branch: Particular branch to build from (default is the repository’s default branch)

  • tag: Particular tag to check out

  • commit: Particular commit hash in the repo

Repositories are cloned into the standard stage source path directory.

bare_clone(dest: str) → None[source]

Execute a bare clone for metadata only

Requires a destination since bare cloning does not provide source and shouldn’t be used for staging.

branch: str | None
property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

commit: str | None
fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

property git
property git_version
mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str]
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str.

submodule_operations()[source]
tag: str | None
url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

static version_from_git(git_exe)[source]

Given a git executable, return the Version (this will fail if the output cannot be parsed into a valid Version).

class spack.fetch_strategy.GoFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that employs the go get infrastructure.

Use like this in a package:

version("name", go="github.com/monochromegane/the_platinum_searcher/...")

Go get does not natively support versions, they can be faked with git.

The fetched source will be moved to the standard stage sourcepath directory during the expand step.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

expand()[source]

Expand the downloaded archive into the stage source path.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

property go
property go_version
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class spack.fetch_strategy.HgFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a Mercurial repository. Use like this in a package:

version("name", hg="https://jay.grs.rwth-aachen.de/hg/lwm2")

Optionally, you can provide a branch, or revision to check out, e.g.:

version("torus", hg="https://jay.grs.rwth-aachen.de/hg/lwm2", branch="torus")

You can use the optional revision attribute to check out a branch, tag, or particular revision in hg. To prevent non-reproducible builds, using a moving target like a branch is discouraged.

  • revision: Particular revision, branch, or tag.

Repositories are cloned into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

property hg

Returns: Executable: the hg executable

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str]
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str.

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

exception spack.fetch_strategy.InvalidArgsError(pkg=None, version=None, **args)[source]

Bases: FetchError

Raised when a version can’t be deduced from a set of arguments.

exception spack.fetch_strategy.NoArchiveFileError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised when an archive file is expected but none exists.

exception spack.fetch_strategy.NoCacheError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised when there is no cached archive for a package.

exception spack.fetch_strategy.NoDigestError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised after attempt to checksum when URL has no digest.

exception spack.fetch_strategy.NoStageError(method)[source]

Bases: FetchError

Raised when fetch operations are called before set_stage().

class spack.fetch_strategy.OCIRegistryFetchStrategy(*, url: str, checksum: str | None = None, **kwargs)[source]

Bases: URLFetchStrategy

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

class spack.fetch_strategy.S3FetchStrategy(*, url: str, checksum: str | None = None, **kwargs)[source]

Bases: URLFetchStrategy

FetchStrategy that pulls from an S3 bucket.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class spack.fetch_strategy.SvnFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a subversion repository. Use like this in a package:

version("name", svn="http://www.example.com/svn/trunk")

Optionally, you can provide a revision for the URL:

version("name", svn="http://www.example.com/svn/trunk", revision="1641")

Repositories are checked out into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str]
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str.

property svn
url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class spack.fetch_strategy.URLFetchStrategy(*, url: str, checksum: str | None = None, **kwargs)[source]

Bases: FetchStrategy

URLFetchStrategy pulls source code from a URL for an archive, check the archive against a checksum, and decompresses the archive.

The destination for the resulting file(s) is the standard stage path.

archive(destination)[source]

Just moves this archive to the destination.

property archive_file

Path to the source archive within this stage directory.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

property candidate_urls
check()[source]

Check the downloaded archive against a checksum digest. No-op if this stage checks code out of a repository.

property curl: Executable
digest: str | None
expand()[source]

Expand the downloaded archive into the stage source path.

expand_archive: bool
extension: str | None
extra_options: dict
fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str]
reset()[source]

Removes the source path if it exists, then re-expands the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str.

url_attr: str | None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class spack.fetch_strategy.VCSFetchStrategy(**kwargs)[source]

Bases: FetchStrategy

Superclass for version control system fetch strategies.

Like all fetchers, VCS fetchers are identified by the attributes passed to the version directive. The optional_attrs for a VCS fetch strategy represent types of revisions, e.g. tags, branches, commits, etc.

The required attributes (git, svn, etc.) are used to specify the URL and to distinguish a VCS fetch strategy from a URL fetch strategy.

archive(destination, *, exclude: str | None = None)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

check()[source]

Checksum the archive fetched by this FetchStrategy.

expand()[source]

Expand the downloaded archive into the stage source path.

spack.fetch_strategy.all_strategies: List[Type[FetchStrategy]]

List of all fetch strategies, created by FetchStrategy metaclass.

spack.fetch_strategy.check_pkg_attributes(pkg)[source]

Find ambiguous top-level fetch attributes in a package.

Currently this only ensures that two or more VCS fetch strategies are not specified at once.

spack.fetch_strategy.fetcher(cls)[source]

Decorator used to register fetch strategies.

spack.fetch_strategy.for_package_version(pkg, version=None)[source]
spack.fetch_strategy.from_kwargs(**kwargs) → FetchStrategy[source]

Construct an appropriate FetchStrategy from the given keyword arguments.

Parameters:

**kwargs – dictionary of keyword arguments, e.g. from a version() directive in a package.

Returns:

The fetch strategy that matches the args, based on attribute names (e.g., git, hg, etc.)

Raises:

spack.error.FetchError – If no fetch_strategy matches the args.

spack.fetch_strategy.from_list_url(pkg)[source]

If a package provides a URL which lists URLs for resources by version, this can can create a fetcher for a URL discovered for the specified package’s version.

spack.fetch_strategy.from_url(url: str) → URLFetchStrategy[source]

Given a URL, find an appropriate fetch strategy for it. Currently just gives you a URLFetchStrategy that uses curl.

TODO: make this return appropriate fetch strategies for other types of URLs.

spack.fetch_strategy.from_url_scheme(url: str, **kwargs) → FetchStrategy[source]

Finds a suitable FetchStrategy by matching its url_attr with the scheme in the given url.

spack.fetch_strategy.stable_target(fetcher)[source]

Returns whether the fetcher target is expected to have a stable checksum. This is only true if the target is a preexisting archive file.

spack.fetch_strategy.verify_checksum(file: str, digest: str, url: str, effective_url: str | None) → None[source]

spack.filesystem_view module

class spack.filesystem_view.FilesystemView(root: str, layout: DirectoryLayout, *, projections: Dict | None = None, ignore_conflicts: bool = False, verbose: bool = False, link_type: Literal['hardlink', 'hard', 'copy', 'relocate', 'add', 'symlink', 'soft'] = 'symlink', link_dirs: bool = False)[source]

Bases: object

Governs a filesystem view that is located at certain root-directory.

Packages are linked from their install directories into a common file hierarchy.

In distributed filesystems, loading each installed package separately can lead to slow-downs due to too many directories being traversed. This can be circumvented by loading all needed modules into a common directory structure.

add_specs(*specs: Spec, **kwargs) → None[source]

Add given specs to view.

Should accept with_dependencies as keyword argument (default True) to indicate whether or not dependencies should be activated as well.

Should except an exclude keyword argument containing a list of regexps that filter out matching spec names.

This method should make use of activate_standalone.

add_standalone(spec: Spec) → bool[source]

Add (link) a standalone package into this view.

check_added(spec: Spec) → bool[source]

Check if the given concrete spec is active in this view.

get_all_specs() → List[Spec][source]

Get all specs currently active in this view.

get_projection_for_spec(spec: Spec) → str[source]

Get the projection in this view for a spec.

get_spec(spec: Spec) → Spec | None[source]

Return the actual spec linked in this view (i.e. do not look it up in the database by name).

spec can be a name or a spec from which the name is extracted.

As there can only be a single version active for any spec the name is enough to identify the spec in the view.

If no spec is present, returns None.

print_status(*specs: Spec, **kwargs) → None[source]

Print a short summary about the given specs, detailing whether..

  • ..they are active in the view.

  • ..they are active but the activated version differs.

  • ..they are not active in the view.

Takes with_dependencies keyword argument so that the status of dependencies is printed as well.

remove_specs(*specs: Spec, **kwargs) → None[source]

Removes given specs from view.

Should accept with_dependencies as keyword argument (default True) to indicate whether or not dependencies should be deactivated as well.

Should accept with_dependents as keyword argument (default True) to indicate whether or not dependents on the deactivated specs should be removed as well.

Should except an exclude keyword argument containing a list of regexps that filter out matching spec names.

This method should make use of deactivate_standalone.

remove_standalone(spec: Spec) → None[source]

Remove (unlink) a standalone package from this view.

class spack.filesystem_view.YamlFilesystemView(root: str, layout: DirectoryLayout, *, projections: Dict | None = None, ignore_conflicts: bool = False, verbose: bool = False, link_type: Literal['hardlink', 'hard', 'copy', 'relocate', 'add', 'symlink', 'soft'] = 'symlink')[source]

Bases: FilesystemView

Filesystem view to work with a yaml based directory layout.

add_specs(*specs, **kwargs)[source]

Add given specs to view.

Should accept with_dependencies as keyword argument (default True) to indicate whether or not dependencies should be activated as well.

Should except an exclude keyword argument containing a list of regexps that filter out matching spec names.

This method should make use of activate_standalone.

add_standalone(spec)[source]

Add (link) a standalone package into this view.

check_added(spec)[source]

Check if the given concrete spec is active in this view.

clean()[source]
get_all_specs()[source]

Get all specs currently active in this view.

get_conflicts(*specs)[source]

Return list of tuples (<spec>, <spec in view>) where the spec active in the view differs from the one to be activated.

get_path_meta_folder(spec)[source]

Get path to meta folder for either spec or spec name.

get_projection_for_spec(spec)[source]

Return the projection for a spec in this view.

Relies on the ordering of projections to avoid ambiguity.

get_spec(spec)[source]

Return the actual spec linked in this view (i.e. do not look it up in the database by name).

spec can be a name or a spec from which the name is extracted.

As there can only be a single version active for any spec the name is enough to identify the spec in the view.

If no spec is present, returns None.

merge(spec, ignore=None)[source]
print_conflict(spec_active, spec_specified, level='error')[source]

Singular print function for spec conflicts.

print_status(*specs, **kwargs)[source]

Print a short summary about the given specs, detailing whether..

  • ..they are active in the view.

  • ..they are active but the activated version differs.

  • ..they are not active in the view.

Takes with_dependencies keyword argument so that the status of dependencies is printed as well.

read_projections()[source]
remove_files(files)[source]
remove_specs(*specs, **kwargs)[source]

Removes given specs from view.

Should accept with_dependencies as keyword argument (default True) to indicate whether or not dependencies should be deactivated as well.

Should accept with_dependents as keyword argument (default True) to indicate whether or not dependents on the deactivated specs should be removed as well.

Should except an exclude keyword argument containing a list of regexps that filter out matching spec names.

This method should make use of deactivate_standalone.

remove_standalone(spec)[source]

Remove (unlink) a standalone package from this view.

unmerge(spec, ignore=None)[source]
write_projections()[source]

spack.graph module

Functions for graphing DAGs of dependencies.

This file contains code for graphing DAGs of software packages (i.e. Spack specs). There are two main functions you probably care about:

graph_ascii() will output a colored graph of a spec in ascii format, kind of like the graph git shows with git log --graph, e.g.

o  mpileaks
|\
| |\
| o |  callpath
|/| |
| |\|
| |\ \
| | |\ \
| | | | o  adept-utils
| |_|_|/|
|/| | | |
o | | | |  mpi
 / / / /
| | o |  dyninst
| |/| |
|/|/| |
| | |/
| o |  libdwarf
|/ /
o |  libelf
 /
o  boost

graph_dot() will output a graph of a spec (or multiple specs) in dot format.

class spack.graph.AsciiGraph[source]

Bases: object

write(spec, color=None, out=None)[source]

Write out an ascii graph of the provided spec.

Parameters:

  • spec – spec to graph. This only handles one spec at a time.

  • out – file object to write out to (default is sys.stdout)

  • color – whether to write in color. Default is to autodetect based on output file.

class spack.graph.DAGWithDependencyTypes[source]

Bases: DotGraphBuilder

DOT graph with link,run nodes grouped together and edges colored according to the dependency types.

edge_entry(edge)[source]

Return a tuple of (parent_id, child_id, edge_options)

node_entry(node)[source]

Return a tuple of (node_id, node_options)

visit(edge)[source]

Visit an edge and builds up entries to render the graph

class spack.graph.DotGraphBuilder[source]

Bases: object

Visit edges of a graph a build DOT options for nodes and edges

context()[source]

Return the context to be used to render the DOT graph template

edge_entry(edge: DependencySpec) → Tuple[str, str, str][source]

Return a tuple of (parent_id, child_id, edge_options)

node_entry(node: Spec) → Tuple[str, str][source]

Return a tuple of (node_id, node_options)

render() → str[source]

Return a string with the output in DOT format

visit(edge: DependencySpec)[source]

Visit an edge and builds up entries to render the graph

class spack.graph.SimpleDAG[source]

Bases: DotGraphBuilder

Simple DOT graph, with nodes colored uniformly and edges without properties

edge_entry(edge)[source]

Return a tuple of (parent_id, child_id, edge_options)

node_entry(node)[source]

Return a tuple of (node_id, node_options)

class spack.graph.StaticDag[source]

Bases: DotGraphBuilder

DOT graph for possible dependencies

edge_entry(edge)[source]

Return a tuple of (parent_id, child_id, edge_options)

node_entry(node)[source]

Return a tuple of (node_id, node_options)

spack.graph.find(seq, predicate)[source]

Find index in seq for which predicate is True.

Searches the sequence and returns the index of the element for which the predicate evaluates to True. Returns -1 if the predicate does not evaluate to True for any element in seq.

spack.graph.graph_ascii(spec, node='o', out=None, debug=False, indent=0, color=None, depflag: int = dt.ALL)[source]
spack.graph.graph_dot(specs: List[Spec], builder: DotGraphBuilder | None = None, depflag: int = dt.ALL, out: TextIO | None = None)[source]

DOT graph of the concrete specs passed as input.

Parameters:

  • specs – specs to be represented

  • builder – builder to use to render the graph

  • depflag – dependency types to consider

  • out – optional output stream. If None sys.stdout is used

spack.graph.static_graph_dot(specs: List[Spec], depflag: int = dt.ALL, out: TextIO | None = None)[source]

Static DOT graph with edges to all possible dependencies.

Parameters:

  • specs – abstract specs to be represented

  • depflag – dependency types to consider

  • out – optional output stream. If None sys.stdout is used

spack.hash_types module

Definitions that control how Spack creates Spec hashes.

class spack.hash_types.SpecHashDescriptor(depflag: int, package_hash: bool, name: str, override: Callable[[Spec], str] | None = None)[source]

Bases: object

This class defines how hashes are generated on Spec objects.

Spec hashes in Spack are generated from a serialized (e.g., with YAML) representation of the Spec graph. The representation may only include certain dependency types, and it may optionally include a canonicalized hash of the package.py for each node in the graph.

We currently use different hashes for different use cases.

attr
depflag
name
override
package_hash
spack.hash_types.dag_hash

The DAG hash includes all inputs that can affect how a package is built.

spack.hash_types.package_hash

Package hash used as part of dag hash

spack.install_test module

class spack.install_test.PackageTest(pkg: PackageBase)[source]

Bases: object

The class that manages stand-alone (post-install) package tests.

add_failure(exception: Exception, msg: str)[source]

Add the failure details to the current list.

archive_install_test_log(dest_dir: str)[source]
property archived_install_test_log: str
counts: Counter
property logger: nixlog | winlog | None

The current logger or, if none, sets to one.

parts() → int[source]

The total number of (checked) test parts.

phase_tests(builder, phase_name: str, method_names: List[str])[source]

Execute the builder’s package phase-time tests.

Parameters:

  • builder – builder for package being tested

  • phase_name – the name of the build-time phase (e.g., build, install)

  • method_names – phase-specific callback method names

pkg_id: str
print_log_path()[source]

Print the test log file path.

ran_tests() → bool[source]

True if ran tests, False otherwise.

stand_alone_tests(kwargs, timeout: int | None = None) → None[source]

Run the package’s stand-alone tests.

Parameters:

kwargs (dict) – arguments to be used by the test process

status(name: str, status: TestStatus, msg: str | None = None)[source]

Track and print the test status for the test part name.

summarize()[source]

Collect test results summary lines for this spec.

test_failures: List[Tuple[BaseException, str]]
test_log_file: str
test_logger(verbose: bool = False, externals: bool = False)[source]

Context manager for setting up the test logger

Parameters:

  • verbose – Display verbose output, including echoing to stdout, otherwise suppress it

  • externals – True for performing tests if external package, False to skip them

test_parts: OrderedDict[str, TestStatus]
write_tested_status()[source]

Write the overall status to the tested file.

If there any test part failures, then the tests failed. If all test parts are skipped, then the tests were skipped. If any tests passed then the tests passed; otherwise, there were not tests executed.

exception spack.install_test.SkipTest[source]

Bases: Exception

Raised when a test (part) is being skipped.

exception spack.install_test.TestFailure(failures: List[Tuple[BaseException, str]])[source]

Bases: SpackError

Raised when package tests have failed for an installation.

spack.install_test.TestFailureType

Stand-alone test failure info type

alias of Tuple[BaseException, str]

class spack.install_test.TestStatus(*values)[source]

Bases: Enum

Names of different stand-alone test states.

FAILED
NO_TESTS
PASSED
SKIPPED
lower()[source]
class spack.install_test.TestSuite(specs: Iterable[Spec], alias: str | None = None)[source]

Bases: object

The class that manages specs for spack test run execution.

property content_hash: str

The hash used to uniquely identify the test suite.

counts: Counter
property current_test_cache_dir: str

Path to the test stage directory where the current spec’s cached build-time files were automatically copied.

Raises:

TestSuiteSpecError – If there is no spec being tested

property current_test_data_dir: str

Path to the test stage directory where the current spec’s custom package (data) files were automatically copied.

Raises:

TestSuiteSpecError – If there is no spec being tested

ensure_stage() → None[source]

Ensure the test suite stage directory exists.

static from_dict(d)[source]

Instantiates a TestSuite based on a dictionary specs and an optional alias:

  • specs: list of the test suite’s specs in dictionary form

  • alias: the test suite alias

Returns:

Instance created from the specs

Return type:

TestSuite

static from_file(filename: str) → TestSuite[source]

Instantiate a TestSuite using the specs and optional alias provided in the given file.

Parameters:

filename – The path to the JSON file containing the test suite specs and optional alias.

Raises:

BaseException – sjson.SpackJSONError if problem parsing the file

log_file_for_spec(spec: Spec) → Prefix[source]

The test log file path for the provided spec.

Parameters:

spec – instance of the spec under test

property name: str

The name (alias or, if none, hash) of the test suite.

reports: List[RequestRecord]
property results_file: Prefix

The path to the results summary file.

property stage: Prefix

The root test suite stage directory

test_dir_for_spec(spec: Spec) → Prefix[source]

The path to the test stage directory for the provided spec.

Parameters:

spec – instance of the spec under test

classmethod test_log_name(spec: Spec) → str[source]

The standard log filename for a spec.

Parameters:

spec – instance of the spec under test

classmethod test_pkg_id(spec: Spec) → str[source]

The standard install test package identifier.

Parameters:

spec – instance of the spec under test

test_status(spec: Spec, externals: bool) → TestStatus[source]

Returns the overall test results status for the spec.

Parameters:

  • spec – instance of the spec under test

  • externals – True if externals are to be tested, else False

tested_file_for_spec(spec: Spec) → str[source]

The test status file path for the spec.

Parameters:

spec – instance of the spec under test

classmethod tested_file_name(spec: Spec) → str[source]

The standard test status filename for the spec.

Parameters:

spec – instance of the spec under test

to_dict() → Dict[str, Any][source]

Build a dictionary for the test suite.

Returns:

The dictionary contains entries for up to two keys.

  • specs: list of the test suite’s specs in dictionary form

  • alias: the alias, or name, given to the test suite if provided

write_reproducibility_data() → None[source]
write_test_result(spec: Spec, result: TestStatus) → None[source]

Write the spec’s test result to the test suite results file.

Parameters:

  • spec – instance of the spec under test

  • result – result from the spec’s test execution (e.g, PASSED)

exception spack.install_test.TestSuiteError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when there is an error with the test suite.

exception spack.install_test.TestSuiteFailure(num_failures)[source]

Bases: SpackError

Raised when one or more tests in a suite have failed.

exception spack.install_test.TestSuiteNameError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when there is an issue with the naming of the test suite.

exception spack.install_test.TestSuiteSpecError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when there is an issue associated with the spec being tested.

spack.install_test.cache_extra_test_sources(pkg: PackageBase, srcs: str | List[str])[source]

Copy relative source paths to the corresponding install test subdir

This routine is intended as an optional install test setup helper for grabbing source files/directories during the installation process and copying them to the installation test subdirectory for subsequent use during install testing.

Parameters:

  • pkg – package being tested

  • srcs – relative path for file(s) and or subdirectory(ies) located in the staged source path that are to be copied to the corresponding location(s) under the install testing directory.

Raises:

spack.error.InstallError – if any of the source paths are absolute or do not exist under the build stage

spack.install_test.check_outputs(expected: list | set | str, actual: str)[source]

Ensure the expected outputs are contained in the actual outputs.

Parameters:

  • expected – expected raw output string(s)

  • actual – actual output string

Raises:

RuntimeError – the expected output is not found in the actual output

spack.install_test.copy_test_files(pkg: PackageBase, test_spec: Spec)[source]

Copy the spec’s cached and custom test files to the test stage directory.

Parameters:

  • pkg – package being tested

  • test_spec – spec being tested, where the spec may be virtual

Raises:

TestSuiteError – package must be part of an active test suite

spack.install_test.find_required_file(root: str, filename: str, expected: int = 1, recursive: bool = True) → str | List[str][source]

Find the required file(s) under the root directory.

Parameters:

  • root – root directory for the search

  • filename – name of the file being located

  • expected – expected number of files to be found under the directory (default is 1)

  • recursive – True if subdirectories are to be recursively searched, else False (default is True)

Returns: the path(s), relative to root, to the required file(s)

Raises:

Exception – SkipTest when number of files detected does not match expected

spack.install_test.get_all_test_suites()[source]

Retrieves all validly staged TestSuites

Returns:

a list of TestSuite objects, which may be empty if there are none

Return type:

list

spack.install_test.get_escaped_text_output(filename: str) → List[str][source]

Retrieve and escape the expected text output from the file

Parameters:

filename – path to the file

Returns:

escaped text lines read from the file

spack.install_test.get_named_test_suites(name)[source]

Retrieves test suites with the provided name.

Returns:

a list of matching TestSuite instances, which may be empty if none

Return type:

list

Raises:

Exception – TestSuiteNameError if no name is provided

spack.install_test.get_test_stage_dir() → str[source]

Retrieves the config:test_stage path to the configured test stage root directory

Returns:

absolute path to the configured test stage root or, if none, the default test stage path

spack.install_test.get_test_suite(name: str) → TestSuite | None[source]

Ensure there is only one matching test suite with the provided name.

Returns:

the name if one matching test suite, else None

Raises:

TestSuiteNameError – If there are more than one matching TestSuites

spack.install_test.install_test_root(pkg: PackageBase) → str[source]

The install test root directory.

spack.install_test.overall_status(current_status: TestStatus, substatuses: List[TestStatus]) → TestStatus[source]

Determine the overall status based on the current and associated sub status values.

Parameters:

  • current_status – current overall status, assumed to default to PASSED

  • substatuses – status of each test part or overall status of each test spec

Returns:

test status encompassing the main test and all subtests

spack.install_test.print_message(logger: nixlog | winlog, msg: str, verbose: bool = False)[source]

Print the message to the log, optionally echoing.

Parameters:

  • logger – instance of the output logger (e.g. nixlog or winlog)

  • msg – message being output

  • verbose – True displays verbose output, False suppresses it (False is default)

spack.install_test.process_test_parts(pkg: PackageBase, test_specs: List[Spec], verbose: bool = False)[source]

Process test parts associated with the package.

Parameters:

  • pkg – package being tested

  • test_specs – list of test specs

  • verbose – Display verbose output (suppress by default)

Raises:

TestSuiteError – package must be part of an active test suite

spack.install_test.results_filename

Name of the test suite results (summary) file

spack.install_test.spack_install_test_log

Name of the Spack install phase-time test log file

spack.install_test.test_function_names(pkg: PackageBase | Type[PackageBase], add_virtuals: bool = False) → List[str][source]

Grab the names of all non-empty test functions.

Parameters:

  • pkg – package or package class of interest

  • add_virtuals – True adds test methods of provided package virtual, False only returns test functions of the package

Returns:

names of non-empty test functions

Raises:

ValueError – occurs if pkg is not a package class

spack.install_test.test_functions(pkg: PackageBase | Type[PackageBase], add_virtuals: bool = False) → List[Tuple[str, Callable]][source]

Grab all non-empty test functions.

Parameters:

  • pkg – package or package class of interest

  • add_virtuals – True adds test methods of provided package virtual, False only returns test functions of the package

Returns:

list of non-empty test functions’ (name, function)

Raises:

ValueError – occurs if pkg is not a package class

spack.install_test.test_part(pkg: PackageBase, test_name: str, purpose: str, work_dir: str = '.', verbose: bool = False)[source]
spack.install_test.test_process(pkg: PackageBase, kwargs)[source]
spack.install_test.test_suite_filename

Name of the test suite’s (JSON) lock file

spack.install_test.virtuals(pkg)[source]

Return a list of unique virtuals for the package.

Parameters:

pkg – package of interest

Returns: names of unique virtual packages

spack.install_test.write_test_suite_file(suite)[source]

Write the test suite to its (JSON) lock file.

spack.install_test.write_test_summary(counts: Counter)[source]

Write summary of the totals for each relevant status category.

Parameters:

counts – counts of the occurrences of relevant test status types

spack.installer module

This module encapsulates package installation functionality.

The PackageInstaller coordinates concurrent builds of packages for the same Spack instance by leveraging the dependency DAG and file system locks. It also proceeds with the installation of non-dependent packages of failed dependencies in order to install as many dependencies of a package as possible.

Bottom-up traversal of the dependency DAG while prioritizing packages with no uninstalled dependencies allows multiple processes to perform concurrent builds of separate packages associated with a spec.

File system locks enable coordination such that no two processes attempt to build the same or a failed dependency package.

If a dependency package fails to install, its dependents’ tasks will be removed from the installing process’s queue. A failure file is also written and locked. Other processes use this file to detect the failure and dequeue its dependents.

This module supports the coordination of local and distributed concurrent installations of packages in a Spack instance.

exception spack.installer.BadInstallPhase(pkg_name, phase)[source]

Bases: InstallError

class spack.installer.BuildProcessInstaller(pkg: PackageBase, install_args: dict)[source]

Bases: object

This class implements the part installation that happens in the child process.

run() → bool[source]

Main entry point from build_process to kick off install in child.

class spack.installer.BuildRequest(pkg: PackageBase, install_args: dict)[source]

Bases: object

Class for representing an installation request.

get_depflags(pkg: PackageBase) → int[source]

Determine the required dependency types for the associated package.

Parameters:

pkg – explicit or implicit package being installed

Returns:

required dependency type(s) for the package

Return type:

tuple

has_dependency(dep_id) → bool[source]

Returns True if the package id represents a known dependency of the requested package, False otherwise.

run_tests(pkg: PackageBase) → bool[source]

Determine if the tests should be run for the provided packages

Parameters:

pkg – explicit or implicit package being installed

Returns:

True if they should be run; False otherwise

Return type:

bool

property spec: Spec

The specification associated with the package.

traverse_dependencies(spec=None, visited=None) → Iterator[Spec][source]

Yield any dependencies of the appropriate type(s)

class spack.installer.BuildStatus(*values)[source]

Bases: Enum

Different build (task) states.

DEQUEUED

Build status indicating the task has been popped from the queue

FAILED

Build status indicating the spec failed to install

INSTALLED

Build status indicating the spec was successfully installed

INSTALLING

Build status indicating the spec is being installed (possibly by another process)

QUEUED

Build status indicating task has been added/queued.

REMOVED

Build status indicating task has been removed (to maintain priority queue invariants).

class spack.installer.BuildTask(pkg: PackageBase, request: BuildRequest, *, compiler: bool = False, start_time: float = 0.0, attempts: int = 0, status: BuildStatus = BuildStatus.QUEUED, installed: Set[str] = set())[source]

Bases: Task

Class for representing a build task for a package.

backup_dir
complete()[source]

Complete the installation of the requested spec and/or dependency represented by the build task.

fail(inner_exception)[source]
no_op: bool
poll()[source]

Check if task has successfully executed, caused an InstallError, or the child process has information ready to receive.

process_handle: BuildProcess | None
start()[source]

Attempt to use the binary cache to install requested spec and/or dependency if requested.

Otherwise, start a process for of the requested spec and/or dependency represented by the BuildTask.

started: bool
succeed()[source]
terminate() → None[source]

Terminate any processes this task still has running.

tmpdir
class spack.installer.ExecuteResult(*values)[source]

Bases: Enum

FAILED
MISSING_BUILD_SPEC
NO_OP
SUCCESS
exception spack.installer.ExternalPackageError(message, long_msg=None, pkg=None)[source]

Bases: InstallError

Raised by install() when a package is only for external use.

class spack.installer.FakeBuildTask(pkg: PackageBase, request: BuildRequest, *, compiler: bool = False, start_time: float = 0.0, attempts: int = 0, status: BuildStatus = BuildStatus.QUEUED, installed: Set[str] = set())[source]

Bases: BuildTask

Blocking BuildTask executed directly in the main thread. Used for –fake installs.

poll()[source]

Check if task has successfully executed, caused an InstallError, or the child process has information ready to receive.

process_handle: BuildProcess | None
class spack.installer.InstallAction(*values)[source]

Bases: Enum

INSTALL

Do a standard install

NONE

Don’t perform an install

OVERWRITE

Do an overwrite install

exception spack.installer.InstallLockError(message, long_msg=None, pkg=None)[source]

Bases: InstallError

Raised during install when something goes wrong with package locking.

spack.installer.InstallPolicy

Type for specifying installation source modes

alias of Literal[‘auto’, ‘cache_only’, ‘source_only’]

class spack.installer.InstallStatus(pkg_count: int)[source]

Bases: object

get_progress() → str[source]
next_pkg(pkg: PackageBase)[source]
pkg_count: int
pkg_ids: Set[str]
pkg_num: int
set_term_title(text: str)[source]
class spack.installer.MockBuildProcess[source]

Bases: object

complete() → bool[source]
terminate() → None[source]
class spack.installer.PackageInstaller(packages: List[PackageBase], *, dirty: bool = False, explicit: Set[str] | bool = False, overwrite: List[str] | Set[str] | None = None, fail_fast: bool = False, fake: bool = False, include_build_deps: bool = False, install_deps: bool = True, install_package: bool = True, install_source: bool = False, keep_prefix: bool = False, keep_stage: bool = False, restage: bool = False, skip_patch: bool = False, stop_at: str | None = None, stop_before: str | None = None, tests: bool | List[str] | Set[str] = False, unsigned: bool | None = None, verbose: bool = False, concurrent_packages: int | None = None, root_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', create_reports: bool = False)[source]

Bases: object

Class for managing the install process for a Spack instance based on a bottom-up DAG approach.

This installer can coordinate concurrent batch and interactive, local and distributed (on a shared file system) builds for the same Spack instance.

all_dependencies: Dict[str, Set[str]]
build_pq: List[Tuple[Tuple[int, int], Task]]
build_tasks: Dict[str, Task]
complete_task(task: Task, install_status: InstallStatus) → Tuple | None[source]

Attempts to complete a package installation.

failed: Dict[str, Lock | None]
install() → None[source]

Install the requested package(s) and/or associated dependencies.

installed: Set[str]
locks: Dict[str, Tuple[str, Lock | None]]
pid: int
reports: Dict[str, RequestRecord]
start_task(task: Task, install_status: InstallStatus, term_status: TermStatusLine) → None[source]

Attempt to start a package installation.

class spack.installer.RewireTask(pkg: PackageBase, request: BuildRequest, *, compiler: bool = False, start_time: float = 0.0, attempts: int = 0, status: BuildStatus = BuildStatus.QUEUED, installed: Set[str] = set())[source]

Bases: Task

Class for representing a rewire task for a package.

complete()[source]

Execute rewire task

Rewire tasks are executed by either rewiring self.package.spec.build_spec that is already installed or downloading and rewiring a binary for the it.

If not available installed or as binary, return ExecuteResult.MISSING_BUILD_SPEC. This will prompt the Installer to requeue the task with a dependency on the BuildTask to install self.pkg.spec.build_spec

poll()[source]

Check if child process has information ready to receive.

start()[source]

Start the work of this task.

class spack.installer.Task(pkg: PackageBase, request: BuildRequest, *, compiler: bool = False, start_time: float = 0.0, attempts: int = 0, status: BuildStatus = BuildStatus.QUEUED, installed: Set[str] = set())[source]

Bases: object

Base class for representing a task for a package.

add_dependency(pkg_id, installed=False)[source]

Ensure the package is in this task’s dependencies list.

Parameters:

  • pkg_id (str) – package identifier of the dependency package

  • installed (bool) – install status of the dependency package

add_dependent(pkg_id: str) → None[source]

Ensure the package is in this task’s dependents list.

Parameters:

pkg_id – package identifier of the dependent package

complete() → ExecuteResult[source]

Complete the work of this task.

error_result: BaseException | None
property explicit: bool
flag_installed(installed: List[str]) → None[source]

Ensure the dependency is not considered to still be uninstalled.

Parameters:

installed – the identifiers of packages that have been installed so far

get_install_action() → InstallAction[source]

Determine whether the installation should be overwritten (if it already exists) or skipped (if has been handled by another process).

If the package has not been installed yet, this will indicate that the installation should proceed as normal (i.e. no need to transactionally preserve the old prefix).

property install_action
property install_policy: Literal['auto', 'cache_only', 'source_only']
property is_build_request: bool

The package was requested directly

property key: Tuple[int, int]

The key is the tuple (# uninstalled dependencies, sequence).

next_attempt(installed) → Task[source]

Create a new, updated task for the next installation attempt.

no_op: bool
poll() → bool[source]

Check if child process has information ready to receive.

property priority

The priority is based on the remaining uninstalled dependencies.

start()[source]

Start the work of this task.

success_result: ExecuteResult | None
terminate() → None[source]

End any processes and clean up any resources allocated by this Task.

By default this is a no-op.

class spack.installer.TermStatusLine(enabled: bool)[source]

Bases: object

This class is used in distributed builds to inform the user that other packages are being installed by another process.

add(pkg_id: str)[source]

Add a package to the waiting list, and if it is new, update the status line.

clear()[source]

Clear the status line.

enabled: bool
pkg_list: List[str]
pkg_set: Set[str]
exception spack.installer.UpstreamPackageError(message, long_msg=None, pkg=None)[source]

Bases: InstallError

Raised during install when something goes wrong with an upstream package.

spack.installer.archive_install_logs(pkg: PackageBase, phase_log_dir: str) → None[source]

Copy install logs to their destination directory(ies) :param pkg: the package that was built and installed :param phase_log_dir: path to the archive directory

spack.installer.build_process(pkg: PackageBase, install_args: dict) → bool[source]

Perform the installation/build of the package.

This runs in a separate child process, and has its own process and python module space set up by build_environment.start_build_process().

This essentially wraps an instance of BuildProcessInstaller so that we can more easily create one in a subprocess.

This function’s return value is returned to the parent process.

Parameters:

  • pkg – the package being installed.

  • install_args – arguments to installer from parent process.

spack.installer.check_db(spec: Spec) → Tuple[InstallRecord | None, bool][source]

Determine if the spec is flagged as installed in the database

Parameters:

spec – spec whose database install status is being checked

Returns:

Tuple of optional database record, and a boolean installed_in_db that’s True iff the spec is considered installed

spack.installer.combine_phase_logs(phase_log_files: List[str], log_path: str) → None[source]

Read set or list of logs and combine them into one file.

Each phase will produce it’s own log, so this function aims to cat all the separate phase log output files into the pkg.log_path. It is written generally to accept some list of files, and a log path to combine them to.

Parameters:

  • phase_log_files – a list or iterator of logs to combine

  • log_path – the path to combine them to

spack.installer.deprecate(spec: Spec, deprecator: Spec, link_fn) → None[source]

Deprecate this package in favor of deprecator spec

spack.installer.dump_packages(spec: Spec, path: str) → None[source]

Dump all package information for a spec and its dependencies.

This creates a package repository within path for every namespace in the spec DAG, and fills the repos with package files and patch files for every node in the DAG.

Parameters:

  • spec – the Spack spec whose package information is to be dumped

  • path – the path to the build packages directory

spack.installer.get_dependent_ids(spec: Spec) → List[str][source]

Return a list of package ids for the spec’s dependents

Parameters:

spec – Concretized spec

Returns: list of package ids

spack.installer.install_msg(name: str, pid: int, install_status: InstallStatus) → str[source]

Colorize the name/id of the package being installed

Parameters:

  • name – Name/id of the package being installed

  • pid – id of the installer process

Return: Colorized installing message

spack.installer.log(pkg: PackageBase) → None[source]

Copy provenance into the install directory on success

Parameters:

pkg – the package that was built and installed

spack.installer.package_id(spec: Spec) → str[source]

A “unique” package identifier for installation purposes

The identifier is used to track tasks, locks, install, and failure statuses.

The identifier needs to distinguish between combinations of compilers and packages for combinatorial environments.

Parameters:

pkg – the package from which the identifier is derived

spack.installer.print_install_test_log(pkg: PackageBase) → None[source]

Output install test log file path but only if have test failures.

Parameters:

pkg – instance of the package under test

spack.installer_dispatch module

spack.installer_dispatch.create_installer(packages: List[spack.package_base.PackageBase], *, dirty: bool = False, explicit: Set[str] | bool = False, overwrite: List[str] | Set[str] | None = None, fail_fast: bool = False, fake: bool = False, include_build_deps: bool = False, install_deps: bool = True, install_package: bool = True, install_source: bool = False, keep_prefix: bool = False, keep_stage: bool = False, restage: bool = True, skip_patch: bool = False, stop_at: str | None = None, stop_before: str | None = None, tests: bool | List[str] | Set[str] = False, unsigned: bool | None = None, verbose: bool = False, concurrent_packages: int | None = None, root_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', create_reports: bool = False) → spack.installer.PackageInstaller | spack.new_installer.PackageInstaller[source]

Create an installer based on the current configuration and feature support.

spack.main module

This is the implementation of the Spack command line executable.

In a normal Spack installation, this is invoked from the bin/spack script after the system path is set up.

class spack.main.SpackArgumentParser(prog=None, usage=None, description=None, epilog=None, parents=[], formatter_class=HelpFormatter, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True, allow_abbrev=True, exit_on_error=True)[source]

Bases: ArgumentParser

add_command(cmd_name)[source]

Add one subcommand to this parser.

add_subparsers(**kwargs)[source]

Ensure that sensible defaults are propagated to subparsers

format_help(level='short')[source]
format_help_sections(level)[source]

Format help on sections for a particular verbosity level.

Parameters:

level (str) – "short" or "long" (more commands shown for long)

class spack.main.SpackCommand(command_name: str)[source]

Bases: object

Callable object that invokes a Spack command (for testing).

Example usage:

install = SpackCommand("install")
install("-v", "mpich")

Use this to invoke Spack commands directly from Python and check their output.

binary_output

Binary output captured from the last command invocation

capture_output(enable: bool = True)[source]

Captures stdout and stderr from the current process and all subprocesses. This uses a temporary file and os.dup2 to redirect file descriptors.

error: BaseException | None

Error raised during the last command invocation, if any

output

Decoded output captured from the last command invocation

returncode: Any

Return code of the last command invocation

exception spack.main.SpackCommandError(code, output)[source]

Bases: Exception

Raised when SpackCommand execution fails, replacing SystemExit.

class spack.main.SpackHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]

Bases: RawTextHelpFormatter

add_argument(action)[source]
add_arguments(actions)[source]
start_section(heading)[source]
spack.main.add_all_commands(parser)[source]

Add all spack subcommands to the parser.

spack.main.add_command_line_scopes(cfg: Configuration, command_line_scopes: List[str]) → None[source]

Add additional scopes from the --config-scope argument, either envs or dirs.

Parameters:

  • cfg – configuration instance

  • command_line_scopes – list of configuration scope paths

Raises:

spack.error.ConfigError – if the path is an invalid configuration scope

spack.main.allows_unknown_args(command)[source]

Implements really simple argument injection for unknown arguments.

Commands may add an optional argument called “unknown args” to indicate they can handle unknown args, and we’ll pass the unknown args in.

spack.main.finish_parse_and_run(parser, cmd_name, main_args, env_format_error)[source]

Finish parsing after we know the command to run.

spack.main.index_commands()[source]

create an index of commands by section for this help level

spack.main.intro_by_level

intro text for help at different levels

spack.main.levels

help levels in order of detail (i.e., number of commands shown)

spack.main.main(argv=None)[source]

This is the entry point for the Spack command.

main() itself is just an error handler – it handles errors for everything in Spack that makes it to the top level.

The logic is all in _main().

Parameters:

argv (list or None) – command line arguments, NOT including the executable name. If None, parses from sys.argv.

spack.main.make_argument_parser(**kwargs)[source]

Create an basic argument parser without any subcommands added.

spack.main.options_by_level

control top-level spack options shown in basic vs. advanced help

spack.main.print_setup_info(*info)[source]

Print basic information needed by setup-env.[c]sh.

Parameters:

info (list) – list of things to print: comma-separated list of "csh", "sh", or "modules"

This is in main.py to make it fast; the setup scripts need to invoke spack in login scripts, and it needs to be quick.

spack.main.required_command_properties

Properties that commands are required to set.

spack.main.resolve_alias(cmd_name: str, cmd: List[str]) → Tuple[str, List[str]][source]

Resolves aliases in the given command.

Parameters:

  • cmd_name – command name.

  • cmd – command line arguments.

Returns:

new command name and arguments.

spack.main.restore_macos_dyld_vars()[source]

Spack mutates DYLD_* variables in spack load and spack env activate. Unlike Linux, macOS SIP clears these variables in new processes, meaning that os.environ["DYLD_*"] in our Python process is not the same as the user’s shell. Therefore, we store the user’s DYLD_* variables in SPACK_DYLD_* and restore them here.

spack.main.section_descriptions

Longer text for each section, to show in help

spack.main.section_order

preferential command order for some sections (e.g., build pipeline is in execution order, not alphabetical)

spack.main.setup_main_options(args)[source]

Configure spack globals based on the basic options.

spack.main.showwarning(message, category, filename, lineno, file=None, line=None)[source]

Redirects messages to tty.warn.

spack.main.stat_names

names of profile statistics

spack.mixins module

This module contains additional behavior that can be attached to any given package.

spack.mixins.filter_compiler_wrappers(*files: str, after: str = 'install', relative_root: str | None = None, ignore_absent: bool = True, backup: bool = False, recursive: bool = False, **kwargs) → None[source]

Registers a phase callback (e.g. post-install) to look for references to Spack’s compiler wrappers in the given files and replace them with the underlying compilers.

Example usage:

class MyPackage(Package):

    filter_compiler_wrappers("mpicc", "mpicxx", relative_root="bin")

This is useful for packages that register the path to the compiler they are built with to be used later at runtime. Spack’s compiler wrappers cannot be used at runtime, as they require Spack’s build environment to be set up. Using this function, the compiler wrappers are replaced with the actual compilers, so that the package works correctly at runtime.

Parameters:

  • *files – files to be filtered relative to the search root (install prefix by default).

  • after – specifies after which phase the files should be filtered (defaults to "install").

  • relative_root – path relative to install prefix where to start searching for the files to be filtered. If not set the install prefix will be used as the search root. It is highly recommended to set this, as searching recursively from the installation prefix can be very slow.

  • ignore_absent – if present, will be forwarded to filter_file()

  • backup – if present, will be forwarded to filter_file()

  • recursive – if present, will be forwarded to find()

spack.multimethod module

This module contains utilities for using multi-methods in spack. You can think of multi-methods like overloaded methods – they’re methods with the same name, and we need to select a version of the method based on some criteria. e.g., for overloaded methods, you would select a version of the method to call based on the types of its arguments.

In spack, multi-methods are used to ease the life of package authors. They allow methods like install() (or other methods called by install()) to declare multiple versions to be called when the package is instantiated with different specs. e.g., if the package is built with OpenMPI on x86_64,, you might want to call a different install method than if it was built for mpich2 on BlueGene/Q. Likewise, you might want to do a different type of install for different versions of the package.

Multi-methods provide a simple decorator-based syntax for this that avoids overly complicated rat nests of if statements. Obviously, depending on the scenario, regular old conditionals might be clearer, so package authors should use their judgement.

exception spack.multimethod.MultiMethodError(message)[source]

Bases: SpackError

Superclass for multimethod dispatch errors

class spack.multimethod.MultiMethodMeta(name, bases, attr_dict)[source]

Bases: type

This allows us to track the class’s dict during instantiation.

exception spack.multimethod.NoSuchMethodError(cls, method_name, spec, possible_specs)[source]

Bases: SpackError

Raised when we can’t find a version of a multi-method.

class spack.multimethod.SpecMultiMethod(default=None)[source]

Bases: object

This implements a multi-method for Spack specs. Packages are instantiated with a particular spec, and you may want to execute different versions of methods based on what the spec looks like. For example, you might want to call a different version of install() for one platform than you call on another.

The SpecMultiMethod class implements a callable object that handles method dispatch. When it is called, it looks through registered methods and their associated specs, and it tries to find one that matches the package’s spec. If it finds one (and only one), it will call that method.

This is intended for use with decorators (see below). The decorator (see docs below) creates SpecMultiMethods and registers method versions with them.

To register a method, you can do something like this:

mm = SpecMultiMethod()
mm.register("^chaos_5_x86_64_ib", some_method)

The object registered needs to be a Spec or some string that will parse to be a valid spec.

When the mm is actually called, it selects a version of the method to call based on the sys_type of the object it is called on.

See the docs for decorators below for more details.

register(spec, method)[source]

Register a version of a method for a particular spec.

spack.multimethod.default_args(**kwargs)[source]

Context manager to override the default arguments of directives.

Example:

with default_args(type=("build", "run")):
    depends_on("py-foo")
    depends_on("py-bar")
    depends_on("py-baz")

Notice that unlike then when() context manager, this one is not composable, as it merely overrides the default argument values for the duration of the context. For example:

with default_args(when="+foo"):
    depends_on("pkg-a")
    depends_on("pkg-b", when="+bar")

is equivalent to:

depends_on("pkg-a", when="+foo")
depends_on("pkg-b", when="+bar")
class spack.multimethod.when(condition: str | bool)[source]

Bases: object

This is a multi-purpose class, which can be used

  1. As a context manager to group directives together that share the same when= argument.

  2. As a decorator for defining multi-methods (multiple methods with the same name are defined, but the version that is called depends on the condition of the package’s spec)

As a context manager it groups directives together. It allows you to write:

with when("+nvptx"):
    conflicts("@:6", msg="NVPTX only supported from gcc 7")
    conflicts("languages=ada")
    conflicts("languages=brig")

instead of the more repetitive:

conflicts("@:6", when="+nvptx", msg="NVPTX only supported from gcc 7")
conflicts("languages=ada", when="+nvptx")
conflicts("languages=brig", when="+nvptx")

This context manager is composable both with nested when contexts and with other when= arguments in directives. For example:

with when("+foo"):
    with when("+bar"):
        depends_on("dependency", when="+baz")

is equilavent to:

depends_on("dependency", when="+foo +bar +baz")

As a decorator, it allows packages to declare multiple versions of methods like install() that depend on the package’s spec. For example:

class SomePackage(Package):
    ...

    def install(self, spec: Spec, prefix: Prefix):
        # Do default install

    @when("target=x86_64:")
    def install(self, spec: Spec, prefix: Prefix):
        # This will be executed instead of the default install if
        # the package's target is in the x86_64 family.

    @when("target=aarch64:")
    def install(self, spec: Spec, prefix: Prefix):
        # This will be executed if the package's target is in
        # the aarch64 family

This allows each package to have a default version of install() AND specialized versions for particular platforms. The version that is called depends on the architecture of the instantiated package.

Note that this works for methods other than install, as well. So, if you only have part of the install that is platform specific, you could do this:

class SomePackage(Package):
    ...
    # virtual dependence on MPI.
    # could resolve to mpich, mpich2, OpenMPI
    depends_on("mpi")

    def setup(self):
        # do nothing in the default case
        pass

    @when("^openmpi")
    def setup(self):
        # do something special when this is built with OpenMPI for its MPI implementations.
        pass

    def install(self, prefix):
        # Do common install stuff
        self.setup()
        # Do more common install stuff

Note that the default version of decorated methods must always come first. Otherwise it will override all of the decorated versions. This is a limitation of the Python language.

spec: Spec | None

spack.new_installer module

New installer that will ultimately replace installer.py. It features an event loop, non-blocking I/O, and a POSIX jobserver to limit concurrency. It also has a more advanced terminal UI. It’s mostly self-contained to avoid interfering with the rest of Spack too much while it’s being developed and tested.

The installer consists of a UI process that manages multiple build processes and handles updates to the database. It detects or creates a jobserver, and then kicks off an event loop in which it runs through a build queue, always running at least one build. Concurrent builds run as jobserver tokens are obtained. This means only one -j flag is needed to control concurrency.

The UI process has two modes: an overview mode where it shows the status of all builds, and a mode where it follows the logs of a specific build. It listens to keyboard input to switch between modes.

The build process does an ordinary install, but also spawns a “tee” thread that forwards its build output to both a log file and the UI process (if the UI process has requested it). This thread also runs an event loop to listen for control messages from the UI process (to enable/disable echoing of logs), and for output from the build process.

exception spack.new_installer.BinaryCacheMiss(message: str, long_message: str | None = None)[source]

Bases: SpackError

class spack.new_installer.BuildGraph(specs: List[Spec], root_policy: Literal['auto', 'cache_only', 'source_only'], dependencies_policy: Literal['auto', 'cache_only', 'source_only'], include_build_deps: bool, install_package: bool, install_deps: bool, database: Database, overwrite_set: Set[str] | None = None, tests: bool | List[str] | Set[str] = False, explicit_set: Set[str] | None = None)[source]

Bases: object

Represents the dependency graph for package installation.

child_to_parent: Dict[str, Set[str]]
done: Set[str]
enqueue_parents(dag_hash: str, pending_builds: List[str]) → None[source]

After a spec is installed, remove it from the graph and enqueue any parents that are now ready to install.

Parameters:

  • dag_hash – The dag_hash of the spec that was just installed

  • pending_builds – List to append parent specs that are ready to build

expand_build_deps(spec_hashes: List[str], pending_builds: List[str], database: Database, dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto') → List[str][source]

Expand build dependencies for a list of specs after binary cache misses.

Adds the spec’s build deps and their transitive runtime deps to the graph. When dependencies_policy is "source_only", build deps of newly added specs are included immediately. Installed deps are skipped without adding edges.

The caller must hold the database read lock and have called db._read().

Returns the list of newly added dag hashes.

force_source: Set[str]
get_unexpanded_build_deps(dag_hash: str) → List[Spec][source]

Returns a list of unprocessed build deps for a spec.

has_unexpanded_build_deps(dag_hash: str) → bool[source]
parent_to_child: Dict[str, Set[str]]
pruned: Set[str]
class spack.new_installer.BuildInfo(spec: Spec, explicit: bool, control_w_conn: Connection | None, log_path: str | None = None, start_time: float = 0.0)[source]

Bases: object

Information about a package being built.

control_w_conn
duration: float | None
explicit: bool
external: bool
finished_time: float | None
hash: str
log_path
log_summary: str | None
name: str
prefix: str
progress_percent: int | None
start_time: float
state: str
version: str
class spack.new_installer.BuildStatus(total: int, stdout: TextIOWrapper = sys.stdout, get_terminal_size: Callable[[], terminal_size] = os.get_terminal_size, get_time: Callable[[], float] = time.monotonic, is_tty: bool | None = None, color: bool | None = None, verbose: bool = False, filter_padding: bool = False)[source]

Bases: object

Tracks the build status display for terminal output.

actual_jobs: int
add_build(spec: Spec, explicit: bool, control_w_conn: Connection | None = None, log_path: str | None = None) → None[source]

Add a new build to the display and mark the display as dirty.

blocked: bool
builds: Dict[str, BuildInfo]
finished_builds: List[BuildInfo]
headless

When True, suppress all terminal output (process is in background). Controlling code is responsible for modifying this variable based on process state

next(direction: int = 1) → None[source]

Follow the logs of the next build in the list.

on_resize() → None[source]

Refresh cached terminal size and trigger a redraw.

parse_log_summary(build_id: str) → None[source]

Parse the build log for errors/warnings and store the summary.

print_logs(build_id: str, data: bytes) → None[source]
remove_build(build_id: str) → None[source]

Remove a build from the display (e.g. after a binary cache miss before retry).

search_input(input: str) → None[source]

Handle keyboard input when in search mode

set_blocked(blocked: bool) → None[source]

Set whether all pending builds are blocked by another Spack process.

set_jobs(actual: int, target: int) → None[source]

Set the actual and target number of jobs to run concurrently.

target_jobs: int
terminal_size_changed: bool
toggle() → None[source]

Toggle between overview mode and following a specific build.

total

Ordered dict of build ID -> info

update(finalize: bool = False) → None[source]

Redraw the interactive display.

update_progress(build_id: str, current: int, total: int) → None[source]

Update the progress of a package and mark the display as dirty.

update_state(build_id: str, state: str) → None[source]

Update the state of a package and mark the display as dirty.

verbose

Verbose mode only applies to non-TTY where we want to track a single build log.

spack.new_installer.CLEANUP_TIMEOUT

How long to display finished packages before graying them out

class spack.new_installer.ChildInfo(proc: Process, spec: Spec, output_r_conn: Connection, state_r_conn: Connection, control_w_conn: Connection, log_path: str, explicit: bool = False)[source]

Bases: DatabaseAction

Information about a child process.

close(selector: BaseSelector) → int[source]

Unregister and close file descriptors, and join the child process. Returns the exit code of the child process.

control_w_conn
explicit
log_path
output_r_conn
proc
save_to_db(db: Database) → None[source]
state_r_conn
spack.new_installer.DATABASE_WRITE_INTERVAL

How often to flush completed builds to the database

class spack.new_installer.DatabaseAction[source]

Bases: object

Base class for objects that need to be persisted to the database.

prefix_lock: Lock | None
release_prefix_lock() → None[source]
save_to_db(db: Database) → None[source]
spec: Spec
class spack.new_installer.ExitCode[source]

Bases: object

BUILD_CACHE_MISS

Exit code used by the child process to signal a binary cache miss (no source fallback)

BUILD_ERROR
STOPPED_AT_PHASE

Exit code used by the child process to signal that the build was stopped at a phase boundary

SUCCESS
class spack.new_installer.FdInfo(pid: int, name: str)[source]

Bases: object

Information about a file descriptor mapping.

name
pid
class spack.new_installer.GlobalState[source]

Bases: object

Global state needed in a build subprocess. This is similar to spack.subprocess_context, but excludes the Spack environment, which is slow to serialize and should not be needed during the build.

config
monkey_patches
repo_cache
restore()[source]
spack_working_dir
store
spack.new_installer.HEADLESS_WAKE_INTERVAL

How often to wake up in headless mode to check for background->foreground transition (seconds)

spack.new_installer.InstallPolicy

Type for specifying installation source modes

alias of Literal[‘auto’, ‘cache_only’, ‘source_only’]

class spack.new_installer.JobServer(num_jobs: int)[source]

Bases: object

Attach to an existing POSIX jobserver or create a FIFO-based one.

acquire(jobs: int) → int[source]

Try and acquire at most ‘jobs’ tokens from the jobserver. Returns the number of tokens actually acquired (may be less than requested, or zero).

close() → None[source]
decrease_parallelism() → None[source]

Request an eventual concurrency decrease by 1.

fifo_path: str | None
has_target_parallelism() → bool[source]
increase_parallelism() → None[source]

Add one token to the jobserver to increase parallelism; this should always work.

makeflags(gmake: Spec | None) → str[source]

Return the MAKEFLAGS for a build process, depending on its gmake build dependency.

maybe_discard_tokens() → None[source]

Try to get reduce parallelism by discarding tokens.

num_jobs

The number of jobs to run concurrently. This translates to num_jobs - 1 tokens in the jobserver.

release() → None[source]

Release a token back to the jobserver.

target_jobs

The target number of jobs to run concurrently, which may differ from num_jobs if the user has requested a decrease in parallelism, but we haven’t consumed enough tokens to reflect that yet. This value is used in the UI. The invariant is that self.target_jobs can only be modified if self.created is True.

tokens_acquired

Keep track of how many tokens Spack itself has acquired, which is used to release them.

class spack.new_installer.MarkExplicitAction(spec: Spec)[source]

Bases: DatabaseAction

Action to mark an already installed spec as explicitly installed. Similar to ChildInfo, but used when no build process was needed.

save_to_db(db: Database) → None[source]
class spack.new_installer.NullReportData[source]

Bases: ReportData

No-op drop-in for ReportData when no reporter is configured.

Avoids creating InstallRecords and reading log files on every completed build.

finalize(reports: Dict[str, RequestRecord], build_graph: BuildGraph) → None[source]

Finalize InstallRecords and append them to RequestRecords after all builds finish.

Parameters:

  • reports – Map of root dag_hash to RequestRecord to append to.

  • build_graph – The build graph containing all nodes and their states.

finish_record(spec: Spec, exitcode: int, log_path: str | None = None) → None[source]

Mark the InstallRecord for a spec as succeeded or failed.

start_record(spec: Spec) → None[source]

Begin an InstallRecord for a spec that is about to be built.

spack.new_installer.OUTPUT_BUFFER_SIZE

Size of the output buffer for child processes

spack.new_installer.OVERWRITE_BACKUP_SUFFIX

Suffix for temporary backup during overwrite install

spack.new_installer.OVERWRITE_GARBAGE_SUFFIX

Suffix for temporary cleanup during failed install

class spack.new_installer.PackageInstaller(packages: List[PackageBase], *, dirty: bool = False, explicit: Set[str] | bool = False, overwrite: List[str] | Set[str] | None = None, fail_fast: bool = False, fake: bool = False, include_build_deps: bool = False, install_deps: bool = True, install_package: bool = True, install_source: bool = False, keep_prefix: bool = False, keep_stage: bool = False, restage: bool = True, skip_patch: bool = False, stop_at: str | None = None, stop_before: str | None = None, tests: bool | List[str] | Set[str] = False, unsigned: bool | None = None, verbose: bool = False, concurrent_packages: int | None = None, root_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', dependencies_policy: Literal['auto', 'cache_only', 'source_only'] = 'auto', create_reports: bool = False)[source]

Bases: object

dependencies_policy: Literal['auto', 'cache_only', 'source_only']
explicit: Set[str]
install() → None[source]
log_paths: Dict[str, str]
overwrite: Set[str]

Set of DAG hashes to overwrite (if already installed)

overwrite_time: float

Time at which the overwrite install was requested; used to detect concurrent overwrites.

pending_builds

queue of packages ready to install (no children)

pending_expansions: List[str]

specs awaiting build-dep expansion (deferred until DB read lock is available)

root_policy: Literal['auto', 'cache_only', 'source_only']
running_builds: Dict[int, ChildInfo]
state_buffers: Dict[int, str]
tests: bool | List[str] | Set[str]
class spack.new_installer.PrefixPivoter(prefix: str, keep_prefix: bool = False)[source]

Bases: object

Manages the installation prefix of a build.

keep_prefix

Whether to keep a failed installation prefix

tmp_prefix: str | None

Temporary location for the original prefix

class spack.new_installer.ReportData(roots: List[Spec])[source]

Bases: object

Data collected for reports during installation.

build_records: Dict[str, InstallRecord]
finalize(reports: Dict[str, RequestRecord], build_graph: BuildGraph) → None[source]

Finalize InstallRecords and append them to RequestRecords after all builds finish.

Parameters:

  • reports – Map of root dag_hash to RequestRecord to append to.

  • build_graph – The build graph containing all nodes and their states.

finish_record(spec: Spec, exitcode: int, log_path: str | None = None) → None[source]

Mark the InstallRecord for a spec as succeeded or failed.

start_record(spec: Spec) → None[source]

Begin an InstallRecord for a spec that is about to be built.

spack.new_installer.SPINNER_INTERVAL

How often to update a spinner in seconds

class spack.new_installer.ScheduleResult(blocked: bool, to_start: List[Tuple[str, Lock]], newly_installed: List[Tuple[str, Spec, Lock]], to_mark_explicit: List[MarkExplicitAction])[source]

Bases: NamedTuple

Return value of schedule_builds().

blocked: bool

True if any pending builds were blocked on locks held by other processes.

newly_installed: List[Tuple[str, Spec, Lock]]

(dag_hash, spec, lock) triples found already installed by another process; the read lock is held and the caller must add it to retained_read_locks.

to_mark_explicit: List[MarkExplicitAction]

Actions to mark already installed specs explicit in the DB.

to_start: List[Tuple[str, Lock]]

(dag_hash, lock) pairs where the write lock is held and the caller must start the build and eventually release the lock.

class spack.new_installer.StdinReader(fd: int)[source]

Bases: object

Helper class to do non-blocking, incremental decoding of stdin, stripping ANSI escape sequences. The input is the backing file descriptor for stdin (instead of the TextIOWrapper) to avoid double buffering issues: the event loop triggers when the fd is ready to read, and if we do a partial read from the TextIOWrapper, it will likely drain the fd and buffer the remainder internally, which the event loop is not aware of, and user input doesn’t come through.

ansi_escape_re

For stripping out arrow and navigation keys

decoder

Handle multi-byte UTF-8 characters

read() → str[source]
class spack.new_installer.Tee(control: Connection, parent: Connection, log_path: str)[source]

Bases: object

Emulates ./build 2>&1 | tee build.log. The output is sent both to a log file and the parent process (if echoing is enabled). The control_fd is used to enable/disable echoing.

close() → None[source]
log_path

The path of the log file

class spack.new_installer.TerminalState(selector: BaseSelector, build_status: BuildStatus, on_suspend: Callable[[], None] | None = None, on_resume: Callable[[], None] | None = None)[source]

Bases: object

Manages terminal settings, stdin selector registration, and suspend/resume signals.

Installs a SIGTSTP handler that restores the terminal before suspending and re-applies it on resume. After waking up it checks whether the process is in the foreground or background and enables or suppresses interactive output accordingly.

Optional on_suspend / on_resume hooks are called just before the process suspends and just after it wakes, allowing callers to pause and resume child processes.

enter_background() → None[source]

Suppress output and stop reading stdin to avoid SIGTTIN/SIGTTOU.

enter_foreground() → None[source]

Restore interactive terminal mode.

handle_continue() → None[source]

Detect whether the process is in the foreground or background and adjust accordingly.

setup() → None[source]

Set cbreak mode, register stdin and signal pipes in the selector.

teardown() → None[source]

Restore terminal settings and signal handlers, close pipes.

spack.new_installer.create_jobserver_fifo(num_jobs: int) → Tuple[int, int, str][source]

Create a new jobserver FIFO with the specified number of job tokens.

spack.new_installer.get_jobserver_config(makeflags: str | None = None) → str | Tuple[int, int] | None[source]

Parse MAKEFLAGS for jobserver. Either it’s a FIFO or (r, w) pair of file descriptors.

Parameters:

makeflags – MAKEFLAGS string to parse. If None, reads from os.environ.

spack.new_installer.install_from_buildcache(mirrors: List[MirrorMetadata], spec: Spec, unsigned: bool | None, state_stream: TextIOWrapper) → bool[source]
spack.new_installer.open_existing_jobserver_fifo(fifo_path: str) → Tuple[int, int] | None[source]

Open an existing jobserver FIFO for reading and writing.

spack.new_installer.schedule_builds(pending: List[str], build_graph: BuildGraph, db: Database, prefix_locker: SpecLocker, overwrite: Set[str], overwrite_time: float, capacity: int, needs_jobserver_token: bool, jobserver: JobServer, explicit: Set[str]) → ScheduleResult[source]

Try to schedule as many pending builds as possible.

For each pending spec, attempts to acquire a non-blocking per-spec write lock. If the write lock times out, a read lock is tried as a fallback: a successful read lock means the first process finished and downgraded its write lock. If the DB confirms the spec is installed, it is captured as newly_installed; if the DB says it is not installed, the concurrent process was likely killed mid-build, and the spec is retried next iteration. Under both the DB read lock and the prefix lock, checks whether another process has already installed the spec. If so, captures it as newly_installed (caller enqueues parents) and keeps a read lock on the prefix to prevent concurrent uninstall. Otherwise, acquires a jobserver token if needed and adds the (dag_hash, lock) pair to to_start (caller launches the build).

Parameters:

  • pending – List of dag hashes pending installation; modified in-place.

  • build_graph – The build dependency graph; used for node lookup and parent enqueueing.

  • db – Package database; used for read lock and installed-status queries.

  • prefix_locker – Per-spec write locker.

  • overwrite – Set of dag hashes to overwrite even if already installed.

  • overwrite_time – Timestamp (from time.time()) at which the overwrite install was requested. A spec in overwrite whose DB installation_time >= overwrite_time was installed by a concurrent process after our request started and should be treated as done.

  • capacity – Maximum number of new builds to add to to_start in this call.

  • needs_jobserver_token – True if a jobserver token is required for the first new build.

  • jobserver – Jobserver for acquiring tokens.

  • explicit – Set of dag hashes to mark explicit in the DB if found already installed.

Returns:

A ScheduleResult with blocked, to_start, and newly_installed fields; see ScheduleResult for field semantics.

spack.new_installer.send_installed_from_binary_cache(state_pipe: TextIOWrapper) → None[source]

Send a notification that the package was installed from binary cache.

spack.new_installer.send_progress(current: int, total: int, state_pipe: TextIOWrapper) → None[source]

Send a progress update message.

spack.new_installer.send_state(state: str, state_pipe: TextIOWrapper) → None[source]

Send a state update message.

spack.new_installer.start_build(spec: Spec, explicit: bool, mirrors: List[MirrorMetadata], unsigned: bool | None, install_policy: Literal['auto', 'cache_only', 'source_only'], dirty: bool, keep_stage: bool, restage: bool, keep_prefix: bool, skip_patch: bool, fake: bool, install_source: bool, run_tests: bool, jobserver: JobServer, log_path: str, stop_before: str | None = None, stop_at: str | None = None) → ChildInfo[source]

Start a new build.

spack.new_installer.tee(control_r: int, log_r: int, log_file: BufferedWriter, parent_w: int) → None[source]

Forward log_r to file_w and parent_w (if echoing is enabled). Echoing is enabled and disabled by reading from control_r.

spack.new_installer.worker_function(spec: Spec, explicit: bool, mirrors: List[MirrorMetadata], unsigned: bool | None, install_policy: Literal['auto', 'cache_only', 'source_only'], dirty: bool, keep_stage: bool, restage: bool, keep_prefix: bool, skip_patch: bool, fake: bool, install_source: bool, run_tests: bool, state: Connection, parent: Connection, echo_control: Connection, makeflags: str, js1: Connection | None, js2: Connection | None, log_path: str, global_state: GlobalState, stop_before: str | None = None, stop_at: str | None = None)[source]

Function run in the build child process. Installs the specified spec, sending state updates and build output back to the parent process.

Parameters:

  • spec – Spec to install

  • explicit – Whether the spec was explicitly requested by the user

  • mirrors – List of buildcache mirrors to try

  • unsigned – Whether to allow unsigned buildcache entries

  • install_policy – "auto", "cache_only", or "source_only"

  • dirty – Whether to preserve user environment in the build environment

  • keep_stage – Whether to keep the build stage after installation

  • restage – Whether to restage the source before building

  • keep_prefix – Whether to keep a failed installation prefix

  • skip_patch – Whether to skip the patch phase

  • run_tests – Whether to run install-time tests for this package

  • state – Connection to send state updates to

  • parent – Connection to send build output to

  • echo_control – Connection to receive echo control messages from

  • makeflags – MAKEFLAGS to set, so that the build process uses the POSIX jobserver

  • js1 – Connection for old style jobserver read fd (if any). Unused, just to inherit fd.

  • js2 – Connection for old style jobserver write fd (if any). Unused, just to inherit fd.

  • log_path – Path to the log file to write build output to

  • global_state – Global state to restore

spack.package_base module

Base class for all Spack packages.

exception spack.package_base.ActivationError(msg, long_msg=None)[source]

Bases: ExtensionError

Raised when there are problems activating an extension.

exception spack.package_base.DependencyConflictError(conflict)[source]

Bases: SpackError

Raised when the dependencies cannot be flattened as asked for.

class spack.package_base.DetectablePackageMeta(name, bases, attr_dict)[source]

Bases: type

Check if a package is detectable and add default implementations for the detection function.

TAG
class spack.package_base.DisableRedistribute(source, binary)[source]

Bases: object

exception spack.package_base.ExtensionError(message, long_msg=None)[source]

Bases: PackageError

Superclass for all errors having to do with extension packages.

exception spack.package_base.InvalidPackageOpError(message, long_msg=None)[source]

Bases: PackageError

Raised when someone tries perform an invalid operation on a package.

exception spack.package_base.ManualDownloadRequiredError(message, long_msg=None)[source]

Bases: InvalidPackageOpError

Raised when attempting an invalid operation on a package that requires a manual download.

class spack.package_base.PackageBase(spec: Spec)[source]

Bases: WindowsRPath, PackageViewMixin

This is the universal base class for all Spack packages.

At its core, a package consists of a set of software to be installed. A package may focus on a piece of software and its associated software dependencies or it may simply be a set, or bundle, of software. The former requires defining how to fetch, verify (via, e.g., sha256), build, and install that software and the packages it depends on, so that dependencies can be installed along with the package itself. The latter, sometimes referred to as a “no-source” package, requires only defining the packages to be built.

There are two main parts of a Spack package:

  1. The package class. Classes contain directives, which are functions such as spack.package.version(), spack.package.patch(), and spack.package.depends_on(), that store metadata on the package class. Directives provide the constraints that are used as input to the concretizer.

  2. Package instances. Once instantiated with a concrete spec, a package can be passed to the spack.installer.PackageInstaller. It calls methods like do_stage() on the package instance, and it uses those to drive user-implemented methods like def patch and install phases like def configure and def install.

Packages are imported from package repositories (see spack.repo).

For most use cases, package creators typically just add attributes like homepage and, for a code-based package, url, or installation phases such as install(). There are many custom PackageBase subclasses in the spack_repo.builtin.build_systems package that make things even easier for specific build systems.

Note

Many methods and attributes that appear to be public interface are not meant to be overridden by packagers. They are “final”, but we currently have not adopted the @final decorator in the Spack codebase. For example, the do_* functions are intended only to be called internally by Spack commands. These aren’t for package writers to override, and doing so may break the functionality of the PackageBase class.

classmethod all_patches()[source]

Retrieve all patches associated with the package.

Retrieves patches on the package itself as well as patches on the dependencies of the package.

property all_urls: List[str]

A list of all URLs in a package.

Check both class-level and version-specific URLs.

Returns a list of URLs

all_urls_for_version(version: StandardVersion) → List[str][source]

Return all URLs derived from version_urls(), url, urls, and list_url (if it contains a version) in a package in that order.

Parameters:

version – the version for which a URL is sought

archive_install_test_log()[source]

Archive the install-phase test log, if present.

build_system_class: str

Used when reporting the build system to users

classmethod build_system_flags(name: str, flags: Iterable[str]) → Tuple[Iterable[str] | None, Iterable[str] | None, Iterable[str] | None][source]

See spack.package.build_system_flags().

property cmake_prefix_paths: List[str]

Return a list of paths to be used in CMake’s CMAKE_PREFIX_PATH.

property command: Executable

Returns the main executable for this package.

compiler
property configure_args_path

Return the configure args file path associated with staging.

conflicts: Dict[Spec, List[Tuple[Spec, str | None]]]

Class level dictionary populated by conflicts() directives

content_hash(content: bytes | None = None) → str[source]

Create a hash based on the artifacts and patches used to build this package.

This includes:

  • source artifacts (tarballs, repositories) used to build;

  • content hashes (sha256’s) of all patches applied by Spack; and

  • canonicalized contents the package.py recipe used to build.

This hash is only included in Spack’s DAG hash for concrete specs, but if it happens to be called on a package with an abstract spec, only applicable (i.e., determinable) portions of the hash will be included.

default_buildsystem: str

Must be defined as a fallback for old specs that don’t have the build_system variant

dependencies: Dict[Spec, Dict[str, Dependency]]

Class level dictionary populated by depends_on() and extends() directives

classmethod dependencies_by_name(when: bool = False)[source]
classmethod dependencies_of_type(deptypes: int)[source]

Get names of dependencies that can possibly have these deptypes.

This analyzes the package and determines which dependencies can be a certain kind of dependency. Note that they may not always be this kind of dependency, since dependencies can be optional, so something may be a build dependency in one configuration and a run dependency in another.

classmethod dependency_names()[source]
detect_dev_src_change() → bool[source]

Method for checking for source code changes to trigger rebuild/reinstall

disable_redistribute: Dict[Spec, DisableRedistribute]

Class level dictionary populated by redistribute() directives

do_clean()[source]

Removes the package’s build stage and source tarball.

do_fetch(mirror_only=False)[source]

Creates a stage directory and downloads the tarball for this package. Working directory will be set to the stage directory.

do_patch()[source]

Applies patches if they haven’t been applied already.

do_restage()[source]

Reverts expanded/checked out source to a pristine state.

do_stage(mirror_only=False)[source]

Unpacks and expands the fetched tarball.

do_test(*, dirty=False, externals=False, timeout: int | None = None)[source]
do_uninstall(force=False)[source]

Uninstall this package by spec.

property download_instr: str

Defines the default manual download instructions. Packages can override the property to provide more information.

Returns:

default manual download instructions

classmethod env_flags(name: str, flags: Iterable[str]) → Tuple[Iterable[str] | None, Iterable[str] | None, Iterable[str] | None][source]

See spack.package.env_flags().

property env_mods_path

Return the build environment modifications file path associated with staging.

property env_path

Return the build environment file path associated with staging.

extendable: bool

Most packages are NOT extendable. Set to True if you want extensions.

property extendee_spec: Spec | None

Spec of the extendee of this package, or None if it is not an extension.

extendees: Dict[str, Tuple[Spec, Spec]]

Class level dictionary populated by extends() directives

extends(spec: Spec) → bool[source]

Returns True if this package extends the given spec.

If self.spec is concrete, this returns whether this package extends the given spec.

If self.spec is not concrete, this returns whether this package may extend the given spec.

fetch_options: Dict[str, Any]

Set of additional options used when fetching package versions.

fetch_remote_versions(concurrency: int | None = None) → Dict[StandardVersion, str][source]

Find remote versions of this package.

Uses list_url and any other URLs listed in the package file.

Returns:

a dictionary mapping versions to URLs

property fetcher
find_valid_url_for_version(version: StandardVersion) → str | None[source]

Returns a URL from which the specified version of this package may be downloaded after testing whether the url is valid. Will try url, urls, and list_url before failing.

Parameters:

version – The version for which a URL is sought.

property flag_handler: Callable[[str, Iterable[str]], Tuple[Iterable[str] | None, Iterable[str] | None, Iterable[str] | None]]
flags_to_build_system_args(flags: Dict[str, List[str]]) → None[source]
classmethod format_doc(**kwargs)[source]

Wrap doc string at 72 characters and format nicely

fullname

Name of this package, including the namespace

fullnames

Fullnames for this package and any packages from which it inherits.

get_variant(name: str) → Variant[source]

Get the highest precedence variant definition matching this package’s spec.

Parameters:

name – name of the variant definition to get

global_license_dir

Returns the directory where license files for all packages are stored.

property global_license_file

Returns the path where a global license file for this particular package should be stored.

has_code: bool

Most Spack packages are used to install source or binary code while those that do not can be used to install a set of other Spack packages.

classmethod has_variant(name) → bool[source]
property home
homepage: str | None | classproperty[str | None]

Package homepage where users can find more information about the package

classmethod inject_flags(name: str, flags: Iterable[str]) → Tuple[Iterable[str] | None, Iterable[str] | None, Iterable[str] | None][source]

See spack.package.inject_flags().

property install_configure_args_path

Return the configure args file path on successful installation.

property install_env_path

Return the build environment file path on successful installation.

property install_log_path

Return the (compressed) build log file path on successful installation

intersects(spec: Spec) → bool[source]

Context-ful intersection that takes into account package information.

By design, Spec.intersects() does not know anything about package metadata. This avoids unnecessary package lookups and keeps things efficient where extra information is not needed, and it decouples Spec from PackageBase.

In many cases, though, we can rule more cases out in intersects() if we know, for example, that certain variants are always single-valued, or that certain variants are conditional on other variants. This adds logic for such cases when they are knowable.

Note that because intersects() is conservative, it can only give false positives (“i.e., the two specs may overlap”), not false negatives. This method can fix false positives (i.e. it may return False when Spec.intersects() would return True, but it will never return True when Spec.intersects() returns False.

property is_extension
property keep_werror: Literal['all', 'specific', 'none'] | None

Keep -Werror flags, matches config:flags:keep_werror to override config.

Valid return values are:

  • "all": keep all -Werror flags.

  • "specific": keep only -Werror=specific-warning flags.

  • "none": filter out all -Werror* flags.

  • None: respect the user’s configuration ("none" by default).

legacy_buildsystem: str

Use default_buildsystem instead of this attribute, which is deprecated

license_comment: str

Contains the symbol used by the license manager to denote a comment. Defaults to #.

license_files: List[str]

These are files that the software searches for when looking for a license. All file paths must be relative to the installation directory. More complex packages like Intel may require multiple licenses for individual components. Defaults to the empty list.

license_required: bool

If set to True, this software requires a license. If set to False, all of the license_* attributes will be ignored. Defaults to False.

license_url: str

A URL pointing to license setup instructions for the software. Defaults to the empty string.

license_vars: List[str]

Environment variables that can be set to tell the software where to look for a license if it is not in the usual location. Defaults to the empty list.

licenses: Dict[Spec, str]

Class level dictionary populated by license() directives

list_depth: int

Link depth to which list_url should be searched for new versions

list_url: str | None | classproperty[str | None]

Default list URL (place to find available versions)

property log_path

Return the build log file path associated with staging.

maintainers: List[str]

List of GitHub usernames of package maintainers. Do not include @ here in order not to unnecessarily ping the users.

manual_download: bool

Boolean. Set to True for packages that require a manual download. This is currently used by package sanity tests and generation of a more meaningful fetch failure error.

property metadata_dir

Return the install metadata directory.

module

Module instance that this package class is defined in.

We use this to add variables to package modules. This makes install() methods easier to write (e.g., can call configure())

name

The name of this package.

namespace

Spack namespace for the package, which identifies its repo.

nearest_url(version)[source]

Finds the URL with the “closest” version to version.

This uses the following precedence order:

  1. Find the next lowest or equal version with a URL.

  2. If no lower URL, return the next higher URL.

  3. If no higher URL, return None.

classmethod needs_commit(version) → bool[source]

Method for checking if the package instance needs a commit sha to be found

non_bindable_shared_objects: List[str]

List of shared objects that should be replaced with a different library at runtime. Typically includes stub libraries like libcuda.so. When linking against a library listed here, the dependent will only record its soname or filename, not its absolute path, so that the dynamic linker will search for it. Note: accepts both file names and directory names, for example ["libcuda.so", "stubs"] will ensure libcuda.so and all libraries in the stubs directory are not bound by path.

classmethod num_variant_definitions() → int[source]

Total number of variant definitions in this class so far.

package_dir

Directory where the package.py file lives.

parallel: bool

By default we build in parallel. Subclasses can override this.

patches: Dict[Spec, List[Patch]]

Class level dictionary populated by patch() directives

property phase_log_files

Find sorted phase log files written to the staging directory

property prefix

Get the prefix into which this package should be installed.

provided: Dict[Spec, Set[Spec]]

Class level dictionary populated by provides() directives

provided_together: Dict[Spec, List[Set[str]]]

Class level dictionary populated by provides() directives

classmethod provided_virtual_names()[source]

Return sorted list of names of virtuals that can be provided by this package.

provides(vpkg_name: str) → bool[source]

True if this package provides a virtual package with the specified name

property redistribute_binary

Whether it should be possible to create a binary out of an installed instance of this package.

classmethod redistribute_source(spec)[source]

Whether it should be possible to add the source of this package to a Spack mirror.

remove_prefix()[source]

Removes the prefix for a package along with any empty parent directories

requirements: Dict[Spec, List[Tuple[Tuple[Spec, ...], str, str | None]]]

Class level dictionary populated by requires() directives

resolve_binary_provenance()[source]

Method to ensure concrete spec has binary provenance. Base implementation will look up git commits when appropriate. Packages may override this implementation for custom implementations

resources: Dict[Spec, List[Resource]]

Class level dictionary populated by resource() directives

property rpath

Get the rpath this package links with, as a list of paths.

property rpath_args

Get the rpath args as a string, with -Wl,-rpath, for each element

run_tests: bool

By default do not run tests within package’s install()

sanity_check_is_dir: List[str]

List of prefix-relative directory paths (or a single path). If these do not exist after install, or if they exist but are not directories, sanity checks will fail.

sanity_check_is_file: List[str]

List of prefix-relative file paths (or a single path). If these do not exist after install, or if they exist but are not files, sanity checks fail.

setup_dependent_package(module, dependent_spec: Spec) → None[source]

Set up module-scope global variables for dependent packages.

This function is called when setting up the build and run environments of a DAG.

Examples:

  1. Extensions often need to invoke the python interpreter from the Python installation being extended. This routine can put a python Executable as a global in the module scope for the extension package to simplify extension installs.

  2. MPI compilers could set some variables in the dependent’s scope that point to mpicc, mpicxx, etc., allowing them to be called by common name regardless of which MPI is used.

Parameters:

  • module – The Python module object of the dependent package. Packages can use this to set module-scope variables for the dependent to use.

  • dependent_spec – The spec of the dependent package about to be built. This allows the extendee (self) to query the dependent’s state. Note that this package’s spec is available as self.spec.

setup_dependent_run_environment(env: EnvironmentModifications, dependent_spec: Spec) → None[source]

Sets up the run environment of packages that depend on this one.

This is similar to setup_run_environment, but it is used to modify the run environment of a package that depends on this one.

This gives packages like Python and others that follow the extension model a way to implement common environment or run-time settings for dependencies.

Parameters:

  • env – environment modifications to be applied when the dependent package is run. Package authors can call methods on it to alter the build environment.

  • dependent_spec – The spec of the dependent package about to be run. This allows the extendee (self) to query the dependent’s state. Note that this package’s spec is available as self.spec

setup_run_environment(env: EnvironmentModifications) → None[source]

Sets up the run environment for a package.

Parameters:

env – environment modifications to be applied when the package is run. Package authors can call methods on it to alter the run environment.

splice_specs: Dict[Spec, Tuple[Spec, None | str | List[str]]]

Class level dictionary populated by can_splice() directives

property stage

Get the build staging area for this package.

This automatically instantiates a Stage object if the package doesn’t have one yet, but it does not create the Stage directory on the filesystem.

test_requires_compiler: bool

Set to True to indicate the stand-alone test requires a compiler. It is used to ensure a compiler and build dependencies like cmake are available to build a custom test code.

test_suite: Any | None

TestSuite instance used to manage stand-alone tests for 1+ specs.

property tester
property times_log_path

Return the times log json file.

transitive_rpaths: bool

When True, add RPATHs for the entire DAG. When False, add RPATHs only for immediate dependencies.

static uninstall_by_spec(spec, force=False, deprecator=None)[source]
unresolved_libraries: List[str]

List of fnmatch patterns of library file names (specifically DT_NEEDED entries) that are not expected to be locatable in RPATHs. Generally this is a problem, and Spack install with config:shared_linking:strict will cause install failures if such libraries are found. However, in certain cases it can be hard if not impossible to avoid accidental linking against system libraries; until that is resolved, this attribute can be used to suppress errors.

url_for_version(version: str | StandardVersion) → str[source]

Returns a URL from which the specified version of this package may be downloaded.

Parameters:

version – The version for which a URL is sought.

url_version(version)[source]

Given a version, this returns a string that should be substituted into the package’s URL to download that version.

By default, this just returns the version string. Subclasses may need to override this, e.g. for boost versions where you need to ensure that there are _’s in the download URL.

classmethod validate_variant_names(spec: Spec)[source]

Check that all variant names on Spec exist in this package.

Raises UnknownVariantError if invalid variants are on the spec.

classmethod variant_definitions(name: str) → List[Tuple[Spec, Variant]][source]

Iterator over (when_spec, Variant) for all variant definitions for a particular name.

classmethod variant_items() → Iterable[Tuple[Spec, Dict[str, Variant]]][source]

Iterate over cls.variants.items() with overridden definitions removed.

classmethod variant_names() → List[str][source]
variants: Dict[Spec, Dict[str, Variant]]

Class level dictionary populated by variant() directives

property version
classmethod version_or_package_attr(attr, version, default=NO_DEFAULT)[source]

Get an attribute that could be on the version or package with preference to the version

classmethod version_urls() → Dict[StandardVersion, str][source]

Dict of explicitly defined URLs for versions of this package.

Returns:

An dict mapping version to url, ordered by version.

A version’s URL only appears in the result if it has an an explicitly defined url argument. So, this list may be empty if a package only defines url at the top level.

versions: Dict[StandardVersion, Dict[str, Any]]

Class level dictionary populated by version() directives

view()[source]

Create a view with the prefix of this package as the root. Extensions added to this view will modify the installation prefix of this package.

virtual: bool

By default, packages are not virtual Virtual packages override this attribute

property virtuals_provided

virtual packages provided by this package with its spec

class spack.package_base.PackageMeta(name, bases, attr_dict)[source]

Bases: PhaseCallbacksMeta, DetectablePackageMeta, DirectiveMeta, MultiMethodMeta

Package metaclass for supporting directives (e.g., depends_on) and phases

exception spack.package_base.PackageStillNeededError(spec, dependents)[source]

Bases: InstallError

Raised when package is still needed by another on uninstall.

class spack.package_base.PackageViewMixin[source]

Bases: object

This collects all functionality related to adding installed Spack package to views. Packages can customize how they are added to views by overriding these functions.

add_files_to_view(view, merge_map, skip_if_exists=True)[source]

Given a map of package files to destination paths in the view, add the files to the view. By default this adds all files. Alternative implementations may skip some files, for example if other packages linked into the view already include the file.

Parameters:

  • view (spack.filesystem_view.FilesystemView) – the view that’s updated

  • merge_map (dict) – maps absolute source paths to absolute dest paths for all files in from this package.

  • skip_if_exists (bool) – when True, don’t link files in view when they already exist. When False, always link files, without checking if they already exist.

remove_files_from_view(view, merge_map)[source]

Given a map of package files to files currently linked in the view, remove the files from the view. The default implementation removes all files. Alternative implementations may not remove all files. For example if two packages include the same file, it should only be removed when both packages are removed.

spec: Spec
view_destination(view)[source]

The target root directory: each file is added relative to this directory.

view_file_conflicts(view, merge_map)[source]

Report any files which prevent adding this package to the view. The default implementation looks for any files which already exist. Alternative implementations may allow some of the files to exist in the view (in this case they would be omitted from the results).

view_source()[source]

The source root directory that will be added to the view: files are added such that their path relative to the view destination matches their path relative to the view source.

class spack.package_base.WindowsRPath[source]

Bases: object

Collection of functionality surrounding Windows RPATH specific features

This is essentially meaningless for all other platforms due to their use of RPATH. All methods within this class are no-ops on non Windows. Packages can customize and manipulate this class as they would a genuine RPATH, i.e. adding directories that contain runtime library dependencies

win_add_library_dependent()[source]

Return extra set of directories that require linking for package

This method should be overridden by packages that produce binaries/libraries/python extension modules/etc that are installed into directories outside a package’s bin, lib, and lib64 directories, but still require linking against one of the packages dependencies, or other components of the package itself. No-op otherwise.

Returns:

List of additional directories that require linking

win_add_rpath()[source]

Return extra set of rpaths for package

This method should be overridden by packages needing to include additional paths to be searched by rpath. No-op otherwise

Returns:

List of additional rpaths

windows_establish_runtime_linkage()[source]

Establish RPATH on Windows

Performs symlinking to incorporate rpath dependencies to Windows runtime search paths

class spack.package_base.WindowsSimulatedRPath(package: PackageBase, base_modification_prefix: str | Path | None = None, link_install_prefix: bool = True)[source]

Bases: object

Class representing Windows filesystem rpath analog

One instance of this class is associated with a package (only on Windows) For each lib/binary directory in an associated package, this class introduces a symlink to any/all dependent libraries/binaries. This includes the packages own bin/lib directories, meaning the libraries are linked to the binary directory and vis versa.

add_library_dependent(*dest: str | Path)[source]

Add paths to directories or libraries/binaries to set of common paths that need to link against other libraries

Specified paths should fall outside of a package’s common link paths, i.e. the bin directories.

add_rpath(*paths: str)[source]

Add libraries found at the root of provided paths to runtime linking

These are libraries found outside of the typical scope of rpath linking that require manual inclusion in a runtime linking scheme. These links are unidirectional, and are only intended to bring outside dependencies into this package

Parameters:

*paths – arbitrary number of paths to be added to runtime linking

(sym)link packages to runtime dependencies based on RPath configuration for Windows heuristics

property library_dependents

Set of directories where package binaries/libraries are located.

property rpaths

Set of libraries this package needs to link against during runtime These packages will each be symlinked into the packages lib and binary dir

spack.package_base.concretization_version_order(version_info: Tuple[GitVersion | StandardVersion, dict]) → Tuple[bool, bool, bool, bool, GitVersion | StandardVersion][source]

Version order key for concretization, where preferred > not preferred, finite > any infinite component; only if all are the same, do we use default version ordering.

Version deprecation needs to be accounted for separately.

spack.package_base.deprecated_version(pkg: PackageBase, version: str | StandardVersion) → bool[source]

Return True iff the version is deprecated.

Parameters:

  • pkg – The package whose version is to be checked.

  • version – The version being checked

spack.package_base.detectable_packages

Registers which are the detectable packages, by repo and package name Need a pass of package repositories to be filled.

spack.package_base.make_package_test_rpath(pkg: PackageBase, test_dir: str | Path) → None[source]

Establishes a temp Windows simulated rpath for the pkg in the testing directory so an executable can test the libraries/executables with proper access to dependent dlls.

Note: this is a no-op on all other platforms besides Windows

Parameters:

  • pkg – the package for which the rpath should be computed

  • test_dir – the testing directory in which we should construct an rpath

spack.package_base.non_default_variant(node: Spec, variant_name: str) → bool[source]

Returns True if the variant in the spec has a non-default value.

spack.package_base.non_preferred_version(node: Spec) → bool[source]

Returns True if the spec version is not the preferred one, according to the package.py

spack.package_base.on_package_attributes(**attr_dict)[source]

Decorator: executes instance function only if object has attr values.

Executes the decorated method only if at the moment of calling the instance has attributes that are equal to certain values.

Parameters:

attr_dict (dict) – dictionary mapping attribute names to their required values

spack.package_base.preferred_version(pkg: PackageBase | Type[PackageBase]) → StandardVersion | GitVersion[source]

Returns the preferred versions of the package according to package.py.

Accounts for version deprecation in the package recipe. Doesn’t account for any user configuration in packages.yaml.

Parameters:

pkg – The package whose versions are to be assessed.

spack.package_base.sort_by_pkg_preference(versions: Iterable[GitVersion | StandardVersion], *, pkg: PackageBase | Type[PackageBase]) → List[GitVersion | StandardVersion][source]

Sorts the list of versions passed in input according to the preferences in the package. The return value does not contain duplicate versions. Most preferred versions first.

spack.package_base.spack_times_log

Filename of json with total build and phase times (seconds)

spack.package_completions module

spack.package_completions.bash_completion_path(root: str | Path) → Path[source]

Return standard path for bash completion scripts under root.

Parameters:

root – The prefix root under which to generate the path.

Returns:

Standard path for bash completion scripts under root.

spack.package_completions.fish_completion_path(root: str | Path) → Path[source]

Return standard path for fish completion scripts under root.

Parameters:

root – The prefix root under which to generate the path.

Returns:

Standard path for fish completion scripts under root.

spack.package_completions.zsh_completion_path(root: str | Path) → Path[source]

Return standard path for zsh completion scripts under root.

Parameters:

root – The prefix root under which to generate the path.

Returns:

Standard path for zsh completion scripts under root.

spack.package_prefs module

class spack.package_prefs.PackagePrefs(pkgname, component, vpkg=None, all=True)[source]

Bases: object

Defines the sort order for a set of specs.

Spack’s package preference implementation uses PackagePrefs to define sort order. The PackagePrefs class looks at Spack’s packages.yaml configuration and, when called on a spec, returns a key that can be used to sort that spec in order of the user’s preferences.

You can use it like this:

# key function sorts CompilerSpecs for `mpich` in order of preference
kf = PackagePrefs("mpich", "compiler")
compiler_list.sort(key=kf)

Or like this:

# key function to sort VersionLists for OpenMPI in order of preference.
kf = PackagePrefs("openmpi", "version")
version_list.sort(key=kf)

Optionally, you can sort in order of preferred virtual dependency providers. To do that, provide "providers" and a third argument denoting the virtual package (e.g., mpi):

kf = PackagePrefs("trilinos", "providers", "mpi")
provider_spec_list.sort(key=kf)
classmethod has_preferred_providers(pkgname, vpkg)[source]

Whether specific package has a preferred vpkg providers.

classmethod has_preferred_targets(pkg_name)[source]

Whether specific package has a preferred vpkg providers.

classmethod order_for_package(pkgname, component, vpkg=None, all=True)[source]

Given a package name, sort component (e.g, version, compiler, …), and an optional vpkg, return the list from the packages config.

classmethod preferred_variants(pkg_name)[source]

Return a VariantMap of preferred variants/values for a spec.

exception spack.package_prefs.VirtualInPackagesYAMLError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when a disallowed virtual is found in packages.yaml

spack.package_prefs.get_package_dir_permissions(spec)[source]

Return the permissions configured for the spec.

Include the GID bit if group permissions are on. This makes the group attribute sticky for the directory. Package-specific settings take precedent over settings for all

spack.package_prefs.get_package_group(spec)[source]

Return the unix group associated with the spec.

Package-specific settings take precedence over settings for all

spack.package_prefs.get_package_permissions(spec)[source]

Return the permissions configured for the spec.

Package-specific settings take precedence over settings for all

spack.package_prefs.is_spec_buildable(spec)[source]

Return true if the spec is configured as buildable

spack.package_test module

spack.package_test.compare_output(current_output: str, blessed_output: str) → None[source]

Compare blessed and current output of executables. Used in package tests.

spack.package_test.compare_output_file(current_output: str, blessed_output_file: str) → None[source]

Same as above, but when the blessed output is given as a file. Used in package tests.

spack.package_test.compile_c_and_execute(source_file: str, include_flags: List[str], link_flags: List[str]) → str[source]

Compile a C source file with the given include and link flags, execute the resulting binary, and return its output as a string. Used in package tests.

spack.patch module

class spack.patch.FilePatch(pkg: PackageBase | Type[PackageBase], relative_path: str, level: int, working_dir: str, reverse: bool = False, ordering_key: Tuple[str, int] | None = None)[source]

Bases: Patch

Describes a patch that is retrieved from a file in the repository.

property sha256: str

Get the patch checksum.

Returns:

The sha256 of the patch file.

to_dict() → Dict[str, Any][source]

Dictionary representation of the patch.

Returns:

A dictionary representation.

class spack.patch.Patch(pkg: PackageBase | Type[PackageBase], path_or_url: str, level: int, working_dir: str, reverse: bool = False, ordering_key: Tuple[str, int] | None = None)[source]

Bases: object

Base class for patches.

The owning package is not necessarily the package to apply the patch to – in the case where a dependent package patches its dependency, it is the dependent’s fullname.

sha256: str
to_dict() → Dict[str, Any][source]

Dictionary representation of the patch.

Returns:

A dictionary representation.

class spack.patch.PatchCache(repository: RepoPath, data: Dict[str, Any] | None = None)[source]

Bases: object

Index of patches used in a repository, by sha256 hash.

This allows us to look up patches without loading all packages. It’s also needed to properly implement dependency patching, as need a way to look up patches that come from packages not in the Spec sub-DAG.

The patch index is structured like this in a file (this is YAML, but we write JSON):

patches:
    sha256:
        namespace1.package1:
            <patch json>
        namespace2.package2:
            <patch json>
        ... etc. ...
classmethod from_json(stream: Any, repository: RepoPath) → PatchCache[source]

Initialize a new PatchCache instance from JSON.

Parameters:

  • stream – stream of data

  • repository – repository containing package

Returns:

A new PatchCache instance.

patch_for_package(sha256: str, pkg: Type[PackageBase], *, validate: bool = False) → Patch[source]

Look up a patch in the index and build a patch object for it.

We build patch objects lazily because building them requires that we have information about the package’s location in its repo.

Parameters:

  • sha256 – sha256 hash to look up

  • pkg – Package class to get patch for.

  • validate – if True, validate the cached entry against the owner’s current package class and raise PatchLookupError if the entry is missing or stale.

Returns:

The patch object.

to_json(stream: Any) → None[source]

Dump a JSON representation to a stream.

Parameters:

stream – stream of data

update(other: PatchCache) → None[source]

Update this cache with the contents of another.

Parameters:

other – another patch cache to merge

update_packages(pkgs_fullname: Set[str]) → None[source]

Update the patch cache.

Parameters:

pkg_fullname – package to update.

class spack.patch.UrlPatch(pkg: PackageBase | Type[PackageBase], url: str, level: int = 1, *, working_dir: str = '.', reverse: bool = False, sha256: str, ordering_key: Tuple[str, int] | None = None, archive_sha256: str | None = None)[source]

Bases: Patch

Describes a patch that is retrieved from a URL.

fetcher() → FetchStrategy[source]

Construct a fetcher that can download (and unpack) this patch.

to_dict() → Dict[str, Any][source]

Dictionary representation of the patch.

Returns:

A dictionary representation.

spack.patch.apply_patch(source_path: str, patch_path: str, level: int = 1, working_dir: str = '.', reverse: bool = False) → None[source]

Apply the patch at patch_path to code in the stage.

Parameters:

  • stage – stage with code that will be patched

  • patch_path – filesystem location for the patch to apply

  • level – patch level

  • working_dir – relative path within the stage to change to

  • reverse – reverse the patch

spack.patch.from_dict(dictionary: Dict[str, Any], repository: RepoPath) → Patch[source]

Create a patch from json dictionary.

Parameters:

  • dictionary – dictionary representation of a patch

  • repository – repository containing package

Returns:

A patch object.

Raises:

ValueError – If owner or url/relative_path are missing in the dictionary.

spack.paths module

Defines paths that are part of Spack’s directory structure.

Do not import other spack modules here. This module is used throughout Spack and should bring in a minimal number of external dependencies.

spack.paths.bin_path

bin directory in the spack prefix

spack.paths.default_misc_cache_path

transient caches for Spack data (virtual cache, patch sha256 lookup, etc.)

spack.paths.default_monitor_path

spack monitor analysis directories

spack.paths.default_test_path

installation test (spack test) output

spack.paths.default_user_bootstrap_path

bootstrap store for bootstrapping clingo and other tools

spack.paths.package_repos_path

default location where remote package repositories are cloned

spack.paths.prefix

This file lives in $prefix/lib/spack/spack/__file__

spack.paths.reports_path

junit, cdash, etc. reports about builds

spack.paths.sbang_script

The sbang script in the spack installation

spack.paths.set_working_dir()[source]

Change the working directory to getcwd, or spack prefix if no cwd.

spack.paths.spack_instance_id

Not a location itself, but used for when Spack instances share the same cache base directory for caches that should not be shared between those instances.

spack.paths.spack_root

synonym for prefix

spack.paths.spack_script

The spack script itself

spack.paths.spack_working_dir

Recorded directory where spack command was originally invoked

spack.paths.system_config_path

System configuration location

spack.paths.user_config_path

User configuration location

spack.paths.user_repos_cache_path

git repositories fetched to compare commits to versions

spack.phase_callbacks module

class spack.phase_callbacks.CallbackTemporaryStage(attribute_name, callbacks)

Bases: tuple

An object of this kind is a shared global state used to collect callbacks during class definition time, and is flushed when the class object is created at the end of the class definition

Parameters:

  • attribute_name (str) – name of the attribute that will be attached to the builder

  • callbacks (list) – container used to temporarily aggregate the callbacks

attribute_name

Alias for field number 0

callbacks

Alias for field number 1

class spack.phase_callbacks.PhaseCallbacksMeta(name, bases, attr_dict)[source]

Bases: type

Permit to register arbitrary functions during class definition and run them later, before or after a given install phase.

Each method decorated with run_before or run_after gets temporarily stored in a global shared state when a class being defined is parsed by the Python interpreter. At class definition time that temporary storage gets flushed and a list of callbacks is attached to the class being defined.

static run_after(phase: str, when: str | None = None)[source]

Decorator to register a function for running after a given phase.

Example:

@run_after("install", when="@2:")
def install_missing_files(self):
    # Do something after the install phase for versions 2.x and above
    pass
Parameters:

  • phase – phase after which the function must run.

  • when – condition under which the function is run (if None, it is always run).

static run_before(phase: str, when: str | None = None)[source]

Decorator to register a function for running before a given phase.

Example:

@run_before("build", when="@2:")
def patch_generated_source_file(pkg):
    # Do something before the build phase for versions 2.x and above
    pass
Parameters:

  • phase – phase before which the function must run.

  • when – condition under which the function is run (if None, it is always run).

spack.phase_callbacks.run_after(phase: str, when: str | None = None)

Decorator to register a function for running after a given phase.

Example:

@run_after("install", when="@2:")
def install_missing_files(self):
    # Do something after the install phase for versions 2.x and above
    pass
Parameters:

  • phase – phase after which the function must run.

  • when – condition under which the function is run (if None, it is always run).

spack.phase_callbacks.run_before(phase: str, when: str | None = None)

Decorator to register a function for running before a given phase.

Example:

@run_before("build", when="@2:")
def patch_generated_source_file(pkg):
    # Do something before the build phase for versions 2.x and above
    pass
Parameters:

  • phase – phase before which the function must run.

  • when – condition under which the function is run (if None, it is always run).

spack.projections module

spack.projections.get_projection(projections, spec)[source]

Get the projection for a spec from a projections dict.

spack.provider_index module

Classes and functions to manage providers of virtual dependencies

class spack.provider_index.ProviderIndex(repository: Repo | RepoPath, specs: Iterable[Spec] | None = None, restrict: bool = False)[source]

Bases: object

copy()[source]

Return a deep copy of this index.

static from_json(stream, repository)[source]

Construct a provider index from its JSON representation.

Parameters:

stream – stream where to read from the JSON data

merge(other)[source]

Merge another provider index into this one.

Parameters:

other (ProviderIndex) – provider index to be merged

providers: Dict[str, Dict[Spec, Set[Spec]]]

This is a dict of dicts used for finding providers of particular virtual dependencies. The dict of dicts looks like:

{ vpkg name :

{ full vpkg spec : set(packages providing spec) } }

Callers can use this to first find which packages provide a vpkg, then find a matching full spec. e.g., in this scenario:

{ ‘mpi’ :
{ mpi@:1.1set([mpich]),

mpi@:2.3 : set([mpich2@1.9:]) } }

Calling providers_for(spec) will find specs that provide a matching implementation of MPI. Derived class need to construct this attribute according to the semantics above.

providers_for(virtual: str | Spec) → List[Spec][source]

Return a list of specs of all packages that provide virtual packages with the supplied spec.

Parameters:

virtual – either a Spec or a string name of a virtual package

remove_providers(pkg_names: Set[str])[source]

Remove a provider from the ProviderIndex.

to_json(stream=None)[source]

Dump a JSON representation of this object.

Parameters:

stream – stream where to dump

update_packages(specs: Iterable[str | Spec])[source]

Update the provider index with additional virtual specs.

Parameters:

spec – spec potentially providing additional virtual specs

exception spack.provider_index.ProviderIndexError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when there is a problem with a ProviderIndex.

spack.relocate module

spack.relocate.fixup_macos_rpath(root, filename)[source]

Apply rpath fixups to the given file.

Parameters:

  • root – absolute path to the parent directory

  • filename – relative path to the library or binary

Returns:

True if fixups were applied, else False

spack.relocate.fixup_macos_rpaths(spec)[source]

Remove duplicate and nonexistent rpaths.

Some autotools packages write their own -rpath entries in addition to those implicitly added by the Spack compiler wrappers. On Linux these duplicate rpaths are eliminated, but on macOS they result in multiple entries which makes it harder to adjust with install_name_tool -delete_rpath.

spack.relocate.is_binary(filename: str) → bool[source]

Returns true iff a file is likely binary

spack.relocate.is_elf_magic(magic: bytes) → bool[source]
spack.relocate.is_macho_binary(path: str) → bool[source]
spack.relocate.is_macho_magic(magic: bytes) → bool[source]
spack.relocate.relocate_elf_binaries(binaries: Iterable[str], prefix_to_prefix: Dict[str, str]) → None[source]

Take a list of binaries, and an ordered prefix to prefix mapping, and update the rpaths accordingly.

Relocate links to a new install prefix.

spack.relocate.relocate_macho_binaries(path_names, prefix_to_prefix)[source]

Use macholib python package to get the rpaths, dependent libraries and library identity for libraries from the MachO object. Modify them with the replacement paths queried from the dictionary mapping old layout prefixes to hashes and the dictionary mapping hashes to the new layout prefixes.

spack.relocate.relocate_text(files: Iterable[str], prefix_to_prefix: Dict[str, str] | Dict[bytes, bytes]) → None[source]

Relocate text file from the original installation prefix to the new prefix.

Relocation also affects the the path in Spack’s sbang script.

Parameters:

  • files – Text files to be relocated

  • prefix_to_prefix – ordered prefix to prefix mapping

spack.relocate.relocate_text_bin(binaries: Iterable[str], prefix_to_prefix: Dict[str, str] | Dict[bytes, bytes]) → List[str][source]

Replace null terminated path strings hard-coded into binaries.

The new install prefix must be shorter than the original one.

Parameters:

  • binaries – paths to binaries to be relocated

  • prefix_to_prefix – ordered prefix to prefix mapping

Raises:

spack.relocate_text.BinaryTextReplaceError – when the new path is longer than the old path

spack.relocate_text module

This module contains pure-Python classes and functions for replacing paths inside text files and binaries.

class spack.relocate_text.BinaryFilePrefixReplacer(prefix_to_prefix: Dict[bytes, bytes], suffix_safety_size: int = 7)[source]

Bases: PrefixReplacer

classmethod binary_text_regex(binary_prefixes: Iterable[bytes], suffix_safety_size: int = 7) → Pattern[bytes][source]

Create a regex that looks for exact matches of prefixes, and also tries to match a C-string type null terminator in a small lookahead window.

Parameters:

  • binary_prefixes – Iterable of byte strings of prefixes to match

  • suffix_safety_size – Sizeof the lookahed for null-terminated string.

classmethod from_strings_or_bytes(prefix_to_prefix: Dict[str, str] | Dict[bytes, bytes], suffix_safety_size: int = 7) → BinaryFilePrefixReplacer[source]

Create a BinaryFilePrefixReplacer from an ordered prefix to prefix map.

Parameters:

  • prefix_to_prefix – Ordered mapping of prefix to prefix.

  • suffix_safety_size – Number of bytes to retain at the end of a C-string to avoid binary string-aliasing issues.

exception spack.relocate_text.BinaryTextReplaceError(msg)[source]

Bases: SpackError

exception spack.relocate_text.CannotGrowString(old, new)[source]

Bases: BinaryTextReplaceError

exception spack.relocate_text.CannotShrinkCString(old, new, full_old_string)[source]

Bases: BinaryTextReplaceError

class spack.relocate_text.PrefixReplacer(prefix_to_prefix: Dict[bytes, bytes])[source]

Bases: object

Base class for applying a prefix to prefix map to a list of binaries or text files. Derived classes implement _apply_to_file to do the actual work, which is different when it comes to binaries and text files.

apply(filenames: Iterable[str]) → List[str][source]

Returns a list of files that were modified

apply_to_file(f: IO[bytes]) → bool[source]
apply_to_filename(filename: str) → bool[source]
property is_noop: bool

Returns true when the prefix to prefix map is mapping everything to the same location (identity) or there are no prefixes to replace.

class spack.relocate_text.TextFilePrefixReplacer(prefix_to_prefix: Dict[bytes, bytes])[source]

Bases: PrefixReplacer

This class applies prefix to prefix mappings for relocation on text files.

Note that UTF-8 encoding is assumed.

classmethod from_strings_or_bytes(prefix_to_prefix: Dict[str, str] | Dict[bytes, bytes]) → TextFilePrefixReplacer[source]

Create a TextFilePrefixReplacer from an ordered prefix to prefix map.

spack.relocate_text.encode_path(p: str | bytes) → bytes[source]
spack.relocate_text.filter_identity_mappings(prefix_to_prefix: Dict[bytes, bytes]) → Dict[bytes, bytes][source]

Drop mappings that are not changed.

spack.relocate_text.utf8_path_to_binary_regex(prefix: str) → Pattern[bytes][source]

Create a binary regex that matches the input path in utf8

spack.relocate_text.utf8_paths_to_single_binary_regex(prefixes: Iterable[str]) → Pattern[bytes][source]

Create a (binary) regex that matches any input path in utf8

spack.repo module

exception spack.repo.BadRepoError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when repo layout is invalid.

exception spack.repo.BadRepoVersionError(api, *args, **kwargs)[source]

Bases: BadRepoError

Raised when repo API version is too high or too low for Spack.

class spack.repo.BrokenRepoDescriptor(name: str | None, error: str)[source]

Bases: RepoDescriptor

A descriptor for a broken repository, used to indicate errors in the configuration that aren’t fatal until the repository is used.

construct(cache: FileCache, overrides: Dict[str, Any] | None = None) → Dict[str, Repo | Exception][source]

Construct Repo instances from the descriptor.

initialize(fetch: bool = True, git: Executable | None = None) → None[source]
exception spack.repo.FailedConstructorError(name, exc_type, exc_obj, exc_tb)[source]

Bases: RepoError

Raised when a package’s class constructor fails.

class spack.repo.FastPackageChecker(packages_path: str, package_api: Tuple[int, int])[source]

Bases: Mapping[str, float]

Cache that maps package names to the modification times of their package.py files.

For each repository a cache is maintained at class level, and shared among all instances referring to it. Update of the global cache is done lazily during instance initialization.

invalidate() → None[source]

Regenerate cache for this checker.

last_mtime() → float[source]
modified_since(since: float) → List[str][source]
class spack.repo.GitExe(packages_path: str)[source]

Bases: object

class spack.repo.Indexer(repository)[source]

Bases: object

Adaptor for indexes that need to be generated when repos are updated.

create()[source]
needs_update(pkg) → bool[source]

Whether an update is needed when the package file hasn’t changed.

Returns:

True iff this package needs its index updated.

We already automatically update indexes when package files change, but other files (like patches) may change underneath the package file. This method can be used to check additional package-specific files whenever they’re loaded, to tell the RepoIndex to update the index just for that package.

abstractmethod read(stream)[source]

Read this index from a provided file object.

abstractmethod update(pkgs_fullname: Set[str])[source]

Update the index in memory with information about a package.

abstractmethod write(stream)[source]

Write the index to a file object.

exception spack.repo.InvalidNamespaceError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when an invalid namespace is encountered.

class spack.repo.LocalRepoDescriptor(name: str | None, path: str)[source]

Bases: RepoDescriptor

construct(cache: FileCache, overrides: Dict[str, Any] | None = None) → Dict[str, Repo | Exception][source]

Construct Repo instances from the descriptor.

spack.repo.NOT_PROVIDED

Guaranteed unused default value for some functions.

exception spack.repo.NoRepoConfiguredError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when there are no repositories configured.

spack.repo.PATH

Global package repository instance.

class spack.repo.PatchIndexer(repository)[source]

Bases: Indexer

Lifecycle methods for patch cache.

needs_update()[source]

Whether an update is needed when the package file hasn’t changed.

Returns:

True iff this package needs its index updated.

We already automatically update indexes when package files change, but other files (like patches) may change underneath the package file. This method can be used to check additional package-specific files whenever they’re loaded, to tell the RepoIndex to update the index just for that package.

read(stream)[source]

Read this index from a provided file object.

update(pkgs_fullname: Set[str])[source]

Update the index in memory with information about a package.

write(stream)[source]

Write the index to a file object.

class spack.repo.ProviderIndexer(repository)[source]

Bases: Indexer

Lifecycle methods for virtual package providers.

read(stream)[source]

Read this index from a provided file object.

update(pkgs_fullname: Set[str])[source]

Update the index in memory with information about a package.

write(stream)[source]

Write the index to a file object.

class spack.repo.RemoteRepoDescriptor(*, name: str | None, repository: str, branch: str | None, commit: str | None, tag: str | None, destination: str, relative_paths: List[str] | None, lock: Lock)[source]

Bases: RepoDescriptor

construct(cache: FileCache, overrides: Dict[str, Any] | None = None) → Dict[str, Repo | Exception][source]

Construct Repo instances from the descriptor.

error: str | None
fetched() → bool[source]
get_commit(git: Executable | None = None)[source]
initialize(fetch: bool = True, git: Executable | None = None) → None[source]

Clone the remote repository if it has not been fetched yet and read the index file if necessary.

read_index_file() → None[source]
update(git: Executable | None = None, remote: str = 'origin') → None[source]
class spack.repo.Repo(root: str, *, cache: FileCache, overrides: Dict[str, Any] | None = None)[source]

Bases: object

Class representing a package repository in the filesystem.

Each package repository must have a top-level configuration file called repo.yaml.

It contains the following keys:

namespace

A Python namespace where the repository’s packages should live.

subdirectory

An optional subdirectory name where packages are placed

api

A string of the form vX.Y that indicates the Package API version. The default is v1.0. For the repo to be compatible with the current version of Spack, the version must be greater than or equal to spack.min_package_api_version and less than or equal to spack.package_api_version.

all_package_classes() → Generator[Type[PackageBase], None, None][source]

Iterator over all package classes in the repository.

Use this with care, because loading packages is slow.

all_package_names(include_virtuals: bool = False) → List[str][source]

Returns a sorted list of all package names in the Repo.

all_package_paths() → Generator[str, None, None][source]
dirname_for_package_name(pkg_name: str) → str[source]

Given a package name, get the directory containing its package.py file.

dump_provenance(spec: Spec, path: str) → None[source]

Dump provenance information for a spec to a particular path.

This dumps the package file and any associated patch files. Raises UnknownPackageError if not found.

exists(pkg_name: str) → bool[source]

Whether a package with the supplied name exists.

extensions_for(extendee_spec: Spec) → List[PackageBase][source]
filename_for_package_name(pkg_name: str) → str[source]

Get the filename for the module we should load for a particular package. Packages for a Repo live in $root/<package_name>/package.py

This will return a proper package.py path even if the package doesn’t exist yet, so callers will need to ensure the package exists before importing.

get(spec: Spec) → PackageBase[source]

Returns the package associated with the supplied spec.

get_patch_index(allow_stale: bool = False) → PatchCache[source]

Index of patches and packages they’re defined on. Set allow_stale is True to bypass cache validation and return a potentially stale index.

get_pkg_class(pkg_name: str) → Type[PackageBase][source]

Get the class for the package out of its module.

First loads (or fetches from cache) a module for the package. Then extracts the package class from the module according to Spack’s naming convention.

property index: RepoIndex

Construct the index for this repo lazily.

is_prefix(fullname: str) → bool[source]

True if fullname is a prefix of this Repo’s namespace.

is_virtual(pkg_name: str) → bool[source]

Return True if the package with this name is virtual, False otherwise.

This function use the provider index. If calling from a code block that is used to construct the provider index use the is_virtual_safe function.

is_virtual_safe(pkg_name: str) → bool[source]

Return True if the package with this name is virtual, False otherwise.

This function doesn’t use the provider index.

last_mtime()[source]

Time a package file in this repo was last updated.

marshal()[source]
namespace: str
property package_api_str: str
package_path(name: str) → str[source]

Get path to package.py file for this repo.

packages_with_tags(*tags: str) → Set[str][source]
partition_package_name(pkg_name: str) → Tuple[str, str][source]
property provider_index: ProviderIndex

A fresh provider index with names specific to this repo.

providers_for(virtual: str | Spec) → List[Spec][source]
python_path: str | None
real_name(import_name: str) → str | None[source]

Allow users to import Spack packages using Python identifiers.

In Package API v1.x, there was no canonical module name for a package, and package’s dir was not necessarily a valid Python module name. For that case we have to guess the actual package directory. From Package API v2.0 there is a one-to-one mapping between Spack package names and Python module names, so there is no guessing.

For Package API v1.x we support the following one-to-many mappings:

  • num3proxy -> 3proxy

  • foo_bar -> foo_bar, foo-bar

  • foo_bar_baz -> foo_bar_baz, foo-bar-baz, foo_bar-baz, foo-bar_baz

property tag_index: TagIndex

Fresh index of tags and which packages they’re defined on.

static unmarshal(root, cache, overrides)[source]

Helper method to unmarshal keyword arguments

class spack.repo.RepoDescriptor(name: str | None)[source]

Bases: object

Abstract base class for repository data.

construct(cache: FileCache, overrides: Dict[str, Any] | None = None) → Dict[str, Repo | Exception][source]

Construct Repo instances from the descriptor.

initialize(fetch: bool = True, git: Executable | None = None) → None[source]
update(git: Executable | None = None, remote: str = 'origin') → None[source]
class spack.repo.RepoDescriptors(descriptors: Dict[str, RepoDescriptor])[source]

Bases: Mapping[str, RepoDescriptor]

A collection of repository descriptors.

construct(cache: FileCache, fetch: bool = True, find_git: Callable[[], Executable | None] = lambda : ..., overrides: Dict[str, Any] | None = None) → Tuple[RepoPath, Dict[str, Exception]][source]

Construct a RepoPath from the descriptors.

If init is True, initialize all remote repositories that have not been fetched yet.

Returns:

A tuple containing a RepoPath instance with all constructed Repos and a dictionary mapping paths to exceptions that occurred during construction.

static from_config(lock: Lock, config: Configuration, scope=None) → RepoDescriptors[source]
exception spack.repo.RepoError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Superclass for repository-related errors.

class spack.repo.RepoIndex(packages_path: str, package_checker: Callable[[], FastPackageChecker], namespace: str, cache: FileCache)[source]

Bases: object

Container class that manages a set of Indexers for a Repo.

This class is responsible for checking packages in a repository for updates (using FastPackageChecker) and for regenerating indexes when they’re needed.

Indexers should be added to the RepoIndex using add_indexer(name, indexer), and they should support the interface defined by Indexer, so that the RepoIndex can read, generate, and update stored indices.

Generated indexes are accessed by name via __getitem__().

add_indexer(name: str, indexer: Indexer)[source]

Add an indexer to the repo index.

Parameters:

  • name – name of this indexer

  • indexer – object implementing the Indexer interface

property checker: FastPackageChecker
get_index(name, allow_stale: bool = False)[source]

Get the index with the specified name. The index will be updated if it is stale, unless allow_stale is True, in which case its contents are not validated against the package repository. When no cache is available, the index will be updated regardless of the value of allow_stale.

indexers: Dict[str, Indexer]
indexes: Dict[str, Any]
is_fresh

Whether the indexes are up to date with the package repository.

class spack.repo.RepoPath(*repos: Repo)[source]

Bases: object

A RepoPath is a list of Repo instances that function as one.

It functions exactly like a Repo, but it operates on the combined results of the Repos in its list instead of on a single package repository.

all_package_classes() → Generator[Type[PackageBase], None, None][source]
all_package_names(include_virtuals: bool = False) → List[str][source]
all_package_paths() → Generator[str, None, None][source]
dirname_for_package_name(pkg_name: str) → str[source]
disable() → None[source]

Disable the search paths for package module loading

dump_provenance(spec, path)[source]

Dump provenance information for a spec to a particular path.

This dumps the package file and any associated patch files. Raises UnknownPackageError if not found.

enable() → None[source]

Set the relevant search paths for package module loading

ensure_unwrapped() → RepoPath[source]

Ensure we unwrap this object from any dynamic wrapper (like Singleton)

exists(pkg_name: str) → bool[source]

Whether package with the give name exists in the path’s repos.

Note that virtual packages do not “exist”.

extensions_for(extendee_spec: Spec) → List[PackageBase][source]
filename_for_package_name(pkg_name: str) → str[source]
first_repo() → Repo | None[source]

Get the first repo in precedence order.

static from_config(config: Configuration) → RepoPath[source]

Create a RepoPath from a configuration object.

static from_descriptors(descriptors: RepoDescriptors, cache: FileCache, overrides: Dict[str, Any] | None = None) → RepoPath[source]
get(spec: Spec) → PackageBase[source]

Returns the package associated with the supplied spec.

get_patch_index(allow_stale: bool = False) → PatchCache[source]

Return the merged patch index for all repos in this path.

Parameters:

allow_stale – if True, return a possibly out-of-date index from cache files, avoiding filesystem calls to check whether the index is up to date.

get_patches_for_package(sha256s: List[str], pkg_cls: Type[PackageBase]) → List[Patch][source]

Look up patches by sha256, trying stale cache first to avoid stat calls.

Parameters:

  • sha256s – ordered list of patch sha256 hashes

  • pkg_cls – package class the patches belong to

Returns:

List of Patch objects in the same order as sha256s.

Raises:

spack.error.PatchLookupError – if a sha256 cannot be found even after a full rebuild.

get_pkg_class(pkg_name: str) → Type[PackageBase][source]

Find a class for the spec’s package and return the class object.

get_repo(namespace: str) → Repo[source]

Get a repository by namespace.

is_virtual(pkg_name: str) → bool[source]

Return True if the package with this name is virtual, False otherwise.

This function use the provider index. If calling from a code block that is used to construct the provider index use the is_virtual_safe function.

Parameters:

pkg_name – name of the package we want to check

is_virtual_safe(pkg_name: str) → bool[source]

Return True if the package with this name is virtual, False otherwise.

This function doesn’t use the provider index.

Parameters:

pkg_name – name of the package we want to check

last_mtime()[source]

Time a package file in this repo was last updated.

marshal()[source]
package_path(name: str) → str[source]

Get path to package.py file for this repo.

packages_with_tags(*tags: str, full: bool = False) → Set[str][source]

Returns a set of packages matching any of the tags in input.

Parameters:

full – if True the package names in the output are fully-qualified

property provider_index: ProviderIndex

Merged ProviderIndex from all Repos in the RepoPath.

providers_for(virtual: str | Spec) → List[Spec][source]
put_first(repo: Repo | RepoPath) → None[source]

Add repo first in the search path.

put_last(repo)[source]

Add repo last in the search path.

python_paths() → List[str][source]

Return a list of all the Python paths in the repos.

remove(repo)[source]

Remove a repo from the search path.

repo_for_pkg(spec: str | Spec) → Repo[source]

Given a spec, get the repository for its package.

repos: List[Repo]
property tag_index: TagIndex

Merged TagIndex from all Repos in the RepoPath.

static unmarshal(repos)[source]
class spack.repo.ReposFinder[source]

Bases: object

MetaPathFinder class that loads a Python module corresponding to an API v1 Spack package.

Returns a loader based on the inspection of the current repository list.

compute_loader(fullname: str)[source]
find_spec(fullname, python_path, target=None)[source]
repo_path: RepoPath

The current list of repositories.

class spack.repo.SpackNamespace(namespace)[source]

Bases: ModuleType

Allow lazy loading of modules.

class spack.repo.SpackNamespaceLoader[source]

Bases: object

create_module(spec)[source]
exec_module(module)[source]
class spack.repo.TagIndexer(repository)[source]

Bases: Indexer

Lifecycle methods for a TagIndex on a Repo.

read(stream)[source]

Read this index from a provided file object.

update(pkgs_fullname: Set[str])[source]

Update the index in memory with information about a package.

write(stream)[source]

Write the index to a file object.

exception spack.repo.UnknownEntityError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when we encounter a package spack doesn’t have.

exception spack.repo.UnknownNamespaceError(namespace, name=None)[source]

Bases: UnknownEntityError

Raised when we encounter an unknown namespace

exception spack.repo.UnknownPackageError(name, repo: Repo | RepoPath | str | None = None, *, get_close_matches=difflib.get_close_matches)[source]

Bases: UnknownEntityError

Raised when we encounter a package spack doesn’t have.

spack.repo.add_package_to_git_stage(packages: List[str], repo: Repo) → None[source]

add a package to the git stage with git add

spack.repo.all_package_names(include_virtuals=False)[source]

Convenience wrapper around spack.repo.all_package_names().

spack.repo.autospec(function)[source]

Decorator that automatically converts the first argument of a function to a Spec.

spack.repo.builtin_repo() → Repo[source]

Get the test repo if it is active, otherwise the builtin repo.

spack.repo.create_and_enable(config: Configuration) → RepoPath[source]

Immediately call enable() on the created RepoPath instance.

spack.repo.create_or_construct(root: str, namespace: str | None = None, package_api: Tuple[int, int] = spack.package_api_version) → Repo[source]

Create a repository, or just return a Repo if it already exists.

spack.repo.create_repo(root, namespace: str | None = None, subdir: str = packages_dir_name, package_api: Tuple[int, int] = spack.package_api_version) → Tuple[str, str][source]

Create a new repository in root with the specified namespace.

If the namespace is not provided, use basename of root. Return the canonicalized path and namespace of the created repository.

spack.repo.diff_packages(rev1: str, rev2: str, repo: Repo) → Tuple[Set[str], Set[str]][source]

Compute packages lists for the two revisions and return a tuple containing all the packages in rev1 but not in rev2 and all the packages in rev2 but not in rev1.

spack.repo.enable_repo(repo_path: RepoPath) → None[source]

Set the global package repository and make them available in module search paths.

spack.repo.from_path(path: str) → Repo[source]

Constructs a Repo using global misc cache.

spack.repo.get_all_package_diffs(type: str, repo: Repo, rev1='HEAD^1', rev2='HEAD') → Set[str][source]

Get packages changed, added, or removed (or any combination of those) since a commit.

Parameters:

  • type – String containing one or more of A, R, C.

  • rev1 – Revision to compare against, default is "HEAD^"

  • rev2 – Revision to compare to rev1, default is "HEAD"

spack.repo.get_repo_yaml_dir(root: str, namespace: str | None, package_api: Tuple[int, int]) → Tuple[str, str][source]

Returns the directory where repo.yaml is located and the effective namespace.

spack.repo.is_package_module(fullname: str) → bool[source]

Check if the given module is a package module.

spack.repo.list_packages(rev: str, repo: Repo) → List[str][source]

List all packages associated with the given revision

spack.repo.name_from_fullname(fullname: str) → str[source]

Return the package name for the full module name.

For instance:

name_from_fullname("spack.pkg.builtin.hdf5") == "hdf5"
name_from_fullname("spack_repo.x.y.z.packages.pkg_name.package") == "pkg_name"
Parameters:

fullname – full name for the Python module

spack.repo.namespace_from_fullname(fullname: str) → str[source]

Return the repository namespace only for the full module name.

For instance:

namespace_from_fullname("spack.pkg.builtin.hdf5") == "builtin"
namespace_from_fullname("spack_repo.x.y.z.packages.pkg_name.package") == "x.y.z"
Parameters:

fullname – full name for the Python module

spack.repo.package_repository_lock() → Lock[source]

Lock for process safety when cloning remote package repositories

spack.repo.parse_config_descriptor(name: str | None, descriptor: Any, lock: Lock) → RepoDescriptor[source]

Parse a repository descriptor from validated configuration. This does not instantiate Repo objects, but merely turns the config into a more useful RepoDescriptor instance.

Parameters:

  • name – the name of the repository, used for error messages

  • descriptor – the configuration for the repository, which can be a string (local path), or a dictionary with git key containing git URL and other options.

Returns:

A RepoDescriptor instance, either LocalRepoDescriptor or RemoteRepoDescriptor.

Raises:

  • BadRepoError – if the descriptor is invalid or cannot be parsed.

  • RuntimeError – if the descriptor is of an unexpected type.

spack.repo.partition_package_name(pkg_name: str) → Tuple[str, str][source]

Given a package name that might be fully-qualified, returns the namespace part, if present and the unqualified package name.

If the package name is unqualified, the namespace is an empty string.

Parameters:

pkg_name – a package name, either unqualified like llvm, or fully-qualified, like builtin.llvm

spack.repo.use_repositories(*paths_and_repos: str | Repo, override: bool = True) → Generator[RepoPath, None, None][source]

Use the repositories passed as arguments within the context manager.

Parameters:

  • *paths_and_repos – paths to the repositories to be used, or already constructed Repo objects

  • override – if True use only the repositories passed as input, if False add them to the top of the list of current repositories.

Returns:

Corresponding RepoPath object

spack.repo_migrate module

spack.repo_migrate.migrate_v1_to_v2(repo: Repo, *, patch_file: IO[bytes] | None, err: IO[str] = sys.stderr) → Tuple[bool, Repo | None][source]

To upgrade a repo from Package API v1 to v2 we need to: 1. ensure spack_repo/<namespace> parent dirs to the repo.yaml file. 2. rename <pkg dir>/package.py to <pkg module>/package.py. 3. bump the version in repo.yaml.

spack.repo_migrate.migrate_v2_imports(packages_dir: str, root: str, patch_file: IO[bytes] | None, err: IO[str] = sys.stderr) → bool[source]

In Package API v2.0, packages need to explicitly import package classes and a few other symbols from the build_systems module. This function automatically adds the missing imports to each package.py file in the repository.

spack.report module

Tools to produce reports of spec installations or tests

class spack.report.InstallRecord(spec)[source]

Bases: SpecRecord

Record class with specialization for install logs.

fetch_log(log_path: str | None = None) → str[source]

Install log comes from log_path if provided, install prefix, or stage dir.

succeed(log_path: str | None = None)[source]

Record success for this spec

class spack.report.NullInstallRecord(spec)[source]

Bases: InstallRecord

No-op drop-in for InstallRecord when no reporter is configured.

Avoids reading log files from disk on every completed build.

fail(exc, log_path: str | None = None) → None[source]

Record failure based on exception type

Errors wrapped by spack.error.InstallError are “failures” Other exceptions are “errors”.

skip(msg: str = '') → None[source]
start() → None[source]
succeed(log_path: str | None = None) → None[source]

Record success for this spec

class spack.report.NullRequestRecord[source]

Bases: RequestRecord

No-op drop-in for RequestRecord when no reporter is configured.

Avoids traversing the DAG and accumulating data that will not be reported.

append_record(record) → None[source]
skip_installed() → None[source]

Insert records for all nodes in the DAG that are no-ops for this request

summarize() → None[source]

Construct request-level summaries of the individual records

class spack.report.Property(name, value)

Bases: tuple

name

Alias for field number 0

value

Alias for field number 1

class spack.report.Record[source]

Bases: dict

Data class that provides attr-style access to a dictionary

Attributes beginning with _ are reserved for the Record class itself.

class spack.report.RequestRecord(spec)[source]

Bases: Record

Data class for recording outcomes for an entire DAG

Each BuildRequest in the installer and each root spec in a TestSuite generates a RequestRecord. The packages list of the RequestRecord is a list of SpecRecord objects recording individual data for each node in the Spec represented by the RequestRecord.

These data classes are collated by the reporters in lib/spack/spack/reporters

append_record(record)[source]
skip_installed()[source]

Insert records for all nodes in the DAG that are no-ops for this request

summarize()[source]

Construct request-level summaries of the individual records

class spack.report.SpecRecord(spec)[source]

Bases: Record

Individual record for a single spec within a request

fail(exc, log_path: str | None = None)[source]

Record failure based on exception type

Errors wrapped by spack.error.InstallError are “failures” Other exceptions are “errors”.

fetch_log(log_path: str | None = None) → str[source]

Fetch the log for this spec record. Subclasses should override.

skip(msg)[source]
start()[source]
succeed(log_path: str | None = None)[source]

Record success for this spec

class spack.report.TestRecord(spec, directory)[source]

Bases: SpecRecord

Record class with specialization for test logs.

fetch_log(log_path: str | None = None) → str[source]

Get output from test log

succeed(externals)[source]

Test reports skip externals by default.

spack.resource module

Describes an optional resource needed for a build.

Typically a bunch of sources that can be built in-tree within another package to enable optional features.

class spack.resource.Resource(name, fetcher, destination, placement)[source]

Bases: object

Represents any resource to be fetched by a package.

This includes the main tarball or source archive, as well as extra archives defined by the resource() directive.

Aggregates a name, a fetcher, a destination and a placement.

spack.rewiring module

exception spack.rewiring.PackageNotInstalledError(spliced_spec, build_spec, dep)[source]

Bases: RewireError

Raised when the build_spec for a splice was not installed.

exception spack.rewiring.RewireError(message, long_msg=None)[source]

Bases: SpackError

Raised when something goes wrong with rewiring.

spack.rewiring.rewire(spliced_spec)[source]

Given a spliced spec, this function conducts all the rewiring on all nodes in the DAG of that spec.

spack.rewiring.rewire_node(spec, explicit)[source]

This function rewires a single node, worrying only about references to its subgraph. Binaries, text, and links are all changed in accordance with the splice. The resulting package is then ‘installed.’

spack.sandbox module

This module implements an unprivileged sandbox for build environments.

It enforces path-based filesystem whitelisting and optional network isolation, dynamically adapting to the host kernel’s supported Landlock ABI version.

By design, to support standard build system behaviors like try_compile tests, read access implicitly includes execution rights. IOCTLs and IPC mechanisms are left unrestricted to ensure compatibility with compilers, terminal output, and build jobservers.

class spack.sandbox.FSAccess(*values)[source]

Bases: IntFlag

EXECUTE
MAKE_BLOCK
MAKE_CHAR
MAKE_DIR
MAKE_FIFO
MAKE_REG
MAKE_SOCK
MAKE_SYM
READ_DIR
READ_FILE
REFER
REMOVE_DIR
REMOVE_FILE
TRUNCATE
WRITE_FILE
class spack.sandbox.LandlockSandbox(libc=None)[source]

Bases: Sandbox

apply(block_network: bool = False)[source]
path_rules: Dict[Path, int]
class spack.sandbox.PathBeneathAttr[source]

Bases: Structure

allowed_access

Structure/Union member

parent_fd

Structure/Union member

class spack.sandbox.RulesetAttr[source]

Bases: Structure

handled_access_fs

Structure/Union member

handled_access_net

Structure/Union member

scoped

Structure/Union member

class spack.sandbox.Sandbox[source]

Bases: ABC

Abstract base class for sandbox implementations.

allow_read(path: str | Path)[source]
allow_write(path: str | Path)[source]
abstractmethod apply(block_network: bool = False)[source]
exception spack.sandbox.SandboxError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when the build sandbox cannot be set up or applied.

spack.sandbox.get_sandbox() → Sandbox[source]

spack.spec module

Spack allows very fine-grained control over how packages are installed and over how they are built and configured. To make this easy, it has its own syntax for declaring a dependence. We call a descriptor of a particular package configuration a “spec”.

The syntax looks like this:

$ spack install mpileaks ^openmpi @1.2:1.4 +debug %intel @12.1 target=zen
                0        1        2        3      4      5     6

The first part of this is the command, spack install. The rest of the line is a spec for a particular installation of the mpileaks package.

  1. The package to install

  2. A dependency of the package, prefixed by ^

  3. A version descriptor for the package. This can either be a specific version, like 1.2, or it can be a range of versions, e.g. 1.2:1.4. If multiple specific versions or multiple ranges are acceptable, they can be separated by commas, e.g. if a package will only build with versions 1.0, 1.2-1.4, and 1.6-1.8 of mvapich, you could say:

    depends_on("mvapich@1.0,1.2:1.4,1.6:1.8")

  4. A compile-time variant of the package. If you need openmpi to be built in debug mode for your package to work, you can require it by adding +debug to the openmpi spec when you depend on it. If you do NOT want the debug option to be enabled, then replace this with -debug. If you would like for the variant to be propagated through all your package’s dependencies use ++ for enabling and -- or ~~ for disabling.

  5. The name of the compiler to build with.

  6. The versions of the compiler to build with. Note that the identifier for a compiler version is the same @ that is used for a package version. A version list denoted by @ is associated with the compiler only if if it comes immediately after the compiler name. Otherwise it will be associated with the current package spec.

  7. The architecture to build with.

exception spack.spec.AmbiguousHashError(msg, *specs)[source]

Bases: SpecError

class spack.spec.CompilerSpec(spec)[source]

Bases: object

Adaptor to the old compiler spec interface. Exposes just a few attributes

property display_str

Equivalent to {compiler.name}{@compiler.version} for Specs, without extra @= for readability.

property name
property version
property versions
exception spack.spec.DuplicateArchitectureError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when the same architecture occurs in a spec twice.

exception spack.spec.DuplicateDependencyError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when the same dependency occurs in a spec twice.

exception spack.spec.InvalidDependencyError(pkg, deps)[source]

Bases: SpecError

Raised when a dependency in a spec is not actually a dependency of the package.

exception spack.spec.InvalidHashError(spec, hash)[source]

Bases: SpecError

class spack.spec.Spec(spec_like=None, *, external_path=None, external_modules=None)[source]

Bases: object

add_dependency_edge(dependency_spec: Spec, *, depflag: int, virtuals: Tuple[str, ...], direct: bool = False, propagation: PropagationPolicy = PropagationPolicy.NONE, when: Spec | None = None)[source]

Add a dependency edge to this spec.

Parameters:

  • dependency_spec – spec of the dependency

  • depflag – dependency type for this edge

  • virtuals – virtuals provided by this edge

  • direct – if True denotes a direct dependency

  • propagation – propagation policy for this edge

  • when – if non-None, condition under which dependency holds

property anonymous
attach_git_version_lookup()[source]
property build_spec
cformat(format_string: str = DEFAULT_FORMAT) → str[source]

Same as format(), but color defaults to auto instead of False.

clear_caches(ignore: Tuple[str, ...] = ()) → None[source]

Clears all cached hashes in a Spec, while preserving other properties.

clear_dependencies()[source]

Trim the dependencies of this spec.

clear_edges()[source]

Trim the dependencies and dependents of this spec.

property clong_spec

Returns an auto-colorized version of long_spec.

property colored_str: str

Auto-colorized string representation of this spec.

compiler
property compilers
property concrete

A spec is concrete if it describes a single build of a package.

More formally, a spec is concrete if concretize() has been called on it and it has been marked _concrete.

Concrete specs either can be or have been built. All constraints have been resolved, optional dependencies have been added or removed, a compiler has been chosen, and all variants have values.

constrain(other, deps=True) → bool[source]

Constrains self with other, and returns True if self changed, False otherwise.

Parameters:

  • other – constraint to be added to self

  • deps – if False, constrain only the root node, otherwise constrain dependencies as well

Raises:

spack.error.UnsatisfiableSpecError – when self cannot be constrained

constrained(other, deps=True)[source]

Return a constrained copy without modifying this spec.

copy(deps: bool | str | List[str] | Tuple[str, ...] | int = True, **kwargs)[source]

Make a copy of this spec.

Parameters:

  • deps – Defaults to True. If boolean, controls whether dependencies are copied (copied if True). If a DepTypes or DepFlag is provided, only matching dependencies are copied.

  • kwargs – additional arguments for internal use (passed to _dup).

Returns:

A copy of this spec.

Examples

Deep copy with dependencies:

spec.copy()
spec.copy(deps=True)

Shallow copy (no dependencies):

spec.copy(deps=False)

Only build and run dependencies:

deps=("build", "run"):
property cshort_spec

Returns an auto-colorized version of short_spec.

dag_hash(length=None)[source]

This is Spack’s default hash, used to identify installations.

NOTE: Versions of Spack prior to 0.18 only included link and run deps. NOTE: Versions of Spack prior to 1.0 only did not include test deps.

dag_hash_bit_prefix(bits)[source]

Get the first <bits> bits of the DAG hash as an integer type.

static default_arch()[source]

Return an anonymous spec for the default architecture

dependencies(name: str | None = None, deptype: str | List[str] | Tuple[str, ...] | int = dt.ALL, *, virtuals: str | Sequence[str] | None = None) → List[Spec][source]

Returns a list of direct dependencies (nodes in the DAG)

Parameters:

  • name – filter dependencies by package name

  • deptype – allowed dependency types

  • virtuals – allowed virtuals

dependents(name: str | None = None, deptype: str | List[str] | Tuple[str, ...] | int = dt.ALL) → List[Spec][source]

Return a list of direct dependents (nodes in the DAG).

Parameters:

  • name – filter dependents by package name

  • deptype – allowed dependency types

detach(deptype='all')[source]

Remove any reference that dependencies have of this node.

Parameters:

deptype (str or tuple) – dependency types tracked by the current spec

property edge_attributes: str

Helper method to print edge attributes in spec strings.

edges_from_dependents(name: str | None = None, depflag: int = dt.ALL, *, virtuals: str | Sequence[str] | None = None) → List[DependencySpec][source]

Return a list of edges connecting this node in the DAG to parents.

Parameters:

  • name – filter dependents by package name

  • depflag – allowed dependency types

  • virtuals – allowed virtuals

edges_to_dependencies(name: str | None = None, depflag: int = dt.ALL, *, virtuals: str | Sequence[str] | None = None) → List[DependencySpec][source]

Returns a list of edges connecting this node in the DAG to children.

Parameters:

  • name – filter dependencies by package name

  • depflag – allowed dependency types

  • virtuals – allowed virtuals

static ensure_no_deprecated(root: Spec) → None[source]

Raise if a deprecated spec is in the dag of the given root spec.

Raises:

spack.spec.SpecDeprecatedError – if any deprecated spec is found

static ensure_valid_variants(spec: Spec) → None[source]

Ensures that the variant attached to the given spec are valid.

Raises:

spack.variant.UnknownVariantError – on the first unknown variant found

eq_dag(other, deptypes=True, vs=None, vo=None)[source]

True if the full dependency DAGs of specs are equal.

eq_node(other)[source]

Equality with another spec, not including dependencies.

property external
property external_path
static extract_json_from_clearsig(data)[source]
format(format_string: str = DEFAULT_FORMAT, color: bool | None = False, *, highlight_version_fn: Callable[[Spec], bool] | None = None, highlight_variant_fn: Callable[[Spec, str], bool] | None = None) → str[source]

Prints out attributes of a spec according to a format string.

Using an {attribute} format specifier, any field of the spec can be selected. Those attributes can be recursive. For example, s.format({compiler.version}) will print the version of the compiler.

If the attribute in a format specifier evaluates to None, then the format specifier will evaluate to the empty string, "".

Commonly used attributes of the Spec for format strings include:

name
version
compiler_flags
compilers
variants
architecture
architecture.platform
architecture.os
architecture.target
prefix
namespace

Some additional special-case properties can be added:

hash[:len]    The DAG hash with optional length argument
spack_root    The spack root directory
spack_install The spack install directory

The ^ sigil can be used to access dependencies by name. s.format({^mpi.name}) will print the name of the MPI implementation in the spec.

The @, %, and / sigils can be used to include the sigil with the printed string. These sigils may only be used with the appropriate attributes, listed below:

  • @: {@version}, {@compiler.version}

  • %: {%compiler}, {%compiler.name}

  • /: {/hash}, {/hash:7}, etc

The @ sigil may also be used for any other property named version. Sigils printed with the attribute string are only printed if the attribute string is non-empty, and are colored according to the color of the attribute.

Variants listed by name naturally print with their sigil. For example, spec.format("{variants.debug}") prints either +debug or ~debug depending on the name of the variant. Non-boolean variants print as name=value. To print variant names or values independently, use spec.format("{variants.<name>.name}") or spec.format("{variants.<name>.value}").

There are a few attributes on specs that can be specified as key-value pairs that are not variants, e.g.: os, arch, architecture, target, namespace, etc. You can format these with an optional key= prefix, e.g. {namespace=namespace} or {arch=architecture}, etc. The key= prefix will be colorized along with the value.

When formatting specs, key-value pairs are separated from preceding parts of the spec by whitespace. To avoid printing extra whitespace when the formatted attribute is not set, you can add whitespace to the key inside the braces of the format string, e.g.:

{ namespace=namespace}

This evaluates to " namespace=builtin" if namespace is set to builtin, and to "" if namespace is None.

Spec format strings use \ as the escape character. Use \{ and \} for literal braces, and \\ for the literal \ character.

Parameters:

  • format_string – string containing the format to be expanded

  • color – True for colorized result; False for no color; None for auto color.

  • highlight_version_fn – optional callable that returns true on nodes where the version needs to be highlighted

  • highlight_variant_fn – optional callable that returns true on variants that need to be highlighted

format_path(format_string: str, _path_ctor: Callable[[Any], PurePath] | None = None) → str[source]

Given a format_string that is intended as a path, generate a string like from format(), but eliminate extra path separators introduced by formatting of Spec properties.

Path separators explicitly added to the string are preserved, so for example {name}/{version} would generate a directory based on the Spec’s name, and a subdirectory based on its version; this function guarantees though that the resulting string would only have two directories (i.e. that if under normal circumstances that str(self.version) would contain a path separator, it would not in this case).

static from_detection(spec_str: str, *, external_path: str, external_modules: List[str] | None = None, extra_attributes: Dict | None = None) → Spec[source]

Construct a spec from a spec string determined during external detection and attach extra attributes to it.

Parameters:

  • spec_str – spec string

  • external_path – prefix of the external spec

  • external_modules – optional module files to be loaded when the external spec is used

  • extra_attributes – dictionary containing extra attributes

static from_dict(data) → Spec[source]

Construct a spec from JSON/YAML.

Parameters:

data – a nested dict/list data structure read from YAML or JSON.

static from_json(stream) → Spec[source]

Construct a spec from JSON.

Parameters:

stream – string or file object to read from.

static from_literal(spec_dict: dict, normal: bool = True) → Spec[source]

Builds a Spec from a dictionary containing the spec literal.

The dictionary must have a single top level key, representing the root, and as many secondary level keys as needed in the spec.

The keys can be either a string or a Spec or a tuple containing the Spec and the dependency types.

Parameters:

  • spec_dict – the dictionary containing the spec literal

  • normal – if True the same key appearing at different levels of the spec_dict will map to the same object in memory.

Examples

A simple spec foo with no dependencies:

{"foo": None}

A spec foo with a (build, link) dependency bar:

{"foo":
    {"bar:build,link": None}
}

A spec with a diamond dependency and various build types:

{"dt-diamond": {
    "dt-diamond-left:build,link": {
        "dt-diamond-bottom:build": None
    },
    "dt-diamond-right:build,link": {
        "dt-diamond-bottom:build,link,run": None
    }
}}

The same spec with a double copy of dt-diamond-bottom and no diamond structure:

Spec.from_literal({"dt-diamond": {
    "dt-diamond-left:build,link": {
        "dt-diamond-bottom:build": None
    },
    "dt-diamond-right:build,link": {
        "dt-diamond-bottom:build,link,run": None
    }
}, normal=False}

Constructing a spec using a Spec object as key:

mpich = Spec("mpich")
libelf = Spec("libelf@1.8.11")
expected_normalized = Spec.from_literal({
    "mpileaks": {
        "callpath": {
            "dyninst": {
                "libdwarf": {libelf: None},
                libelf: None
            },
            mpich: None
        },
        mpich: None
    },
})
static from_signed_json(stream)[source]

Construct a spec from clearsigned json spec file.

Parameters:

stream – string or file object to read from.

static from_specfile(path)[source]

Construct a spec from a JSON or YAML spec file path

static from_yaml(stream) → Spec[source]

Construct a spec from YAML.

Parameters:

stream – string or file object to read from.

property fullname
has_virtual_dependency(virtual: str) → bool[source]
index(deptype='all')[source]

Return a dictionary that points to all the dependencies in this spec.

install_status() → InstallStatus[source]

Helper for tree to print DB install status.

property installed

Installation status of a package.

Returns:

True if the package has been installed, False otherwise.

property installed_upstream

Whether the spec is installed in an upstream repository.

Returns:

True if the package is installed in an upstream, False otherwise.

intersects(other: str | Spec, deps: bool = True) → bool[source]

Return True if there exists at least one concrete spec that matches both self and other, otherwise False.

This operation is commutative, and if two specs intersect it means that one can constrain the other.

Parameters:

  • other – spec to be checked for compatibility

  • deps – if True check compatibility of dependency nodes too, if False only check root

property is_develop

Return whether the Spec represents a user-developed package in a Spack Environment (i.e. using spack develop).

property long_spec

Long string of the spec, including dependencies.

lookup_hash()[source]

Given a spec with an abstract hash, return a copy of the spec with all properties and dependencies by looking up the hash in the environment, store, or finally, binary caches. This is non-destructive.

mutate(mutator, rehash=True) → bool[source]

Mutate concrete spec to match constraints represented by mutator.

Mutation can modify the spec version, variants, compiler flags, and architecture. Mutation cannot change the spec name, namespace, dependencies, or abstract_hash. Any attribute which is unset will not be touched. Variant values can be replaced with the literal None to remove the variant. None as a variant value is represented by VariantValue(..., (None,)).

If rehash, concrete spec and its dependents have hashes updated.

Returns whether the spec was modified by the mutation

property namespace_if_anonymous
node_dict_with_hashes(hash: SpecHashDescriptor = ht.dag_hash) → Dict[str, Any][source]

Returns a node dict of this spec with the dag hash, and the provided hash (if not the dag hash).

original_spec_format() → int[source]

Returns the spec format originally used for this spec.

property os
static override(init_spec, change_spec)[source]
property package
package_hash()[source]

Compute the hash of the contents of the package for this node

property patches

Return patch objects for any patch sha256 sums on this Spec.

This is for use after concretization to iterate over any patches associated with this spec.

TODO: this only checks in the package; it doesn’t resurrect old patches from install directories, but it probably should.

property platform
property prefix: Prefix
replace_hash()[source]

Given a spec with an abstract hash, attempt to populate all properties and dependencies by looking up the hash in the environment, store, or finally, binary caches. This is destructive.

property root

Follow dependent links and find the root of this spec’s DAG.

Spack specs have a single root (the package being installed).

satisfies(other: str | Spec, deps: bool = True) → bool[source]

Return True if all concrete specs matching self also match other, otherwise False.

Parameters:

  • other – spec to be satisfied

  • deps – if True, descend to dependencies, otherwise only check root node

set_prefix(value: str) → None[source]
property short_spec

Short string of the spec, with hash and without dependencies.

property spack_install

Special field for using {spack_install} in format().

property spack_root

Special field for using {spack_root} in format().

spec_hash(hash: SpecHashDescriptor) → str[source]

Utility method for computing different types of Spec hashes.

Parameters:

hash – type of hash to generate.

splice(other: Spec, transitive: bool = True) → Spec[source]

Returns a new, spliced concrete Spec with the other dependency and, optionally, its dependencies.

Parameters:

  • other – alternate dependency

  • transitive – include other’s dependencies

Returns: a concrete, spliced version of the current Spec

When transitive is True, use the dependencies from other to reconcile conflicting dependencies. When transitive is False, use dependencies from self.

For example, suppose we have the following dependency graph:

T
| \
Z<-H

Spec T depends on H and Z, and H also depends on Z. Now we want to use a different H, called H'. This function can be used to splice in H' to create a new spec, called T*. If H' was built with Z', then transitive=True will ensure H' and T* both depend on Z':

T*
| \
Z'<-H'

If transitive=False, then H' and T* will both depend on the original Z, resulting in a new H'*:

T*
| \
Z<-H'*

Provenance of the build is tracked through the build_spec property of the spliced spec and any correspondingly modified dependency specs. The build specs are set to that of the original spec, so the original spec’s provenance is preserved unchanged.

property spliced

Returns whether or not this Spec is being deployed as built i.e. whether or not this Spec has ever been spliced.

property target
to_dict(hash: SpecHashDescriptor = ht.dag_hash) → Dict[str, Any][source]

Create a dictionary suitable for writing this spec to YAML or JSON.

This dictionary is like the one that is ultimately written to a spec.json file in each Spack installation directory. For example, for sqlite:

{
    "spec": {
        "_meta": {"version": 5},
        "nodes": [
            {
                "name": "sqlite",
                "version": "3.46.0",
                "arch": {
                    "platform": "linux",
                    "platform_os": "ubuntu24.04",
                    "target": "x86_64_v3"
                },
                "namespace": "builtin",
                "parameters": {
                    "build_system": "autotools",
                    "column_metadata": True,
                    "dynamic_extensions": True,
                    "fts": True,
                    "functions": False,
                    "rtree": True,
                    "cflags": [],
                    "cppflags": [],
                    "cxxflags": [],
                    "fflags": [],
                    "ldflags": [],
                    "ldlibs": [],
                },
                "package_hash": "umcghjlve5347o...xdzmfdba23nkdhe5jntlhia====",
                "dependencies": [
                    {
                        "name": "compiler-wrapper",
                        "hash": "c5bxlim3zge4snwrwtd6rzuvq2unek6s",
                        "parameters": {"deptypes": ("build",), "virtuals": ()},
                    },
                    {
                        "name": "gcc",
                        "hash": "6dzveld2rtt2dkhklxfnery5wbtb5uus",
                        "parameters": {"deptypes": ("build",), "virtuals": ("c",)},
                    },
                    ...
                ],
                "annotations": {"original_specfile_version": 5},
                "hash": "a2ubvvqnula6zdppckwqrjf3zmsdzpoh",
            },
            ...
        ],
    }
}

Note that this dictionary starts with the spec key, and what follows is a list starting with the root spec, followed by its dependencies in preorder.

The method from_dict() can be used to read back in a spec that has been converted to a dictionary, serialized, and read back in.

to_json(stream=None, *, hash=ht.dag_hash, pretty=False)[source]
to_node_dict(hash: SpecHashDescriptor = ht.dag_hash) → Dict[str, Any][source]

Create a dictionary representing the state of this Spec.

This method creates the content that is eventually hashed by Spack to create identifiers like the DAG hash (see dag_hash()). Example result of this function for the sqlite package:

{
    "name": "sqlite",
    "version": "3.46.0",
    "arch": {"platform": "linux", "platform_os": "ubuntu24.04", "target": "x86_64_v3"},
    "namespace": "builtin",
    "parameters": {
        "build_system": "autotools",
        "column_metadata": True,
        "dynamic_extensions": True,
        "fts": True,
        "functions": False,
        "rtree": True,
        "cflags": [],
        "cppflags": [],
        "cxxflags": [],
        "fflags": [],
        "ldflags": [],
        "ldlibs": [],
    },
    "package_hash": "umcghjlve5347o3q2odo7vfcso2zhxdzmfdba23nkdhe5jntlhia====",
    "dependencies": [
        {
            "name": "compiler-wrapper",
            "hash": "c5bxlim3zge4snwrwtd6rzuvq2unek6s",
            "parameters": {"deptypes": ("build",), "virtuals": ()},
        },
        {
            "name": "gcc",
            "hash": "6dzveld2rtt2dkhklxfnery5wbtb5uus",
            "parameters": {"deptypes": ("build",), "virtuals": ("c",)},
        },
        ...
    ],
    "annotations": {"original_specfile_version": 5},
}

Note that the dictionary returned does not include the hash of the root of the spec, though it does include hashes for each dependency and its own package hash.

See to_dict() for a “complete” spec hash, with hashes for each node and nodes for each dependency (instead of just their hashes).

Parameters:

hash – type of hash to generate.

to_yaml(stream=None, hash=ht.dag_hash)[source]
traverse(*, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[False] = False, key: Callable[[Spec], Any] = id, visited: Set[Any] | None = None) → Iterable[Spec][source]
traverse(*, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[True], key: Callable[[Spec], Any] = id, visited: Set[Any] | None = None) → Iterable[Tuple[int, Spec]]

Shorthand for traverse_nodes()

traverse_edges(*, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[False] = False, key: Callable[[Spec], Any] = id, visited: Set[Any] | None = None) → Iterable[DependencySpec][source]
traverse_edges(*, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[True], key: Callable[[Spec], Any] = id, visited: Set[Any] | None = None) → Iterable[Tuple[int, DependencySpec]]

Shorthand for traverse_edges()

tree(*, color: bool | None = None, depth: bool = False, hashes: bool = False, hashlen: int | None = None, cover: Literal['nodes', 'edges', 'paths'] = 'nodes', indent: int = 0, format: str = DEFAULT_FORMAT, deptypes: str | List[str] | Tuple[str, ...] | int = dt.ALL, show_types: bool = False, depth_first: bool = False, recurse_dependencies: bool = True, status_fn: Callable[[Spec], InstallStatus] | None = None, prefix: Callable[[Spec], str] | None = None, key=id, highlight_version_fn: Callable[[Spec], bool] | None = None, highlight_variant_fn: Callable[[Spec, str], bool] | None = None) → str[source]

Prints out this spec and its dependencies, tree-formatted with indentation.

See multi-spec spack.spec.tree() function for details.

Parameters:

  • specs – List of specs to format.

  • color – if True, always colorize the tree. If False, don’t colorize the tree. If None, use the default from spack.llnl.tty.color

  • depth – print the depth from the root

  • hashes – if True, print the hash of each node

  • hashlen – length of the hash to be printed

  • cover – either "nodes" or "edges"

  • indent – extra indentation for the tree being printed

  • format – format to be used to print each node

  • deptypes – dependency types to be represented in the tree

  • show_types – if True, show the (merged) dependency type of a node

  • depth_first – if True, traverse the DAG depth first when representing it as a tree

  • recurse_dependencies – if True, recurse on dependencies

  • status_fn – optional callable that takes a node as an argument and return its installation status

  • prefix – optional callable that takes a node as an argument and return its installation prefix

  • highlight_version_fn – optional callable that returns true on nodes where the version needs to be highlighted

  • highlight_variant_fn – optional callable that returns true on variants that need to be highlighted

trim(dep_name)[source]

Remove any package that is or provides dep_name transitively from this tree. This can also remove other dependencies if they are only present because of dep_name.

validate_or_raise()[source]

Checks that names and values in this spec are real. If they’re not, it will raise an appropriate exception.

property version
exception spack.spec.SpecDeprecatedError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when a spec concretizes to a deprecated spec or dependency.

exception spack.spec.UnsatisfiableArchitectureSpecError(provided, required)[source]

Bases: UnsatisfiableSpecError

Raised when a spec architecture conflicts with package constraints.

exception spack.spec.UnsatisfiableDependencySpecError(provided, required)[source]

Bases: UnsatisfiableSpecError

Raised when some dependency of constrained specs are incompatible

exception spack.spec.UnsatisfiableSpecNameError(provided, required)[source]

Bases: UnsatisfiableSpecError

Raised when two specs aren’t even for the same package.

exception spack.spec.UnsatisfiableVersionSpecError(provided, required)[source]

Bases: UnsatisfiableSpecError

Raised when a spec version conflicts with package constraints.

exception spack.spec.UnsupportedCompilerError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when the user asks for a compiler spack doesn’t know about.

exception spack.spec.UnsupportedPropagationError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when propagation (==) is used with reserved variant names.

spack.spec_filter module

class spack.spec_filter.SpecFilter(factory: Callable[[], List[Spec]], is_usable: Callable[[Spec], bool], include: List[str], exclude: List[str])[source]

Bases: object

Given a method to produce a list of specs, this class can filter them according to different criteria.

is_selected(s: Spec) → bool[source]
selected_specs() → List[Spec][source]

spack.spec_parser module

Parser for spec literals

Here is the EBNF grammar for a spec:

spec          = [name] [node_options] { ^[edge_properties] node } |
                [name] [node_options] hash |
                filename

node          =  name [node_options] |
                 [name] [node_options] hash |
                 filename

node_options    = [@(version_list|version_pair)] [%compiler] { variant }
edge_properties = [ { bool_variant | key_value } ]

hash          = / id
filename      = (.|/|[a-zA-Z0-9-_]*/)([a-zA-Z0-9-_./]*)(.json|.yaml)

name          = id | namespace id
namespace     = { id . }

variant       = bool_variant | key_value | propagated_bv | propagated_kv
bool_variant  =  +id |  ~id |  -id
propagated_bv = ++id | ~~id | --id
key_value     =  id=id |  id=quoted_id
propagated_kv = id==id | id==quoted_id

compiler      = id [@version_list]

version_pair  = git_version=vid
version_list  = (version|version_range) [ { , (version|version_range)} ]
version_range = vid:vid | vid: | :vid | :
version       = vid

git_version   = git.(vid) | git_hash
git_hash      = [A-Fa-f0-9]{40}

quoted_id     = " id_with_ws " | ' id_with_ws '
id_with_ws    = [a-zA-Z0-9_][a-zA-Z_0-9-.\s]*
vid           = [a-zA-Z0-9_][a-zA-Z_0-9-.]*
id            = [a-zA-Z0-9_][a-zA-Z_0-9-]*

Identifiers using the <name>=<value> command, such as architectures and compiler flags, require a space before the name.

There is one context-sensitive part: ids in versions may contain ., while other ids may not.

There is one ambiguity: since - is allowed in an id, you need to put whitespace space before -variant for it to be tokenized properly. You can either use whitespace, or you can just use ~variant since it means the same thing. Spack uses ~variant in directory names and in the canonical form of specs to avoid ambiguity. Both are provided because ~ can cause shell expansion when it is the first character in an id typed on the command line.

class spack.spec_parser.EdgeAttributeParser(ctx, literal_str)[source]

Bases: object

ctx
literal_str
parse()[source]
class spack.spec_parser.FileParser(ctx)[source]

Bases: object

Parse a single spec from a JSON or YAML file

ctx
parse(initial_spec: Spec) → Spec[source]

Parse a spec tree from a specfile.

Parameters:

initial_spec – object where to parse the spec

Returns:

The initial_spec passed as argument, once constructed

spack.spec_parser.GIT_REF

Git refs include branch names, and can contain . and /

spack.spec_parser.IDENTIFIER

Valid name for specs and variants. Here we are not using the previous w[\w.-]* since that would match most characters that can be part of a word in any language

spack.spec_parser.NO_QUOTES_NEEDED

Values that match this (e.g., variants, flags) can be left unquoted in Spack output

spack.spec_parser.QUOTED_VALUE

Quoted values can be anything in between quotes, including escaped quotes.

spack.spec_parser.SPEC_TOKENIZER

Tokenizer that includes all the regexes in the SpecTokens enum

spack.spec_parser.STRIP_QUOTES

Regex to strip quotes. Group 2 will be the unquoted string.

class spack.spec_parser.SpecNodeParser(ctx, literal_str)[source]

Bases: object

Parse a single spec node from a stream of tokens

ctx
has_version
literal_str
parse(initial_spec: Spec | None = None, root: bool = True) → Spec[source]

Parse a single spec node from a stream of tokens

Parameters:

  • initial_spec – object to be constructed

  • root – True if we’re parsing a root, False if dependency after ^ or %

Returns:

The object passed as argument

class spack.spec_parser.SpecParser(literal_str: str)[source]

Bases: object

Parse text into specs

all_specs() → List[Spec][source]

Return all the specs that remain to be parsed

ctx
literal_str
next_spec(initial_spec: Spec | None = None) → Spec | None[source]

Return the next spec parsed from text.

Parameters:

initial_spec – object where to parse the spec. If None a new one will be created.

Returns:

The spec that was parsed

tokens() → List[Token][source]

Return the entire list of token from the initial text. White spaces are filtered out.

exception spack.spec_parser.SpecParsingError(message, token, text)[source]

Bases: SpecSyntaxError

Error when parsing tokens

exception spack.spec_parser.SpecTokenizationError(tokens: List[Token], text: str)[source]

Bases: SpecSyntaxError

Syntax error in a spec string

class spack.spec_parser.SpecTokens(*values)[source]

Bases: TokenBase

Enumeration of the different token kinds of tokens in the spec grammar.

Order of declaration is extremely important, since text containing specs is parsed with a single regex obtained by "|".join(...) of all the regex in the order of declaration.

BOOL_VARIANT
DAG_HASH
DEPENDENCY
END_EDGE_PROPERTIES
FILENAME
FULLY_QUALIFIED_PACKAGE_NAME
GIT_VERSION
KEY_VALUE_PAIR
PROPAGATED_BOOL_VARIANT
PROPAGATED_KEY_VALUE_PAIR
START_EDGE_PROPERTIES
UNEXPECTED
UNQUALIFIED_PACKAGE_NAME
VERSION
VERSION_HASH_PAIR
WS
class spack.spec_parser.TokenContext(token_stream: Iterator[Token])[source]

Bases: object

Token context passed around by parsers

accept(kind: SpecTokens)[source]

If the next token is of the specified kind, advance the stream and return True. Otherwise return False.

advance()[source]

Advance one token

current_token
expect(*kinds: SpecTokens)[source]
next_token
push_front(token=Token)[source]

Push a token onto the front of the stream. Enables a bit of lookahead.

pushed_tokens: List[Token]
token_stream
spack.spec_parser.VALUE

These are legal values that can be parsed bare, without quotes on the command line.

spack.spec_parser.VIRTUAL_ASSIGNMENT

Substitute a package for a virtual, e.g., c,cxx=gcc. NOTE: Overlaps w/KVP; this should be first if matched in sequence.

spack.spec_parser.WINDOWS_FILENAME

A filename starts either with a . or a / or a {name}/, or on Windows, a drive letter followed by a colon and \ or . or {name}\

spack.spec_parser.expand_toolchains(spec: Spec, toolchains: Dict, *, _cache: Dict[str, Spec] | None = None) → None[source]

Replace toolchain placeholder deps with expanded toolchain constraints.

Walks every node in the spec DAG. For each node, finds direct dependency edges whose child name is a key in toolchains. Removes the placeholder edge, parses the toolchain config, copies with the edge’s propagation policy, and constrains the node.

spack.spec_parser.parse(text: str, *, toolchains: Dict | None = None) → List[Spec][source]

Parse text into a list of specs

Parameters:

  • text – text to be parsed

  • toolchains – optional toolchain definitions to expand after parsing

Returns:

List of specs

spack.spec_parser.parse_one_or_raise(text: str, initial_spec: Spec | None = None, *, toolchains: Dict | None = None) → Spec[source]

Parse exactly one spec from text and return it, or raise

Parameters:

  • text – text to be parsed

  • initial_spec – buffer where to parse the spec. If None a new one will be created.

  • toolchains – optional toolchain definitions to expand after parsing

spack.spec_parser.parse_virtual_assignment(context: TokenContext) → Tuple[str][source]

Look at subvalues and, if present, extract virtual and a push a substitute token.

This handles things like:

  • ^c=gcc

  • ^c,cxx=gcc

  • %[when=+bar] c=gcc

  • %[when=+bar] c,cxx=gcc

Virtual assignment can happen anywhere a dependency node can appear. It is shorthand for %[virtuals=c,cxx] gcc.

The virtuals=substitute key value pair appears in the subvalues of DEPENDENCY and END_EDGE_PROPERTIES tokens. We extract the virtuals and create a token from the substitute, which is then pushed back on the parser stream so that the head of the stream can be parsed like a regular node.

Returns:

the virtuals assigned, or None if there aren’t any

spack.spec_parser.parseable_tokens(text: str) → Iterator[Token][source]

Return non-whitespace tokens from the text passed as input

Raises:

SpecTokenizationError – when unexpected characters are found in the text

spack.spec_parser.quote_if_needed(value: str) → str[source]

Add quotes around the value if it requires quotes.

This will add quotes around the value unless it matches NO_QUOTES_NEEDED.

This adds:

  • single quotes by default

  • double quotes around any value that contains single quotes

If double quotes are used, we json-escape the string. That is, we escape \, ", and control codes.

spack.spec_parser.strip_quotes_and_unescape(string: str) → str[source]

Remove surrounding single or double quotes from string, if present.

spack.spec_parser.tokenize(text: str) → Iterator[Token][source]

Return a token generator from the text passed as input.

Raises:

SpecTokenizationError – when unexpected characters are found in the text

spack.stage module

class spack.stage.AbstractStage(name, path, keep, lock)[source]

Bases: ABC

Abstract base class for all stage types.

A stage is a directory whose lifetime can be managed with a context manager (but persists if the user requests it). Instances can have a specified name and if they do, then for all instances that have the same name, only one can enter the context manager at a time.

This class defines the interface that all stage types must implement.

abstract property archive_file: str | None

Return the path to the archive file, or None.

abstractmethod cache_local()[source]

Cache the resources locally.

cache_mirror(mirror: spack.caches.MirrorCache, stats: spack.mirrors.utils.MirrorStatsForOneSpec) → None[source]

Cache the resources to a mirror (can be no-op).

abstractmethod check()[source]

Check the integrity of the fetched resources.

create()[source]

Ensures the top-level (config:build_stage) directory exists.

abstractmethod destroy()[source]

Remove the stage directory and its contents.

abstractmethod expand_archive()[source]

Expand any downloaded archives.

abstract property expanded: bool

Return True if the source has been expanded.

abstractmethod fetch(mirror_only: bool = False, err_msg: str | None = None) → None[source]

Fetch the source code or resources for this stage.

requires_patch_success

Set to True to error out if patches fail

abstractmethod restage()[source]

Remove the expanded source and re-expand it.

abstract property source_path: str

Return the path to the expanded source code.

steal_source(dest: str) → None[source]

Copy source to another location (can be no-op).

class spack.stage.DevelopStage(name, dev_path, reference_link)[source]

Bases: AbstractStage

property archive_file

Return the path to the archive file, or None.

cache_local()[source]

Cache the resources locally.

check()[source]

Check the integrity of the fetched resources.

create()[source]

Ensures the top-level (config:build_stage) directory exists.

destroy()[source]

Remove the stage directory and its contents.

expand_archive()[source]

Expand any downloaded archives.

property expanded

Returns True since the source_path must exist.

fetch(mirror_only: bool = False, err_msg: str | None = None) → None[source]

Fetch the source code or resources for this stage.

requires_patch_success

Set to True to error out if patches fail

restage()[source]

Remove the expanded source and re-expand it.

property source_path

Returns the development source path.

class spack.stage.ResourceStage(fetch_strategy: FetchStrategy, root: Stage, resource: Resource, *, name=None, mirror_paths: MirrorLayout | None = None, mirrors: Iterable[Mirror] | None = None, keep=False, path=None, lock=True, search_fn=None)[source]

Bases: Stage

expand_archive()[source]

Changes to the stage directory and attempt to expand the downloaded archive. Fail if the stage is not set up or if the archive is not yet downloaded.

restage()[source]

Removes the expanded archive path if it exists, then re-expands the archive.

exception spack.stage.RestageError(message: str, long_message: str | None = None)[source]

Bases: StageError

Error encountered during restaging.

class spack.stage.Stage(url_or_fetch_strategy, *, name=None, mirror_paths: MirrorLayout | None = None, mirrors: Iterable[Mirror] | None = None, keep=False, path=None, lock=True, search_fn=None)[source]

Bases: AbstractStage

Manages a temporary stage directory for building.

A Stage object is a context manager that handles a directory where some source code is downloaded and built before being installed. It handles fetching the source code, either as an archive to be expanded or by checking it out of a repository. A stage’s lifecycle looks like this:

with Stage() as stage:      # Context manager creates and destroys the
                            # stage directory
    stage.fetch()           # Fetch a source archive into the stage.
    stage.expand_archive()  # Expand the archive into source_path.
    <install>               # Build and install the archive.
                            # (handled by user of Stage)

When used as a context manager, the stage is automatically destroyed if no exception is raised by the context. If an exception is raised, the stage is left in the filesystem and NOT destroyed, for potential reuse later.

You can also use the stage’s create/destroy functions manually, like this:

stage = Stage()
try:
    stage.create()          # Explicitly create the stage directory.
    stage.fetch()           # Fetch a source archive into the stage.
    stage.expand_archive()  # Expand the archive into source_path.
    <install>               # Build and install the archive.
                            # (handled by user of Stage)
finally:
    stage.destroy()         # Explicitly destroy the stage directory.

There are two kinds of stages: named and unnamed. Named stages can persist between runs of spack, e.g. if you fetched a tarball but didn’t finish building it, you won’t have to fetch it again.

Unnamed stages are created using standard mkdtemp mechanisms or similar, and are intended to persist for only one run of spack.

property archive_file: str | None

Path to the source archive within this stage directory.

cache_local()[source]

Cache the resources locally.

cache_mirror(mirror: spack.caches.MirrorCache, stats: spack.mirrors.utils.MirrorStatsForOneSpec) → None[source]

Perform a fetch if the resource is not already cached

Parameters:

  • mirror – the mirror to cache this Stage’s resource in

  • stats – this is updated depending on whether the caching operation succeeded or failed

check()[source]

Check the downloaded archive against a checksum digest.

destroy()[source]

Removes this stage directory.

expand_archive()[source]

Changes to the stage directory and attempt to expand the downloaded archive. Fail if the stage is not set up or if the archive is not yet downloaded.

property expanded

Returns True if source path expanded; else False.

property expected_archive_files: List[str]

Possible archive file paths.

fetch(mirror_only: bool = False, err_msg: str | None = None) → None[source]

Retrieves the code or archive

Parameters:

  • mirror_only – only fetch from a mirror

  • err_msg – the error message to display if all fetchers fail or None for the default fetch failure message

requires_patch_success

Set to True to error out if patches fail

restage()[source]

Removes the expanded archive path if it exists, then re-expands the archive.

property save_filename
property single_file
property source_path

Returns the well-known source directory path.

steal_source(dest)[source]

Copy the source_path directory in its entirety to directory dest

This operation creates/fetches/expands the stage if it is not already, and destroys the stage when it is done.

class spack.stage.StageComposite[source]

Bases: object

Composite for Stage type objects. The first item in this composite is considered to be the root package, and operations that return a value are forwarded to it.

append(stage: AbstractStage) → None[source]

Add a stage to the composite.

property archive_file
cache_local() → None[source]

Cache all stages locally.

cache_mirror(mirror: spack.caches.MirrorCache, stats: spack.mirrors.utils.MirrorStatsForOneSpec) → None[source]

Cache all stages to mirror.

check() → None[source]

Check all stages.

create() → None[source]

Create all stages.

destroy() → None[source]

Destroy all stages.

disable_mirrors() → None[source]

Disable mirrors for all stages that support it.

expand_archive() → None[source]

Expand archives for all stages.

property expanded
extend(stages: Iterable[AbstractStage]) → None[source]

Add multiple stages to the composite.

fetch(mirror_only: bool = False, err_msg: str | None = None) → None[source]

Fetch all stages.

classmethod from_iterable(iterable: Iterable[AbstractStage]) → StageComposite[source]

Create a new composite from an iterable of stages.

property keep
property path
property requires_patch_success
restage() → None[source]

Restage all stages.

property source_path
steal_source(dest: str) → None[source]

Steal source from all stages.

exception spack.stage.StageError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Superclass for all errors encountered during staging.

exception spack.stage.StagePathError(message: str, long_message: str | None = None)[source]

Bases: StageError

Error encountered with stage path.

exception spack.stage.VersionFetchError(message: str, long_message: str | None = None)[source]

Bases: StageError

Raised when we can’t determine a URL to fetch a package.

spack.stage.compute_stage_name(spec)[source]

Determine stage name given a spec

spack.stage.create_stage_root(path: str) → None[source]

Create the stage root directory and ensure appropriate access perms.

spack.stage.ensure_access(file)[source]

Ensure we can access a directory and die with an error if we can’t.

spack.stage.get_checksums_for_versions(url_by_version: Dict[StandardVersion, str], package_name: str, *, first_stage_function: Callable[[str, str], None] | None = None, keep_stage: bool = False, concurrency: int | None = None, fetch_options: Dict[str, str] | None = None) → Dict[StandardVersion, str][source]

Computes the checksums for each version passed in input, and returns the results.

Archives are fetched according to the usl dictionary passed as input.

The first_stage_function argument allows the caller to inspect the first downloaded archive, e.g., to determine the build system.

Parameters:

  • url_by_version – URL keyed by version

  • package_name – name of the package

  • first_stage_function – function that takes an archive file and a URL; this is run on the stage of the first URL downloaded

  • keep_stage – whether to keep staging area when command completes

  • batch – whether to ask user how many versions to fetch (false) or fetch all versions (true)

  • fetch_options – options used for the fetcher (such as timeout or cookies)

  • concurrency – maximum number of workers to use for retrieving archives

Returns:

A dictionary mapping each version to the corresponding checksum

spack.stage.get_stage_root()[source]
spack.stage.interactive_version_filter(url_dict: Dict[StandardVersion, str], known_versions: Iterable[StandardVersion] = (), *, initial_verion_filter: VersionList | None = None, url_changes: Set[StandardVersion] = set(), input: Callable[[...], str] = input) → Dict[StandardVersion, str] | None[source]

Interactively filter the list of spidered versions.

Parameters:

  • url_dict – Dictionary of versions to URLs

  • known_versions – Versions that can be skipped because they are already known

Returns:

Filtered dictionary of versions to URLs or None if the user wants to quit

spack.stage.purge()[source]

Remove all build directories in the top-level stage path.

spack.store module

Components that manage Spack’s installation tree.

An install tree, or “build store” consists of two parts:

  1. A package database that tracks what is installed.

  2. A directory layout that determines how the installations are laid out.

The store contains all the install prefixes for packages installed by Spack. The simplest store could just contain prefixes named by DAG hash, but we use a fancier directory layout to make browsing the store and debugging easier.

spack.store.DEFAULT_INSTALL_TREE_ROOT

default installation root, relative to the Spack install path

exception spack.store.MatchError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Error occurring when trying to match specs in store against a constraint

spack.store.STORE

Singleton store instance

class spack.store.Store(root: str, unpadded_root: str | None = None, projections: Dict[str, str] | None = None, hash_length: int | None = None, upstreams: List[Database] | None = None, lock_cfg: LockConfiguration = spack.database.NO_LOCK)[source]

Bases: object

A store is a path full of installed Spack packages.

Stores consist of packages installed according to a DirectoryLayout, along with a database of their contents.

The directory layout controls what paths look like and how Spack ensures that each unique spec gets its own unique directory (or not, though we don’t recommend that).

The database is a single file that caches metadata for the entire Spack installation. It prevents us from having to spider the install tree to figure out what’s there.

The store is also able to lock installation prefixes, and to mark installation failures.

Parameters:

  • root – path to the root of the install tree

  • unpadded_root – path to the root of the install tree without padding. The sbang script has to be installed here to work with padded roots

  • projections – expression according to guidelines that describes how to construct a path to a package prefix in this store

  • hash_length – length of the hashes used in the directory layout. Spec hash suffixes will be truncated to this length

  • upstreams – optional list of upstream databases

  • lock_cfg – lock configuration for the database

has_padding() → bool[source]

Returns True if the store layout includes path padding.

install_sbang() → None[source]

Install the sbang script in this store’s bin directory.

sbang is a short shell script that Spack prepends to scripts with shebangs that are too long for the OS. It must live in the store so its path is short enough to fit on a shebang line.

reindex() → None[source]

Convenience function to reindex the store DB with its own layout.

spack.store.create(configuration: Configuration) → Store[source]

Create a store from the configuration passed as input.

Parameters:

configuration – configuration to create a store.

spack.store.ensure_singleton_created() → None[source]

Ensures the lazily evaluated singleton is created

spack.store.find(constraints: str | List[str] | List[Spec], multiple: bool = False, query_fn: Callable[[Any], List[Spec]] | None = None, **kwargs) → List[Spec][source]

Returns a list of specs matching the constraints passed as inputs.

At least one spec per constraint must match, otherwise the function will error with an appropriate message.

By default, this function queries the current store, but a custom query function can be passed to hit any other source of concretized specs (e.g. a binary cache).

The query function must accept a spec as its first argument.

Parameters:

  • constraints – spec(s) to be matched against installed packages

  • multiple – if True multiple matches per constraint are admitted

  • query_fn (Callable) – query function to get matching specs. By default, spack.store.STORE.db.query

  • **kwargs – keyword arguments forwarded to the query function

spack.store.parse_install_tree(config_dict: dict) → Tuple[str, str, Dict[str, str]][source]

Parse config settings and return values relevant to the store object.

Parameters:

config_dict – dictionary of config values, as returned from spack.config.get("config")

Returns:

triple of the install tree root, the unpadded install tree root (before padding was applied), and the projections for the install tree

Encapsulate backwards compatibility capabilities for install_tree and deprecated values that are now parsed as part of install_tree.

spack.store.reinitialize()[source]

Restore globals to the same state they would have at start-up. Return a token containing the state of the store before reinitialization.

spack.store.restore(token)[source]

Restore the environment from a token returned by reinitialize

spack.store.specfile_matches(filename: str, **kwargs) → List[Spec][source]

Same as find but reads the query from a spec file.

Parameters:

  • filename – YAML or JSON file from which to read the query.

  • **kwargs – keyword arguments forwarded to find()

spack.store.use_store(path: str | Path, extra_data: Dict[str, Any] | None = None) → Generator[Store, None, None][source]

Use the store passed as argument within the context manager.

Parameters:

  • path – path to the store.

  • extra_data – extra configuration under config:install_tree to be taken into account.

Yields:

Store object associated with the context manager’s store

spack.subprocess_context module

This module handles transmission of Spack state to child processes started using the "spawn" start method. Notably, installations are performed in a subprocess and require transmitting the Package object (in such a way that the repository is available for importing when it is deserialized); installations performed in Spack unit tests may include additional modifications to global state in memory that must be replicated in the child process.

class spack.subprocess_context.GlobalStateMarshaler(*, ctx: BaseContext | None = None)[source]

Bases: object

Class to serialize and restore global state for child processes if needed.

Spack may modify state that is normally read from disk or command line in memory; this object is responsible for properly serializing that state to be applied to a subprocess.

restore()[source]
spack.subprocess_context.MONKEYPATCHES: list

Used in tests to track monkeypatches that need to be restored in child processes

class spack.subprocess_context.PackageInstallContext(pkg: PackageBase, *, ctx: BaseContext | None = None)[source]

Bases: object

Captures the in-memory process state of a package installation that needs to be transmitted to a child process.

restore() → PackageBase[source]
class spack.subprocess_context.SpackTestProcess(fn)[source]

Bases: object

create()[source]
class spack.subprocess_context.TestPatches(module_patches, class_patches)[source]

Bases: object

static create()[source]
restore()[source]
spack.subprocess_context.deserialize(serialized_pkg: BytesIO) → PackageBase[source]
spack.subprocess_context.serialize(pkg: PackageBase) → BytesIO[source]

spack.tag module

Classes and functions to manage package tags

class spack.tag.TagIndex[source]

Bases: object

Maps tags to list of package names.

static from_json(stream) → TagIndex[source]
get_packages(tag: str) → List[str][source]

Returns all packages associated with the tag.

merge(other: TagIndex) → None[source]

Merge another tag index into this one.

Parameters:

other – tag index to be merged

to_json(stream) → None[source]
update_packages(pkg_names: Set[str], repo: Repo) → None[source]

Updates packages in the tag index.

Parameters:

  • pkg_names – names of the packages to be updated

  • repo – the repository to get package classes from

exception spack.tag.TagIndexError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when there is a problem with a TagIndex.

spack.tengine module

class spack.tengine.Context[source]

Bases: object

Base class for context classes that are used with the template engine.

context_properties: List[str]
to_dict() → Dict[str, Any][source]

Returns a dictionary containing all the context properties.

class spack.tengine.ContextMeta(name, bases, attr_dict)[source]

Bases: type

Metaclass for Context. It helps reduce the boilerplate in client code.

classmethod context_property(func)[source]

Decorator that adds a function name to the list of new context properties, and then returns a property.

spack.tengine.context_property(func)

A saner way to use the decorator

spack.tengine.curly_quote(text)[source]

Encloses each line of text in curly braces

spack.tengine.default_template_dirs(configuration: Configuration) → Tuple[str, ...][source]
spack.tengine.make_environment(dirs: Tuple[str, ...] | None = None) → spack.vendor.jinja2.Environment[source]

Returns a configured environment for template rendering.

spack.tengine.make_environment_from_dirs(dirs: Tuple[str, ...]) → spack.vendor.jinja2.Environment[source]
spack.tengine.prepend_to_line(text, token)[source]

Prepends a token to each line in text

spack.tengine.quote(text)[source]

Quotes each line in text

spack.tokenize module

This module provides building blocks for tokenizing strings. Users can define tokens by inheriting from TokenBase and defining tokens as ordered enum members. The Tokenizer class can then be used to iterate over tokens in a string.

class spack.tokenize.Token(kind: TokenBase, value: str, start: int = 0, end: int = 0, **kwargs)[source]

Bases: object

Represents tokens; generated from input by lexer and fed to parse().

end
kind
start
subvalues
value
class spack.tokenize.TokenBase(new_class_name, /, names, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Base class for an enum type with a regex value

class spack.tokenize.Tokenizer(tokens: Type[TokenBase])[source]

Bases: object

tokenize(text: str) → Generator[Token, None, None][source]
spack.tokenize.token_match_regex(token: TokenBase)[source]

Generate a regular expression that matches the provided token and its subvalues.

This will extract named capture groups from the provided regex and prefix them with token name, so they can coexist together in a larger, joined regular expression.

Returns:

A regex with a capture group for the token and rewritten capture groups for any subvalues.

spack.traverse module

spack.traverse.traverse_edges(specs: Sequence['spack.spec.Spec'], *, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[False] = False, key: Callable[['spack.spec.Spec'], Any] = id, visited: Set[Any] | None = None) → Iterable['spack.spec.DependencySpec'][source]
spack.traverse.traverse_edges(specs: Sequence['spack.spec.Spec'], *, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[True], key: Callable[['spack.spec.Spec'], Any] = id, visited: Set[Any] | None = None) → Iterable[Tuple[int, 'spack.spec.DependencySpec']]
spack.traverse.traverse_edges(specs: Sequence['spack.spec.Spec'], *, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: bool, key: Callable[['spack.spec.Spec'], Any] = id, visited: Set[Any] | None = None) → Iterable['spack.spec.DependencySpec' | Tuple[int, 'spack.spec.DependencySpec']]

Iterable of edges from the DAG, starting from a list of root specs.

Parameters:

  • specs – List of root specs (considered to be depth 0)

  • root – Yield the root nodes themselves

  • order – What order of traversal to use in the DAG. For depth-first search this can be pre or post. For BFS this should be breadth. For topological order use topo

  • cover – Determines how extensively to cover the dag. Possible values: nodes – Visit each unique node in the dag only once. edges – If a node has been visited once but is reached along a new path, it’s accepted, but not recursively followed. This traverses each ‘edge’ in the DAG once. paths – Explore every unique path reachable from the root. This descends into visited subtrees and will accept nodes multiple times if they’re reachable by multiple paths.

  • direction – children or parents. If children, does a traversal of this spec’s children. If parents, traverses upwards in the DAG towards the root.

  • deptype – allowed dependency types

  • depth – When False, yield just edges. When True yield the tuple (depth, edge), where depth corresponds to the depth at which edge.spec was discovered.

  • key – function that takes a spec and outputs a key for uniqueness test.

  • visited – a set of nodes not to follow

Returns:

An iterable of DependencySpec if depth is False or a tuple of (depth, DependencySpec) if depth is True.

spack.traverse.traverse_nodes(specs: Sequence['spack.spec.Spec'], *, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[False] = False, key: Callable[['spack.spec.Spec'], Any] = id, visited: Set[Any] | None = None) → Iterable['spack.spec.Spec'][source]
spack.traverse.traverse_nodes(specs: Sequence['spack.spec.Spec'], *, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: Literal[True], key: Callable[['spack.spec.Spec'], Any] = id, visited: Set[Any] | None = None) → Iterable[Tuple[int, 'spack.spec.Spec']]
spack.traverse.traverse_nodes(specs: Sequence['spack.spec.Spec'], *, root: bool = True, order: Literal['pre', 'post', 'breadth', 'topo'] = 'pre', cover: Literal['nodes', 'edges', 'paths'] = 'nodes', direction: Literal['children', 'parents'] = 'children', deptype: int | str | List[str] | Tuple[str, ...] = 'all', depth: bool, key: Callable[['spack.spec.Spec'], Any] = id, visited: Set[Any] | None = None) → Iterable['spack.spec.Spec' | Tuple[int, 'spack.spec.Spec']]

Iterable of specs from the DAG, starting from a list of root specs.

Parameters:

  • specs – List of root specs (considered to be depth 0)

  • root – Yield the root nodes themselves

  • order – What order of traversal to use in the DAG. For depth-first search this can be pre or post. For BFS this should be breadth.

  • cover – Determines how extensively to cover the dag. Possible values: nodes – Visit each unique node in the dag only once. edges – If a node has been visited once but is reached along a new path, it’s accepted, but not recursively followed. This traverses each ‘edge’ in the DAG once. paths – Explore every unique path reachable from the root. This descends into visited subtrees and will accept nodes multiple times if they’re reachable by multiple paths.

  • direction – children or parents. If children, does a traversal of this spec’s children. If parents, traverses upwards in the DAG towards the root.

  • deptype – allowed dependency types

  • depth – When False, yield just edges. When True yield the tuple (depth, edge), where depth corresponds to the depth at which edge.spec was discovered.

  • key – function that takes a spec and outputs a key for uniqueness test.

  • visited – a set of nodes not to follow

Yields:

By default Spec, or a tuple (depth, Spec) if depth is set to True.

spack.traverse.traverse_tree(specs: Sequence[spack.spec.Spec], cover: Literal['nodes', 'edges', 'paths'] = 'nodes', deptype: int | str | List[str] | Tuple[str, ...] = 'all', key: Callable[[spack.spec.Spec], Any] = id, depth_first: bool = True) → Iterable[Tuple[int, spack.spec.DependencySpec]][source]

Generator that yields (depth, DependencySpec) tuples in the depth-first pre-order, so that a tree can be printed from it.

Parameters:

  • specs – List of root specs (considered to be depth 0)

  • cover – Determines how extensively to cover the dag. Possible values: nodes – Visit each unique node in the dag only once. edges – If a node has been visited once but is reached along a new path, it’s accepted, but not recursively followed. This traverses each ‘edge’ in the DAG once. paths – Explore every unique path reachable from the root. This descends into visited subtrees and will accept nodes multiple times if they’re reachable by multiple paths.

  • deptype – allowed dependency types

  • key – function that takes a spec and outputs a key for uniqueness test.

  • depth_first – Explore the tree in depth-first or breadth-first order. When setting depth_first=True and cover=nodes, each spec only occurs once at the shallowest level, which is useful when rendering the tree in a terminal.

Returns:

A generator that yields (depth, DependencySpec) tuples in such an order that a tree can be printed.

spack.url module

This module has methods for parsing names and versions of packages from URLs. The idea is to allow package creators to supply nothing more than the download location of the package, and figure out version and name information from there.

Example: when spack is given the following URL:

https://www.hdfgroup.org/ftp/HDF/releases/HDF4.2.12/src/hdf-4.2.12.tar.gz

It can figure out that the package name is hdf, and that it is at version 4.2.12. This is useful for making the creation of packages simple: a user just supplies a URL and skeleton code is generated automatically.

Spack can also figure out that it can most likely download 4.2.6 at this URL:

https://www.hdfgroup.org/ftp/HDF/releases/HDF4.2.6/src/hdf-4.2.6.tar.gz

This is useful if a user asks for a package at a particular version number; spack doesn’t need anyone to tell it where to get the tarball even though it’s never been told about that version before.

exception spack.url.UndetectableNameError(path)[source]

Bases: UrlParseError

Raised when we can’t parse a package name from a string.

exception spack.url.UndetectableVersionError(path)[source]

Bases: UrlParseError

Raised when we can’t parse a version from a string.

exception spack.url.UrlParseError(msg, path)[source]

Bases: SpackError

Raised when the URL module can’t parse something correctly.

spack.url.color_url(path, **kwargs)[source]

Color the parts of the url according to Spack’s parsing.

Colors are:

Parameters:

  • path (str) – The filename or URL for the package

  • errors (bool) – Append parse errors at end of string.

  • subs (bool) – Color substitutions as well as parsed name/version.

spack.url.find_all(substring, string)[source]

Returns a list containing the indices of every occurrence of substring in string.

spack.url.find_versions_of_archive(archive_urls: str | Sequence[str], list_url: str | None = None, list_depth: int = 0, concurrency: int | None = 32, reference_package: Any | None = None) → Dict[StandardVersion, str][source]

Scrape web pages for new versions of a tarball. This function prefers URLs in the following order: links found on the scraped page that match a url generated by the reference package, found and in the archive_urls list, found and derived from those in the archive_urls list, and if none are found for a version then the item in the archive_urls list is included for the version.

Parameters:

  • archive_urls – URL or sequence of URLs for different versions of a package. Typically these are just the tarballs from the package file itself. By default, this searches the parent directories of archives.

  • list_url – URL for a listing of archives. Spack will scrape these pages for download links that look like the archive URL.

  • list_depth – max depth to follow links on list_url pages. Defaults to 0.

  • concurrency – maximum number of concurrent requests

  • reference_package – a spack package used as a reference for url detection. Uses the url_for_version method on the package to produce reference urls which, if found, are preferred.

spack.url.parse_name(path, ver=None)[source]

Try to determine the name of a package from its filename or URL.

Parameters:

  • path (str) – The filename or URL for the package

  • ver (str) – The version of the package

Returns:

The name of the package

Return type:

str

Raises:

UndetectableNameError – If the URL does not match any regexes

spack.url.parse_name_and_version(path: str) → Tuple[str, StandardVersion][source]

Try to determine the name of a package and extract its version from its filename or URL.

Parameters:

path – The filename or URL for the package

Returns:

a tuple containing the package (name, version)

Return type:

tuple

Raises:
spack.url.parse_name_offset(path: str, v: str | StandardVersion | None = None) → Tuple[str, int, int, int, str][source]

Try to determine the name of a package from its filename or URL.

Parameters:

  • path – The filename or URL for the package

  • v – The version of the package

Returns:

A tuple containing

  • name of the package

  • first index of name

  • length of name

  • the index of the matching regex

  • the matching regex

Raises:

UndetectableNameError – If the URL does not match any regexes

spack.url.parse_version(path: str) → StandardVersion[source]

Try to extract a version string from a filename or URL.

Parameters:

path – The filename or URL for the package

Returns: The version of the package

Raises:

UndetectableVersionError – If the URL does not match any regexes

spack.url.parse_version_offset(path: str) → Tuple[str, int, int, int, str][source]

Try to extract a version string from a filename or URL.

Parameters:

path (str) – The filename or URL for the package

Returns:

A tuple containing

  • version of the package

  • first index of version

  • length of version string

  • the index of the matching regex

  • the matching regex

Raises:

UndetectableVersionError – If the URL does not match any regexes

spack.url.strip_name_suffixes(path: str, version: str | StandardVersion) → str[source]

Most tarballs contain a package name followed by a version number. However, some also contain extraneous information in-between the name and version:

  • rgb-1.0.6

  • converge_install_2.3.16

  • jpegsrc.v9b

These strings are not part of the package name and should be ignored. This function strips the version number and any extraneous suffixes off and returns the remaining string. The goal is that the name is always the last thing in path:

  • rgb

  • converge

  • jpeg

Parameters:

  • path – The filename or URL for the package

  • version – The version detected for this URL

Returns:

The path with any extraneous suffixes removed

Return type:

str

spack.url.substitute_version(path: str, new_version) → str[source]

Given a URL or archive name, find the version in the path and substitute the new version for it. Replace all occurrences of the version if they don’t overlap with the package name.

Simple example:

>>> substitute_version("http://www.mr511.de/software/libelf-0.8.13.tar.gz", "2.9.3")
"http://www.mr511.de/software/libelf-2.9.3.tar.gz"

Complex example:

>>> substitute_version("https://www.hdfgroup.org/ftp/HDF/releases/HDF4.2.12/src/hdf-4.2.12.tar.gz", "2.3")
"https://www.hdfgroup.org/ftp/HDF/releases/HDF2.3/src/hdf-2.3.tar.gz"
spack.url.substitution_offsets(path)[source]

This returns offsets for substituting versions and names in the provided path. It is a helper for substitute_version().

spack.url.wildcard_version(path)[source]

Find the version in the supplied path, and return a regular expression that will match this path with any version in its place.

spack.url_buildcache module

class spack.url_buildcache.BlobRecord(content_length: int, media_type: str, compression_alg: str, checksum_alg: str, checksum: str)[source]

Bases: object

Class to describe a single data element (blob) from a manifest

classmethod from_dict(record_dict)[source]
to_dict()[source]
class spack.url_buildcache.BuildcacheComponent(*values)[source]

Bases: Enum

Enumeration of the kinds of things that live in a URL buildcache

These enums serve two purposes: They allow different buildcache layout versions to specify different relative location of these entities, and they’re used to map buildcache objects to their respective media types.

BLOB
INDEX
KEY
KEY_INDEX
LAYOUT_JSON
MANIFEST
SPEC
TARBALL
exception spack.url_buildcache.BuildcacheEntryError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised for problems finding or accessing binary cache entry on mirror

class spack.url_buildcache.BuildcacheManifest(layout_version: int, data: List[BlobRecord] | None = None)[source]

Bases: object

A class to represent a buildcache manifest, which consists of a version number and an array of data blobs, each of which is represented by a BlobRecord.

data: List[BlobRecord]
classmethod from_dict(manifest_json: Dict[str, Any]) → BuildcacheManifest[source]
get_blob_records(media_type: str) → List[BlobRecord][source]

Return any blob records from the manifest matching the given media type

to_dict()[source]
version: int
spack.url_buildcache.CURRENT_BUILD_CACHE_LAYOUT_VERSION

The build cache layout version that this version of Spack creates. Version 3: Introduces content-addressable tarballs

spack.url_buildcache.INDEX_MANIFEST_FILE

The name of the default buildcache index manifest file

exception spack.url_buildcache.InvalidMetadataFile(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when spack encounters a spec file it cannot understand or process

exception spack.url_buildcache.ListMirrorSpecsError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when unable to retrieve list of specs from the mirror

class spack.url_buildcache.MirrorMetadata(url: str, version: int, view: str | None = None)[source]

Bases: object

Simple class to hold a mirror url and a buildcache layout version

This class is used by BinaryIndexCache to produce a key used to keep track of downloaded/processed buildcache index files from remote mirrors in some layout version.

classmethod from_string(s: str) → MirrorMetadata[source]
strip_view() → MirrorMetadata[source]
url
version
view
exception spack.url_buildcache.MirrorMetadataError(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when unable to interpret a MirrorMetadata string

exception spack.url_buildcache.NoSuchBlobException(message: str, long_message: str | None = None)[source]

Bases: SpackError

Raised when manifest does have some requested type of requested type

exception spack.url_buildcache.NoVerifyException(message: str, long_message: str | None = None)[source]

Bases: BuildcacheEntryError

Raised if file fails signature verification

class spack.url_buildcache.URLBuildcacheEntry(mirror_url: str, spec: Spec | None = None, allow_unsigned: bool = False)[source]

Bases: object

A class for managing URL-style buildcache entries

This class manages access to a versioned buildcache entry by providing a means to download both the metadata (spec file) and compressed archive. It also provides methods for accessing the paths/urls associated with buildcache entries.

Starting with buildcache layout version 3, it is not possible to know the full path to a compressed archive without either building it locally, or else fetching and reading the metadata first. This class provides api for fetching the metadata, as well as fetching the archive, and it enforces the need to fetch the metadata first.

To help with downloading, this class manages two spack.spec.Stage objects internally, which must be destroyed when finished. Specifically, if you call either of the following methods on an instance, you must eventually also call destroy():

fetch_metadata()
fetch_archive()

This class also provides generic manifest and blob management api, and it can be used to fetch and push other kinds of buildcache entries aside from just binary packages. It can be used to work with public keys, buildcache indices, and any other type of data represented as a manifest which refers to blobs of data.

BUILDCACHE_INDEX_FILE
BUILDCACHE_INDEX_MEDIATYPE
COMPONENT_PATHS
LAYOUT_VERSION
PUBLIC_KEY_INDEX_MEDIATYPE
PUBLIC_KEY_MEDIATYPE
SPEC_MEDIATYPE
SPEC_URL_REGEX
TARBALL_MEDIATYPE
allow_unsigned: bool
check_blob_exists(record: BlobRecord) → bool[source]

Return True if the blob given by record exists on the mirror, False otherwise

classmethod check_layout_json_exists(mirror_url: str) → bool[source]

Return True if layout.json exists in the expected location, False otherwise

classmethod component_to_media_type(component: BuildcacheComponent) → str[source]

Mapping from buildcache component to media type

destroy()[source]

Destroy any existing stages

exists(components: List[BuildcacheComponent]) → bool[source]

Check whether blobs exist for all specified components

Returns True if there is a blob present in the mirror for every given component type.

fetch_archive() → str[source]

Retrieve the archive file and return the local archive file path

fetch_blob(record: BlobRecord) → str[source]

Given a blob record, find associated blob in the manifest and stage it

Returns the local path to the staged blob

fetch_metadata() → dict[source]

Retrieve metadata for the spec, returns the validated spec dict

get_archive_stage() → Stage | None[source]
classmethod get_base_url(manifest_url: str) → str[source]

Given any manifest url (i.e. one containing v3/manifests/) return the base part of the url

classmethod get_blob_path_components(record: BlobRecord) → List[str][source]

Given a BlobRecord, return the relative path of the blob within a mirror as a list of path components

get_blob_record(blob_type: BuildcacheComponent) → BlobRecord[source]

Return the first blob record of the given type. Assumes the manifest has already been fetched.

classmethod get_blob_url(mirror_url: str, record: BlobRecord) → str[source]

Return the full url of the blob given by record

classmethod get_buildcache_component_include_pattern(buildcache_component: BuildcacheComponent) → str[source]

Given a buildcache component, return the glob pattern that can be used to match it in a directory listing. If None is provided, return a catch-all pattern that will match all buildcache components.

classmethod get_index_url(mirror_url: str, view: str | None = None)[source]
classmethod get_layout_version() → int[source]

Returns the layout version of this class

get_local_archive_path() → str[source]

Convenience method to return the local path of a fetched tarball

get_local_spec_path() → str[source]

Convenience method to return the local path of a fetched spec file

classmethod get_manifest_filename(spec: Spec) → str[source]

Given a concrete spec, compute and return the name (i.e. basename) of the manifest file representing it

classmethod get_manifest_url(spec: Spec, mirror_url: str) → str[source]

Given a concrete spec and a base url, return the full url where the spec manifest should be found

classmethod get_relative_path_components(component: BuildcacheComponent) → List[str][source]

Given any type of buildcache component, return its relative location within a mirror as a list path elements

get_staged_blob_path(record: BlobRecord) → str[source]

Convenience method to return the local path of a staged blob

manifest: BuildcacheManifest | None
classmethod maybe_push_layout_json(mirror_url: str) → None[source]

This function does nothing if layout.json already exists, otherwise it pushes layout.json to the expected location in the mirror

mirror_url: str
push_binary_package(spec: Spec, tarball_path: str, checksum_algorithm: str, tarball_checksum: str, tmpdir: str, signing_key: str | None) → None[source]

Convenience method to push tarball, specfile, and manifest to the remote mirror

Pushing should only be done after checking for the pre-existence of a buildcache entry for this spec, and represents a force push if one is found. Thus, any pre-existing files are first removed.

classmethod push_blob(mirror_url: str, blob_path: str, record: BlobRecord) → None[source]

Push the blob_path file to mirror as a blob represented by the given record

classmethod push_local_file_as_blob(local_file_path: str, mirror_url: str, manifest_name: str, component_type: BuildcacheComponent, compression: str = 'none') → None[source]

Convenience method to push a local file to a mirror as a blob. Both manifest and blob are pushed as a component of the given component_type. If compression is "gzip" the blob will be compressed before pushing, otherwise it will be pushed uncompressed.

classmethod push_manifest(mirror_url: str, manifest_name: str, manifest: BuildcacheManifest, tmpdir: str, component_type: BuildcacheComponent = BuildcacheComponent.SPEC, signing_key: str | None = None) → None[source]

Given a BuildcacheManifest, push it to the mirror using the given manifest name. The component_type is used to indicate what type of thing the manifest represents, so it can be placed in the correct relative path within the mirror. If a signing_key is provided, it will be used to clearsign the manifest before pushing it.

read_manifest(manifest_url: str | None = None) → BuildcacheManifest[source]

Read and process the the buildcache entry manifest.

If no manifest url is provided, build the url from the internal spec and base push url.

remote_manifest_url: str
remove()[source]

Remove a binary package (spec file and tarball) and the associated manifest from the mirror.

spec: Spec | None
stages: Dict[BlobRecord, Stage]
classmethod verify_and_extract_manifest(manifest_contents: str, verify: bool = False) → dict[source]

Possibly verify clearsig, then extract contents and return as json

class spack.url_buildcache.URLBuildcacheEntryV2(push_url_base: str, spec: Spec | None = None, allow_unsigned: bool = False)[source]

Bases: URLBuildcacheEntry

This class exists to provide read-only support for reading older buildcache layouts in a way that is transparent to binary_distribution code responsible for downloading and extracting binary packages. Since support for layout v2 is read-only, and since v2 did not have support for manifests and blobs, many class and instance methods are overridden simply to raise, hopefully making the intended use and limitations of the class clear to developers.

BUILDCACHE_INDEX_FILE
COMPONENT_PATHS
LAYOUT_VERSION
SPEC_URL_REGEX
allow_unsigned: bool
archive_stage: Stage | None
check_blob_exists(record: BlobRecord) → bool[source]

Return True if the blob given by record exists on the mirror, False otherwise

destroy()[source]

Destroy any existing stages

exists(components: List[BuildcacheComponent]) → bool[source]

Check whether blobs exist for all specified components

Returns True if there is a blob present in the mirror for every given component type.

fetch_archive() → str[source]

Retrieve the archive file and return the local archive file path

fetch_blob(record: BlobRecord) → str[source]

Given a blob record, find associated blob in the manifest and stage it

Returns the local path to the staged blob

fetch_metadata() → dict[source]

Retrieve the v2 specfile for the spec, yields the validated spec+ dict

get_archive_stage() → Stage | None[source]
classmethod get_blob_path_components(record: BlobRecord) → List[str][source]

Given a BlobRecord, return the relative path of the blob within a mirror as a list of path components

get_blob_record(blob_type: BuildcacheComponent) → BlobRecord[source]

Return the first blob record of the given type. Assumes the manifest has already been fetched.

classmethod get_blob_url(mirror_url: str, record: BlobRecord) → str[source]

Return the full url of the blob given by record

classmethod get_buildcache_component_include_pattern(buildcache_component: BuildcacheComponent) → str[source]

Given a buildcache component, return the glob pattern that can be used to match it in a directory listing. If None is provided, return a catch-all pattern that will match all buildcache components.

classmethod get_layout_version() → int[source]

Returns the layout version of this class

classmethod get_manifest_filename(spec: Spec) → str[source]

Given a concrete spec, compute and return the name (i.e. basename) of the manifest file representing it

classmethod get_manifest_url(spec: Spec, mirror_url: str) → str[source]

Given a concrete spec and a base url, return the full url where the spec manifest should be found

get_staged_blob_path(record: BlobRecord) → str[source]

Convenience method to return the local path of a staged blob

has_metadata: bool
has_signed: bool
has_tarball: bool
has_unsigned: bool
local_archive_path: str
local_specfile_path: str
classmethod maybe_push_layout_json(mirror_url: str) → None[source]

This function does nothing if layout.json already exists, otherwise it pushes layout.json to the expected location in the mirror

mirror_url: str
push_binary_package(spec: Spec, tarball_path: str, checksum_algorithm: str, tarball_checksum: str, tmpdir: str, signing_key: str | None) → None[source]

Convenience method to push tarball, specfile, and manifest to the remote mirror

Pushing should only be done after checking for the pre-existence of a buildcache entry for this spec, and represents a force push if one is found. Thus, any pre-existing files are first removed.

classmethod push_blob(mirror_url: str, blob_path: str, record: BlobRecord) → None[source]

Push the blob_path file to mirror as a blob represented by the given record

classmethod push_local_file_as_blob(local_file_path: str, mirror_url: str, manifest_name: str, component_type: BuildcacheComponent, compression: str = 'none') → None[source]

Convenience method to push a local file to a mirror as a blob. Both manifest and blob are pushed as a component of the given component_type. If compression is "gzip" the blob will be compressed before pushing, otherwise it will be pushed uncompressed.

classmethod push_manifest(mirror_url: str, manifest_name: str, manifest: BuildcacheManifest, tmpdir: str, component_type: BuildcacheComponent = BuildcacheComponent.SPEC, signing_key: str | None = None) → None[source]

Given a BuildcacheManifest, push it to the mirror using the given manifest name. The component_type is used to indicate what type of thing the manifest represents, so it can be placed in the correct relative path within the mirror. If a signing_key is provided, it will be used to clearsign the manifest before pushing it.

read_manifest(manifest_url: str | None = None) → BuildcacheManifest[source]

Read and process the the buildcache entry manifest.

If no manifest url is provided, build the url from the internal spec and base push url.

remote_archive_checksum_algorithm: str
remote_archive_checksum_hash: str
remote_archive_url: str
remote_spec_url: str
remove()[source]

Remove a binary package (spec file and tarball) and the associated manifest from the mirror.

spec: Spec | None
spec_dict: Dict[Any, Any]
spec_stage: Stage | None
classmethod verify_and_extract_manifest(manifest_contents: str, verify: bool = False) → dict[source]

Possibly verify clearsig, then extract contents and return as json

exception spack.url_buildcache.UnknownBuildcacheLayoutError(message: str, long_message: str | None = None)[source]

Bases: BuildcacheEntryError

Raised when unrecognized buildcache layout version is encountered

spack.url_buildcache.check_mirror_for_layout(mirror: Mirror)[source]

Check specified mirror, and warn if missing layout.json

spack.url_buildcache.compressed_json_from_dict(output_path: str, spec_dict: dict, checksum_algo: str) → Tuple[str, int][source]

Compress the spec dict and write it to the given path

Return the checksum (using the given algorithm) and size on disk of the file

spack.url_buildcache.compression_writer(output_path: str, compression: str, checksum_algo: str)[source]

Create and return a writer capable of writing compressed data. Available options for compression are "gzip" or "none", checksum_algo is used to pick the checksum algorithm used by the ChecksumWriter.

Yields:

A tuple containing

spack.url_buildcache.get_entries_from_cache(url: str, tmpspecsdir: str, component_type: BuildcacheComponent)[source]

Get a list of all the manifests in the mirror and a function to read them.

Parameters:

  • url – Base url of mirror (location of spec files)

  • tmpspecsdir – Temporary location for writing files

  • component_type – type of buildcache component to sync (spec, index, key, etc.)

Returns:

A tuple where the first item is a list of absolute file paths or urls pointing to the manifests that should be read from the mirror, and the second item is a function taking a url or file path and returning a URLBuildcacheEntry for that manifest.

spack.url_buildcache.get_url_buildcache_class(layout_version: int = CURRENT_BUILD_CACHE_LAYOUT_VERSION) → Type[URLBuildcacheEntry][source]

Given a layout version, return the class responsible for managing access to buildcache entries of that version

spack.url_buildcache.get_valid_spec_file(path: str, max_supported_layout: int) → Tuple[Dict, int][source]

Read and validate a spec file, returning the spec dict with its layout version, or raising InvalidMetadataFile if invalid.

spack.url_buildcache.sign_file(key: str, file_path: str) → str[source]

sign and return the path to the signed file

spack.url_buildcache.try_verify(specfile_path)[source]

Utility function to attempt to verify a local file. Assumes the file is a clearsigned signature file.

Parameters:

specfile_path (str) – Path to file to be verified.

Returns:

True if the signature could be verified, False otherwise.

spack.url_buildcache.validate_checksum(file_path, checksum_algorithm, expected_checksum) → None[source]

Compute the checksum of the given file and raise if invalid

spack.user_environment module

spack.user_environment.environment_modifications_for_specs(*specs: Spec, view=None, set_package_py_globals: bool = True)[source]

List of environment (shell) modifications to be processed for spec.

This list is specific to the location of the spec or its projection in the view.

Parameters:

  • specs – spec(s) for which to list the environment modifications

  • view – view associated with the spec passed as first argument

  • set_package_py_globals – whether or not to set the global variables in the package.py files (this may be problematic when using buildcaches that have been built on a different but compatible OS)

spack.user_environment.prefix_inspections(platform: str) → dict[source]

Get list of prefix inspections for platform

Parameters:

platform – the name of the platform to consider. The platform determines what environment variables Spack will use for some inspections.

Returns:

A dictionary mapping subdirectory names to lists of environment variables to modify with that directory if it exists.

spack.user_environment.project_env_mods(*specs: Spec, view, env: EnvironmentModifications) → None[source]

Given a list of environment modifications, project paths changes to the view.

spack.user_environment.spack_loaded_hashes_var

Environment variable name Spack uses to track individually loaded packages

spack.user_environment.unconditional_environment_modifications(view)[source]

List of environment (shell) modifications to be processed for view.

This list does not depend on the specs in this environment

spack.variant module

The variant module contains data structures that are needed to manage variants both in packages and in specs.

spack.variant.BoolValuedVariant(name: str, value: bool, propagate: bool = False) → VariantValue[source]
class spack.variant.ConditionalValue(value: Any, when: Spec | None)[source]

Bases: object

Conditional value for a variant.

value: Any
when: Spec | None
class spack.variant.ConditionalVariantValues(iterable)[source]

Bases: TypedMutableSequence

A list, just with a different type

class spack.variant.DisjointSetsOfValues(*sets: Tuple[str, ...])[source]

Bases: Sequence

Allows combinations from one of many mutually exclusive sets.

The value ('none',) is reserved to denote the empty set and therefore no other set can contain the item 'none'.

Parameters:

*sets (list) – mutually exclusive sets of values

allow_empty_set()[source]

Adds the empty set to the current list of disjoint sets.

feature_values

Attribute used to track values which correspond to features which can be enabled or disabled as understood by the package’s build system.

prohibit_empty_set()[source]

Removes the empty set from the current list of disjoint sets.

property validator
with_default(default)[source]

Sets the default value and returns self.

with_error(error_fmt)[source]

Sets the error message format and returns self.

with_non_feature_values(*values)[source]

Marks a few values as not being tied to a feature.

exception spack.variant.DuplicateVariantError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when the same variant occurs in a spec twice.

exception spack.variant.InconsistentValidationError(vspec, variant)[source]

Bases: SpecError

Raised if the wrong validator is used to validate a variant.

exception spack.variant.InvalidVariantValueError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Raised when variants have invalid values.

spack.variant.MultiValuedVariant(name: str, value: Tuple[bool | str, ...], propagate: bool = False) → VariantValue[source]
exception spack.variant.MultipleValuesInExclusiveVariantError(variant: VariantValue, pkg_name: str | None = None)[source]

Bases: SpecError, ValueError

Raised when multiple values are present in a variant that wants only one.

spack.variant.RESERVED_NAMES

These are variant names used by Spack internally; packages can’t use them

spack.variant.SerializedValueType

Type of variant value when output for JSON, YAML, etc.

alias of str | bool | List[str | bool]

spack.variant.SingleValuedVariant(name: str, value: bool | str, propagate: bool = False) → VariantValue[source]
exception spack.variant.UnknownVariantError(msg: str, unknown_variants: List[str])[source]

Bases: SpecError

Raised when an unknown variant occurs in a spec.

exception spack.variant.UnsatisfiableVariantSpecError(provided, required)[source]

Bases: UnsatisfiableSpecError

Raised when a spec variant conflicts with package constraints.

spack.variant.ValueType

Type for value of a variant

alias of Tuple[bool | str, …]

class spack.variant.Variant(name: str, *, default: bool | str, description: str, values: Collection | Callable = (True, False), multi: bool = False, validator: Callable | None = None, sticky: bool = False, precedence: int = 0)[source]

Bases: object

Represents a variant definition, created by the variant() directive.

There can be multiple definitions of the same variant, and they are given precedence by order of appearance in the package. Later definitions have higher precedence. Similarly, definitions in derived classes have higher precedence than those in their superclasses.

property allowed_values

Returns a string representation of the allowed values for printing purposes

Returns:

representation of the allowed values

Return type:

str

default: bool | str
description: str
group_validator: Callable | None
make_default() → VariantValue[source]

Factory that creates a variant holding the default value(s).

make_variant(*value: str | bool) → VariantValue[source]

Factory that creates a variant holding the value(s) passed.

multi: bool
name: str
precedence: int
single_value_validator: Callable
sticky: bool
validate_or_raise(vspec: VariantValue, pkg_name: str)[source]

Validate a variant spec against this package variant. Raises an exception if any error is found.

Parameters:

  • vspec – variant spec to be validated

  • pkg_name – the name of the package class that required this validation (for errors)

Raises:
values: Collection | None

if None, valid values are defined only by validators

values_defined_by_validator() → bool[source]
property variant_type: VariantType

String representation of the type of this variant (single/multi/bool)

class spack.variant.VariantType(*values)[source]

Bases: IntEnum

Enum representing the three concrete variant types.

BOOL
INDICATOR
MULTI
SINGLE
property string: str

Convert the variant type to a string.

class spack.variant.VariantValue(type: VariantType, name: str, value: Tuple[bool | str, ...], *, propagate: bool = False, concrete: bool = False)[source]

Bases: object

A VariantValue is a key-value pair that represents a variant. It can have zero or more values. Values have set semantics, so they are unordered and unique. The variant type can be narrowed from multi to single to boolean, this limits the number of values that can be stored in the variant. Multi-valued variants can either be concrete or abstract: abstract means that the variant takes at least the values specified, but may take more when concretized. Concrete means that the variant takes exactly the values specified. Lastly, a variant can be marked as propagating, which means that it should be propagated to dependencies.

append(value: str | bool) → None[source]
concrete: bool
constrain(other: VariantValue) → bool[source]

Constrain self with other if they intersect. Returns true iff self was changed.

copy() → VariantValue[source]
static from_concretizer(name: str, value: str, type: str) → VariantValue[source]

Reconstruct a variant from concretizer output.

static from_node_dict(name: str, value: str | List[str], *, propagate: bool = False, abstract: bool = False) → VariantValue[source]

Reconstruct a variant from a node dict.

static from_string_or_bool(name: str, value: str | bool, *, propagate: bool = False, concrete: bool = False) → VariantValue[source]
intersects(other: VariantValue) → bool[source]

True iff there exists a concretization that satisfies both lhs and rhs.

name: str
propagate: bool
satisfies(other: VariantValue) → bool[source]

The lhs satisfies the rhs if all possible concretizations of lhs are also possible concretizations of rhs.

set(*value: bool | str) → None[source]

Set the value(s) of the variant.

slots
type: VariantType
property value: Tuple[bool | str, ...] | bool | str
property values: Tuple[bool | str, ...]
yaml_entry() → Tuple[str, str | bool | List[str | bool]][source]

Returns a (key, value) tuple suitable to be an entry in a yaml dict.

Returns:

(name, value_representation)

Return type:

tuple

class spack.variant.VariantValueRemoval(name)[source]

Bases: VariantValue

Indicator class for Spec.mutate to remove a variant

spack.variant.any_combination_of(*values: str) → DisjointSetsOfValues[source]

Multi-valued variant that allows either any combination of the specified values, or none at all (using variant=none). The literal value none is used as sentinel for the empty set, since in the spec DSL we have to always specify a value for a variant.

It is up to the package implementation to handle the value none specially, if at all.

See also auto_or_any_combination_of() and disjoint_sets().

Parameters:

*values – allowed variant values

Example:

variant("cuda_arch", values=any_combination_of("10", "11"))
Returns:

a properly initialized instance of DisjointSetsOfValues

spack.variant.auto_or_any_combination_of(*values: str) → DisjointSetsOfValues[source]

Multi-valued variant that allows any combination of a set of values (but not the empty set) or auto.

See also any_combination_of() and disjoint_sets().

Parameters:

*values – allowed variant values

Example:

variant(
    "file_systems",
    values=auto_or_any_combination_of("lustre", "gpfs", "nfs", "ufs"),
)
Returns:

a properly initialized instance of DisjointSetsOfValues

spack.variant.disjoint_sets(*sets: Tuple[str, ...]) → DisjointSetsOfValues[source]

Multi-valued variant that allows any combination picking from one of multiple disjoint sets of values, and also allows the user to specify none to choose none of them.

It is up to the package implementation to handle the value none specially, if at all.

See also any_combination_of() and auto_or_any_combination_of().

Parameters:

*sets – sets of allowed values, each set is a tuple of strings

Returns:

a properly initialized instance of DisjointSetsOfValues

spack.variant.prevalidate_variant_value(pkg_cls: Type[PackageBase], variant: VariantValue, spec: Spec | None = None, strict: bool = False) → List[Variant][source]

Do as much validation of a variant value as is possible before concretization.

This checks that the variant value is valid for some definition of the variant, and it raises if we know before concretization that the value cannot occur. On success it returns the variant definitions for which the variant is valid.

Parameters:

  • pkg_cls – package in which variant is (potentially multiply) defined

  • variant – variant spec with value to validate

  • spec – optionally restrict validation only to variants defined for this spec

  • strict – if True, raise an exception if no variant definition is valid for any constraint on the spec.

Returns:

list of variant definitions that will accept the given value. List will be empty only if the variant is a reserved variant.

spack.verify module

class spack.verify.VerificationResults[source]

Bases: object

add_error(path, field)[source]
has_errors()[source]
json_string()[source]
spack.verify.check_entry(path, data)[source]
spack.verify.check_file_manifest(filename)[source]
spack.verify.check_spec_manifest(spec)[source]
spack.verify.compute_hash(path: str, block_size: int = 1048576) → str[source]
spack.verify.create_manifest_entry(path: str) → Dict[str, Any][source]
spack.verify.write_manifest(spec)[source]

spack.verify_libraries module

spack.verify_libraries.ALLOW_UNRESOLVED

Patterns for names of libraries that are allowed to be unresolved when just looking at RPATHs added by Spack. These are libraries outside of Spack’s control, and assumed to be located in default search paths of the dynamic linker.

class spack.verify_libraries.Problem(resolved: Dict[bytes, bytes], unresolved: List[bytes], relative_rpaths: List[bytes])[source]

Bases: object

class spack.verify_libraries.ResolveSharedElfLibDepsVisitor(allow_unresolved_patterns: List[str])[source]

Bases: BaseDirectoryVisitor

allow_unresolved(needed: bytes) → bool[source]
before_visit_dir(root: str, rel_path: str, depth: int) → bool[source]

Return True from this function to recurse into the directory at os.path.join(root, rel_path). Return False in order not to recurse further.

Parameters:

  • root – root directory

  • rel_path – relative path to current directory from root

  • depth – depth of current directory from the root directory

Returns:

True when the directory should be recursed into. False when not

Return type:

bool

before_visit_symlinked_dir(root: str, rel_path: str, depth: int) → bool[source]

Return True to recurse into the symlinked directory and False in order not to. Note: rel_path is the path to the symlink itself. Following symlinked directories blindly can cause infinite recursion due to cycles.

Parameters:

  • root – root directory

  • rel_path – relative path to current symlink from root

  • depth – depth of current symlink from the root directory

Returns:

True when the directory should be recursed into. False when not

Return type:

bool

problems: Dict[str, Problem]
visit_file(root: str, rel_path: str, depth: int) → None[source]

Handle the non-symlink file at os.path.join(root, rel_path)

Parameters:

  • root – root directory

  • rel_path – relative path to current file from root

  • depth (int) – depth of current file from the root directory

visit_symlinked_file(root: str, rel_path: str, depth: int) → None[source]

Handle the symlink to a file at os.path.join(root, rel_path). Note: rel_path is the location of the symlink, not to what it is pointing to. The symlink may be dangling.

Parameters:

  • root – root directory

  • rel_path – relative path to current symlink from root

  • depth – depth of current symlink from the root directory

write(output: IO[str], *, indent=0, brief: bool = False) → None[source]
spack.verify_libraries.candidate_matches(current_elf: ElfFile, candidate_path: bytes) → bool[source]
spack.verify_libraries.is_compatible(parent: ElfFile, child: ElfFile) → bool[source]