Configuration Files

Spack has many configuration files. Here is a quick list of them, in case you want to skip directly to specific docs:

You can also add any of these as inline configuration in the YAML manifest file (spack.yaml) describing an environment.

YAML Format

Spack configuration files are written in YAML. We chose YAML because it’s human-readable but also versatile in that it supports dictionaries, lists, and nested sections. For more details on the format, see yaml.org. Here is an example config.yaml file:

config:
  install_tree:
    root: $spack/opt/spack
  build_stage:
  - $tempdir/$user/spack-stage
  - ~/.spack/stage

Each Spack configuration file is nested under a top-level section corresponding to its name. So, config.yaml starts with config:, mirrors.yaml starts with mirrors:, etc.

Tip

Validation and autocompletion of Spack config files can be enabled in your editor using JSON Schema Store.

Configuration Scopes

Spack pulls configuration data from files in several directories. There are multiple configuration scopes. From lowest to highest precedence:

  1. defaults: Stored in $(prefix)/etc/spack/defaults/. These are the “factory” settings. Users should generally not modify the settings here, but should override them in other configuration scopes. The defaults here will change from version to version of Spack.

  2. system: Stored in /etc/spack/. These are settings for this machine or for all machines on which this file system is mounted. The system scope overrides the defaults scope. It can be used for settings idiosyncratic to a particular machine, such as the locations of compilers or external packages. Be careful when modifying this scope, as changes here affect all Spack users on a machine. Before putting configuration here, instead consider using the site scope, which only affects the spack instance it’s part of.

  3. site: Stored in $(prefix)/etc/spack/site/. Settings here affect only this instance of Spack, and they override the defaults and system scopes. The site scope is intended for site-wide settings on multi-user machines (e.g., for a common Spack instance).

  4. plugin: Read from a Python package’s entry points. Settings here affect all instances of Spack running with the same Python installation. This scope takes higher precedence than site, system, and default scopes.

  5. user: Stored in the home directory: ~/.spack/. These settings affect all instances of Spack and take higher precedence than site, system, plugin, or defaults scopes.

  6. spack: Stored in $(prefix)/etc/spack/. Settings here affect only this instance of Spack, and they override user and lower configuration scopes. This is intended for project-specific or single-user spack installations. This is the topmost built-in spack scope, and modifying it gives you full control over configuration scopes. For example, it defines the user, site, and system scopes, so you can use it to remove them completely if you want.

  7. environment: When using Spack Environments (spack.yaml, spack.lock), Spack reads additional configuration from the environment file. See Configuring Environments for further details on these scopes. Environment scopes can be referenced from the command line as env:name (e.g., to reference environment foo, use env:foo).

  8. custom: Stored in a custom directory specified by --config-scope. If multiple scopes are listed on the command line, they are ordered from lowest to highest precedence.

  9. command line: Build settings specified on the command line take precedence over all other scopes.

Each configuration directory may contain several configuration files, such as config.yaml, packages.yaml, or mirrors.yaml. When configurations conflict, settings from higher-precedence scopes override lower-precedence settings.

All of these except spack and defaults are initially empty, so you don’t have to think about the others unless you need them. The most commonly used scopes are environment, user, and spack. If you forget, you can always see the available configuration scopes in order of precedence with the spack config scopes command:

> spack config scopes -p
Scope            Path
command_line
spack            /home/username/spack/etc/spack
user             /home/username/.spack/
site             /home/username/spack/etc/spack/site/
defaults         /home/username/spack/etc/spack/defaults/
defaults:darwin  /home/username/spack/etc/spack/defaults/darwin/
defaults:base    /home/username/spack/etc/spack/defaults/base/
_builtin

Commands that modify scopes (e.g., spack compilers, spack repo, spack external find, etc.) take a --scope=<name> parameter that you can use to control which scope is modified. By default, they modify the highest-precedence available scope that is not read-only (like defaults).

Custom scopes

You may add configuration scopes directly on the command line with the --config-scope argument, or -C for short. Custom command-line scopes override any active environments, as well as the defaults, system, site, user, and spack scopes,

For example, the following adds two configuration scopes, named scope-a and scope-b, to a spack spec command:

$ spack -C ~/myscopes/scope-a -C ~/myscopes/scope-b spec ncurses

Custom scopes come after the spack command and before the subcommand, and they specify a single path to a directory containing configuration files. You can add the same configuration files to that directory that you can add to any other scope (e.g., config.yaml, packages.yaml, etc.).

If multiple scopes are provided:

  1. Each must be preceded with the --config-scope or -C flag.

  2. They must be ordered from lowest to highest precedence.

Example: scopes for release and development

Suppose that you need to support simultaneous building of release and development versions of mypackage, where mypackage depends on pkg-a, which in turn depends on pkg-b. You could create the following files:

~/myscopes/release/packages.yaml
packages:
  mypackage:
    prefer: ["@1.7"]
  pkg-a:
    prefer: ["@2.3"]
  pkg-b:
    prefer: ["@0.8"]
~/myscopes/develop/packages.yaml
packages:
  mypackage:
    prefer: ["@develop"]
  pkg-a:
    prefer: ["@develop"]
  pkg-b:
    prefer: ["@develop"]

You can switch between release and develop configurations using configuration arguments. You would type spack -C ~/myscopes/release when you want to build the designated release versions of mypackage, pkg-a, and pkg-b, and you would type spack -C ~/myscopes/develop when you want to build all of these packages at the develop version.

Example: swapping MPI providers

Suppose that you need to build two software packages, pkg-a and pkg-b. For pkg-b you want a newer Python version and a different MPI implementation than for pkg-a. You can create different configuration scopes for use with pkg-a and pkg-b:

~/myscopes/pkg-a/packages.yaml
packages:
  python:
    require: ["@3.11"]
  mpi:
    require: [openmpi]
~/myscopes/pkg-b/packages.yaml
packages:
  python:
    require: ["@3.13"]
  mpi:
    require: [mpich]

Plugin scopes

Note

Python version >= 3.8 is required to enable plugin configuration.

Spack can be made aware of configuration scopes that are installed as part of a Python package. To do so, register a function that returns the scope’s path to the "spack.config" entry point. Consider the Python package my_package that includes Spack configurations:

my-package/
├── src
│   ├── my_package
│   │   ├── __init__.py
│   │   └── spack/
│   │   │   └── config.yaml
└── pyproject.toml

Adding the following to my_package’s pyproject.toml will make my_package’s spack/ configurations visible to Spack when my_package is installed:

[project.entry_points."spack.config"]
my_package = "my_package:get_config_path"

The function my_package.get_config_path (matching the entry point definition) in my_package/__init__.py might look like:

import importlib.resources


def get_config_path():
    dirname = importlib.resources.files("my_package").joinpath("spack")
    if dirname.exists():
        return str(dirname)

Platform-specific Configuration

Warning

Prior to v1.0, each scope above – except environment scopes – had a corresponding platform-specific scope (e.g., defaults/linux, system/windows). This can now be accomplished through a suitably placed include.yaml file.

There is often a need for platform-specific configuration settings. For example, on most platforms, GCC is the preferred compiler. However, on macOS (darwin), Clang often works for more packages, and is set as the default compiler. This configuration is set in $(prefix)/etc/spack/defaults/darwin/packages.yaml, which is included by $(prefix)/etc/spack/defaults/include.yaml. Since it is an included configuration of the defaults scope, settings in the defaults scope will take precedence.

For example, if $(prefix)/etc/spack/defaults/include.yaml contains:

include:
- path: "${platform}"
  optional: true
- path: base

then, on macOS (darwin), configuration settings for files under the $(prefix)/etc/spack/defaults/darwin directory would be picked up if they are present. Because ${platform} is above the base include in the list, ${platform} settings will override anything in base if there are conflicts.

Note

You can get the name to use for <platform> by running spack arch --platform.

Platform-specific configuration files can similarly be set up for any other scope by creating an include.yaml similar to the one above for defaults – under the appropriate configuration paths (see Overriding entire sections) and creating a subdirectory with the platform name that contains the configurations.

Scope Precedence

When Spack queries for configuration parameters, it searches in higher-precedence scopes first. So, settings in a higher-precedence file can override those with the same key in a lower-precedence one. For list-valued settings, Spack merges lists by prepending items from higher-precedence configurations to items from lower-precedence configurations by default. Completely ignoring lower-precedence configuration options is supported with the :: notation for keys (see Overriding entire sections below).

Note

Settings in a scope take precedence over those provided in any included configuration files (i.e., files listed in include.yaml or an include: section in spack.yaml).

There are also special notations for string concatenation and precedence override:

  • +: will force prepending strings or lists. For lists, this is the default behavior.

  • -: works similarly, but for appending values.

See String Concatenation for more details.

Simple keys

Let’s look at an example of overriding a single key in a Spack configuration file. If your configurations look like this:

$(prefix)/etc/spack/defaults/config.yaml
config:
  install_tree:
    root: $spack/opt/spack
  build_stage:
  - $tempdir/$user/spack-stage
  - ~/.spack/stage
~/.spack/config.yaml
config:
  install_tree:
    root: /some/other/directory

Spack will only override install_tree in the config section, and will take the site preferences for other settings. You can see the final, combined configuration with the spack config get <configtype> command:

$ spack config get config
config:
  install_tree:
    root: /some/other/directory
  build_stage:
  - $tempdir/$user/spack-stage
  - ~/.spack/stage

String Concatenation

Above, the user config.yaml completely overrides specific settings in the default config.yaml. Sometimes, it is useful to add a suffix/prefix to a path or name. To do this, you can use the -: notation for append string concatenation at the end of a key in a configuration file. For example:

~/.spack/config.yaml
config:
  install_tree:
    root-: /my/custom/suffix/

Spack will then append to the lower-precedence configuration under the root key:

$ spack config get config
config:
  install_tree:
    root: /some/other/directory/my/custom/suffix
  build_stage:
  - $tempdir/$user/spack-stage
  - ~/.spack/stage

Similarly, +: can be used to prepend to a path or name:

~/.spack/config.yaml
config:
  install_tree:
    root+: /my/custom/suffix/

Overriding entire sections

Above, the user config.yaml only overrides specific settings in the default config.yaml. Sometimes, it is useful to completely override lower-precedence settings. To do this, you can use two colons at the end of a key in a configuration file. For example:

~/.spack/config.yaml
config::
  install_tree:
    root: /some/other/directory

Spack will ignore all lower-precedence configuration under the config:: section:

$ spack config get config
config:
  install_tree:
    root: /some/other/directory

List-valued settings

Let’s revisit the config.yaml example one more time. The build_stage setting’s value is an ordered list of directories:

$(prefix)/etc/spack/defaults/config.yaml
config:
  build_stage:
  - $tempdir/$user/spack-stage
  - ~/.spack/stage

Suppose the user configuration adds its own list of build_stage paths:

~/.spack/config.yaml
config:
  build_stage:
  - /lustre-scratch/$user/spack
  - ~/mystage

Spack will first look at the paths in the defaults config.yaml, then the paths in the user’s ~/.spack/config.yaml. The list in the higher-precedence scope is prepended to the defaults. spack config get config shows the result:

$ spack config get config
config:
  install_tree:
    root: /some/other/directory
  build_stage:
  - /lustre-scratch/$user/spack
  - ~/mystage
  - $tempdir/$user/spack-stage
  - ~/.spack/stage

As in Overriding entire sections, the higher-precedence scope can completely override the lower-precedence scope using ::. So if the user config looked like this:

~/.spack/config.yaml
config:
  build_stage::
  - /lustre-scratch/$user/spack
  - ~/mystage

The merged configuration would look like this:

$ spack config get config
config:
  install_tree:
    root: /some/other/directory
  build_stage:
    - /lustre-scratch/$user/spack
    - ~/mystage

Config File Variables

Spack understands several variables which can be used in config file paths wherever they appear. There are three sets of these variables: Spack-specific variables, environment variables, and user path variables. Spack-specific variables and environment variables are both indicated by prefixing the variable name with $. User path variables are indicated at the start of the path with ~ or ~user.

Spack-specific variables

Spack understands over a dozen special variables. These are:

  • $env: name of the currently active environment

  • $spack: path to the prefix of this Spack installation

  • $tempdir: default system temporary directory (as specified in Python’s tempfile.tempdir variable.

  • $user: name of the current user

  • $user_cache_path: user cache directory (~/.spack unless overridden)

  • $architecture: the architecture triple of the current host, as detected by Spack.

  • $arch: alias for $architecture.

  • $platform: the platform of the current host, as detected by Spack.

  • $operating_system: the operating system of the current host, as detected by the distro Python module.

  • $os: alias for $operating_system.

  • $target: the ISA target for the current host, as detected by ArchSpec. E.g. skylake or neoverse-n1.

  • $target_family. The target family for the current host, as detected by ArchSpec. E.g. x86_64 or aarch64.

  • $date: the current date in the format YYYY-MM-DD

  • $spack_short_version: the Spack version truncated to the first components.

Note that, as with shell variables, you can write these as $varname or with braces to distinguish the variable from surrounding characters: ${varname}. Their names are also case insensitive, meaning that $SPACK works just as well as $spack. These special variables are substituted first, so any environment variables with the same name will not be used.

Environment variables

After Spack-specific variables are evaluated, environment variables are expanded. These are formatted like Spack-specific variables, e.g., ${varname}. You can use this to insert environment variables in your Spack configuration.

User home directories

Spack performs Unix-style tilde expansion on paths in configuration files. This means that tilde (~) will expand to the current user’s home directory, and ~user will expand to a specified user’s home directory. The ~ must appear at the beginning of the path, or Spack will not expand it.

Environment Modifications

Spack allows users to prescribe custom environment modifications in a few places within its configuration files. Every time these modifications are allowed, they are specified as a dictionary, like in the following example:

environment:
  set:
    LICENSE_FILE: "/path/to/license"
  unset:
  - CPATH
  - LIBRARY_PATH
  append_path:
    PATH: "/new/bin/dir"

The possible actions that are permitted are set, unset, append_path, prepend_path, and finally remove_path. They all require a dictionary of variable names mapped to the values used for the modification, with the exception of unset, which requires just a list of variable names. No particular order is ensured for the execution of each of these modifications.

Seeing Spack’s Configuration

With so many scopes overriding each other, it can sometimes be difficult to understand what Spack’s final configuration looks like.

Spack provides two useful ways to view the final “merged” version of any configuration file: spack config get and spack config blame.

spack config get

spack config get shows a fully merged configuration file, taking into account all scopes. For example, to see the fully merged config.yaml, you can type:

$ spack config get config
config:
  debug: false
  checksum: true
  verify_ssl: true
  dirty: false
  build_jobs: 8
  install_tree:
    root: $spack/opt/spack
  template_dirs:
  - $spack/templates
  directory_layout: {architecture}/{compiler.name}-{compiler.version}/{name}-{version}-{hash}
  build_stage:
  - $tempdir/$user/spack-stage
  - ~/.spack/stage
  - $spack/var/spack/stage
  source_cache: $spack/var/spack/cache
  misc_cache: ~/.spack/cache
  locks: true

Likewise, this will show the fully merged packages.yaml:

$ spack config get packages

You can use this in conjunction with the -C / --config-scope argument to see how your scope will affect Spack’s configuration:

$ spack -C /path/to/my/scope config get packages

spack config blame

spack config blame functions much like spack config get, but it shows exactly which configuration file each setting came from. If you do not know why Spack is behaving a certain way, this command can help you track down the source of the configuration:

$ spack --insecure -C ./my-scope -C ./my-scope-2 config blame config
==> Warning: You asked for --insecure. Will NOT check SSL certificates.
---                                                   config:
_builtin                                                debug: False
/home/myuser/spack/etc/spack/defaults/config.yaml:72    checksum: True
command_line                                            verify_ssl: False
./my-scope-2/config.yaml:2                              dirty: False
_builtin                                                build_jobs: 8
./my-scope/config.yaml:2                                install_tree: /path/to/some/tree
/home/myuser/spack/etc/spack/defaults/config.yaml:23    template_dirs:
/home/myuser/spack/etc/spack/defaults/config.yaml:24    - $spack/templates
/home/myuser/spack/etc/spack/defaults/config.yaml:28    directory_layout: {architecture}/{compiler.name}-{compiler.version}/{name}-{version}-{hash}
/home/myuser/spack/etc/spack/defaults/config.yaml:49    build_stage:
/home/myuser/spack/etc/spack/defaults/config.yaml:50    - $tempdir/$user/spack-stage
/home/myuser/spack/etc/spack/defaults/config.yaml:51    - ~/.spack/stage
/home/myuser/spack/etc/spack/defaults/config.yaml:52    - $spack/var/spack/stage
/home/myuser/spack/etc/spack/defaults/config.yaml:57    source_cache: $spack/var/spack/cache
/home/myuser/spack/etc/spack/defaults/config.yaml:62    misc_cache: ~/.spack/cache
/home/myuser/spack/etc/spack/defaults/config.yaml:86    locks: True

You can see above that the build_jobs and debug settings are built-in and are not overridden by a configuration file. The verify_ssl setting comes from the --insecure option on the command line. The dirty and install_tree settings come from the custom scopes ./my-scope and ./my-scope-2, and all other configuration options come from the default configuration files that ship with Spack.

Overriding Local Configuration

Spack’s system and user scopes provide ways for administrators and users to set global defaults for all Spack instances, but for use cases where one wants a clean Spack installation, these scopes can be undesirable. For example, users may want to opt out of global system configuration, or they may want to ignore their own home directory settings when running in a continuous integration environment.

Spack also, by default, keeps various caches and user data in ~/.spack, but users may want to override these locations.

Spack provides three environment variables that allow you to override or opt out of configuration locations:

  • SPACK_USER_CONFIG_PATH: Override the path to use for the user scope (~/.spack by default).

  • SPACK_SYSTEM_CONFIG_PATH: Override the path to use for the system scope (/etc/spack by default).

  • SPACK_DISABLE_LOCAL_CONFIG: Set this environment variable to completely disable both the system and user configuration directories. Spack will then only consider its own defaults and site configuration locations.

And one that allows you to move the default cache location:

  • SPACK_USER_CACHE_PATH: Override the default path to use for user data (misc_cache, tests, reports, etc.)

With these settings, if you want to isolate Spack in a CI environment, you can do this:

$ export SPACK_DISABLE_LOCAL_CONFIG=true
$ export SPACK_USER_CACHE_PATH=/tmp/spack