Custom Extensions

Spack extensions allow you to add custom subcommands to the spack command. This is extremely useful when developing and maintaining a command whose purpose is too specific to be included in the Spack codebase. It’s also useful for evolving a command through its early stages before starting a discussion to merge it upstream.

From Spack’s point of view, an extension is any path in your filesystem that respects the following naming and layout for files:

spack-scripting/ # The top level directory must match the format 'spack-{extension_name}'
├── pytest.ini # Optional file if the extension ships its own tests
├── scripting # Folder that may contain modules that are needed for the extension commands
│   ├── cmd # Folder containing extension commands
│   │   └── filter.py # A new command that will be available
│   └── functions.py # Module with internal details
├── tests # Tests for this extension
│   ├── conftest.py
│   └── test_filter.py
└── templates # Templates that may be needed by the extension

In the example above, the extension is named scripting. It adds an additional command (spack filter) and unit tests to verify its behavior.

The extension can import any core Spack module in its implementation. When loaded by the spack command, the extension itself is imported as a Python package in the spack.extensions namespace. In the example above, since the extension is named “scripting”, the corresponding Python module is spack.extensions.scripting.

The code for this example extension can be obtained by cloning the corresponding git repository:

$ git -C /tmp clone https://github.com/spack/spack-scripting.git

Configure Spack to Use Extensions

To make your current Spack instance aware of extensions you should add their root paths to config.yaml. In the case of our example, this means ensuring that:

config:
  extensions:
  - /tmp/spack-scripting

is part of your configuration file. Once this is set up, any command that the extension provides will be available from the command line:

$ spack filter --help
usage: spack filter [-h] [--installed | --not-installed]
                    [--explicit | --implicit] [--output OUTPUT]
                    ...

filter specs based on their properties

positional arguments:
  specs            specs to be filtered

optional arguments:
  -h, --help       show this help message and exit
  --installed      select installed specs
  --not-installed  select specs that are not yet installed
  --explicit       select specs that were installed explicitly
  --implicit       select specs that are not installed or were installed implicitly
  --output OUTPUT  where to dump the result

The corresponding unit tests can be run giving the appropriate options to spack unit-test:

$ spack unit-test --extension=scripting
========================================== test session starts ===========================================
platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/culpo/github/spack-scripting
configfile: pytest.ini
testpaths: tests
plugins: xdist-3.5.0
collected 5 items

tests/test_filter.py .....                                                                         [100%]

========================================== slowest 30 durations ==========================================
2.31s setup    tests/test_filter.py::test_filtering_specs[kwargs0-specs0-expected0]
0.57s call     tests/test_filter.py::test_filtering_specs[kwargs2-specs2-expected2]
0.56s call     tests/test_filter.py::test_filtering_specs[kwargs4-specs4-expected4]
0.54s call     tests/test_filter.py::test_filtering_specs[kwargs3-specs3-expected3]
0.54s call     tests/test_filter.py::test_filtering_specs[kwargs1-specs1-expected1]
0.48s call     tests/test_filter.py::test_filtering_specs[kwargs0-specs0-expected0]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs4-specs4-expected4]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs2-specs2-expected2]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs1-specs1-expected1]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs3-specs3-expected3]

(5 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================================== 5 passed in 5.06s ============================================

Registering Extensions via Entry Points

Note

Python version >= 3.8 is required to register extensions via entry points.

Spack can be made aware of extensions that are installed as part of a Python package. To do so, register a function that returns the extension path, or paths, to the "spack.extensions" entry point. Consider the Python package my_package that includes a Spack extension:

my-package/
├── src
│   ├── my_package
│   │   └── __init__.py
│   └── spack-scripting/  # the spack extensions
└── pyproject.toml

adding the following to my_package’s pyproject.toml will make the spack-scripting extension visible to Spack when my_package is installed:

[project.entry_points."spack.extensions"]
my_package = "my_package:get_extension_path"

The function my_package.get_extension_path in my_package/__init__.py might look like

import importlib.resources


def get_extension_path():
    dirname = importlib.resources.files("my_package").joinpath("spack-scripting")
    if dirname.exists():
        return str(dirname)