Custom Extensions

Spack extensions allow you to extend Spack capabilities by deploying your own custom commands or logic in an arbitrary location on your filesystem. This might be extremely useful e.g. to develop and maintain a command whose purpose is too specific to be considered for reintegration into the mainline or to evolve a command through its early stages before starting a discussion to merge it upstream.

From Spack’s point of view an extension is any path in your filesystem which respects the following naming and layout for files:

spack-scripting/ # The top level directory must match the format 'spack-{extension_name}'
├── pytest.ini # Optional file if the extension ships its own tests
├── scripting # Folder that may contain modules that are needed for the extension commands
│   ├── cmd # Folder containing extension commands
│   │   └── filter.py # A new command that will be available
│   └── functions.py # Module with internal details
└── tests # Tests for this extension
│   ├── conftest.py
│   └── test_filter.py
└── templates # Templates that may be needed by the extension

In the example above, the extension is named scripting. It adds an additional command (spack filter) and unit tests to verify its behavior.

The extension can import any core Spack module in its implementation. When loaded by the spack command, the extension itself is imported as a Python package in the spack.extensions namespace. In the example above, since the extension is named “scripting”, the corresponding Python module is spack.extensions.scripting.

The code for this example extension can be obtained by cloning the corresponding git repository:

$ git -C /tmp clone https://github.com/spack/spack-scripting.git

Configure Spack to Use Extensions

To make your current Spack instance aware of extensions you should add their root paths to config.yaml. In the case of our example this means ensuring that:

config:
  extensions:
  - /tmp/spack-scripting

is part of your configuration file. Once this is setup any command that the extension provides will be available from the command line:

$ spack filter --help
usage: spack filter [-h] [--installed | --not-installed]
                    [--explicit | --implicit] [--output OUTPUT]
                    ...

filter specs based on their properties

positional arguments:
  specs            specs to be filtered

optional arguments:
  -h, --help       show this help message and exit
  --installed      select installed specs
  --not-installed  select specs that are not yet installed
  --explicit       select specs that were installed explicitly
  --implicit       select specs that are not installed or were installed implicitly
  --output OUTPUT  where to dump the result

The corresponding unit tests can be run giving the appropriate options to spack unit-test:

$ spack unit-test --extension=scripting
========================================== test session starts ===========================================
platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/culpo/github/spack-scripting
configfile: pytest.ini
testpaths: tests
plugins: xdist-3.5.0
collected 5 items

tests/test_filter.py .....                                                                         [100%]

========================================== slowest 30 durations ==========================================
2.31s setup    tests/test_filter.py::test_filtering_specs[kwargs0-specs0-expected0]
0.57s call     tests/test_filter.py::test_filtering_specs[kwargs2-specs2-expected2]
0.56s call     tests/test_filter.py::test_filtering_specs[kwargs4-specs4-expected4]
0.54s call     tests/test_filter.py::test_filtering_specs[kwargs3-specs3-expected3]
0.54s call     tests/test_filter.py::test_filtering_specs[kwargs1-specs1-expected1]
0.48s call     tests/test_filter.py::test_filtering_specs[kwargs0-specs0-expected0]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs4-specs4-expected4]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs2-specs2-expected2]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs1-specs1-expected1]
0.01s setup    tests/test_filter.py::test_filtering_specs[kwargs3-specs3-expected3]

(5 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================================== 5 passed in 5.06s ============================================

Registering Extensions via Entry Points

Note

Python version >= 3.8 is required to register extensions via entry points.

Spack can be made aware of extensions that are installed as part of a python package. To do so, register a function that returns the extension path, or paths, to the "spack.extensions" entry point. Consider the Python package my_package that includes a Spack extension:

my-package/
├── src
│   ├── my_package
│   │   └── __init__.py
│   └── spack-scripting/  # the spack extensions
└── pyproject.toml

adding the following to my_package’s pyproject.toml will make the spack-scripting extension visible to Spack when my_package is installed:

[project.entry_points."spack.extenions"]
my_package = "my_package:get_extension_path"

The function my_package.get_extension_path in my_package/__init__.py might look like

import importlib.resources

def get_extension_path():
    dirname = importlib.resources.files("my_package").joinpath("spack-scripting")
    if dirname.exists():
        return str(dirname)