Using Spack to Replace Homebrew/Conda

Spack is an incredibly powerful package manager, designed for supercomputers where users have diverse installation needs. But Spack can also be used to handle simple single-user installations on your laptop. Most macOS users are already familiar with package managers like Homebrew and Conda, where all installed packages are symlinked to a single central location like /usr/local. In this section, we will show you how to emulate the behavior of Homebrew/Conda using Environments (spack.yaml)!

Setup

First, let’s create a new environment. We’ll assume that Spack is already set up correctly, and that you’ve already sourced the setup script for your shell. To create a new environment, simply run:

$ spack env create myenv

Here, myenv can be anything you want to name your environment. Next, we can add a list of packages we would like to install into our environment. Let’s say we want a newer version of Bash than the one that comes with macOS, and we want a few Python libraries. We can run:

$ spack -e myenv add bash@5 python py-numpy py-scipy py-matplotlib

Each package can be listed on a separate line, or combined into a single line like we did above. Notice that we’re explicitly asking for Bash 5 here. You can use any spec you would normally use on the command line with other Spack commands.

Next, we want to manually configure a couple of things:

$ spack -e myenv config edit
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs: [bash@5, python, py-numpy, py-scipy, py-matplotlib]
  view: true

You can see the packages we added earlier in the specs: section. If you ever want to add more packages, you can either use spack add or manually edit this file.

We also need to change the concretizer:unify option. By default, Spack concretizes each spec separately, allowing multiple versions of the same package to coexist. Since we want a single consistent environment, we want to concretize all of the specs together.

Here is what your spack.yaml looks like with this new setting:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs: [bash@5, python, py-numpy, py-scipy, py-matplotlib]
  view: true
  concretizer:
    unify: true

Installation

To actually concretize the environment, run:

$ spack -e myenv concretize

This will tell you which if any packages are already installed, and alert you to any conflicting specs.

To actually install these packages and symlink them to your view: directory, simply run:

$ spack -e myenv install
$ spack env activate myenv

Now, when you type which python3, it should find the one you just installed.

In order to change the default shell to our newer Bash installation, we first need to add it to this list of acceptable shells. Run:

$ sudo vim /etc/shells

and add the absolute path to your bash executable. Then run:

$ chsh -s /path/to/bash

Now, when you log out and log back in, echo $SHELL should point to the newer version of Bash.

Updating Installed Packages

Let’s say you upgraded to a new version of macOS, or a new version of Python was released, and you want to rebuild your entire software stack. To do this, simply run the following commands:

$ spack env activate myenv
$ spack concretize --force
$ spack install

The --force flag tells Spack to overwrite its previous concretization decisions, allowing you to choose a new version of Python. If any of the new packages like Bash are already installed, spack install won’t re-install them, it will keep the symlinks in place.

Uninstallation

If you decide that Spack isn’t right for you, uninstallation is simple. Just run:

$ spack env activate myenv
$ spack uninstall --all

This will uninstall all packages in your environment and remove the symlinks.