Spack Package Build Systems

You may begin to notice after writing a couple of package template files a pattern emerge for some packages. For example, you may find yourself writing an install() method that invokes: configure, cmake, make, make install. You may also find yourself writing "prefix=" + prefix as an argument to configure or cmake. Rather than having you repeat these lines for all packages, Spack has classes that can take care of these patterns. In addition, these package files allow for finer grained control of these build systems. In this section, we will describe each build system and give examples on how these can be manipulated to install a package.

Package Class Hierarchy

digraph G { node [ shape = "record" ] edge [ arrowhead = "empty" ] PackageBase -> Package [dir=back] PackageBase -> MakefilePackage [dir=back] PackageBase -> AutotoolsPackage [dir=back] PackageBase -> CMakePackage [dir=back] PackageBase -> PythonPackage [dir=back] }

The above diagram gives a high level view of the class hierarchy and how each package relates. Each subclass inherits from the PackageBaseClass super class. The bulk of the work is done in this super class which includes fetching, extracting to a staging directory and installing. Each subclass then adds additional build-system-specific functionality. In the following sections, we will go over examples of how to utilize each subclass and to see how powerful these abstractions are when packaging.

Package

We’ve already seen examples of a Package class in our walkthrough for writing package files, so we won’t be spending much time with them here. Briefly, the Package class allows for abitrary control over the build process, whereas subclasses rely on certain patterns (e.g. configure make make install) to be useful. Package classes are particularly useful for packages that have a non-conventional way of being built since the packager can utilize some of Spack’s helper functions to customize the building and installing of a package.

Autotools

As we have seen earlier, packages using Autotools use configure, make and make install commands to execute the build and install process. In our Package class, your typical build incantation will consist of the following:

def install(self, spec, prefix):
    configure("--prefix=" + prefix)
    make()
    make("install")

You’ll see that this looks similar to what we wrote in our packaging tutorial.

The Autotools subclass aims to simplify writing package files and provides convenience methods to manipulate each of the different phases for a Autotools build system.

Autotools packages consist of four phases:

  1. autoreconf()
  2. configure()
  3. build()
  4. install()

Each of these phases have sensible defaults. Let’s take a quick look at some the internals of the Autotools class:

$ spack edit --build-system autotools

This will open the AutotoolsPackage file in your text editor.

Note

The examples showing code for these classes is abridged to avoid having long examples. We only show what is relevant to the packager.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

    They all have sensible defaults and for many packages the only thing
    necessary will be to override the helper method
    :py:meth:`~.AutotoolsPackage.configure_args`.
    For a finer tuning you may also override:

        +-----------------------------------------------+--------------------+
        | **Method**                                    | **Purpose**        |
        +===============================================+====================+
        | :py:attr:`~.AutotoolsPackage.build_targets`   | Specify ``make``   |
        |                                               | targets for the    |
        |                                               | build phase        |
        +-----------------------------------------------+--------------------+
        | :py:attr:`~.AutotoolsPackage.install_targets` | Specify ``make``   |
        |                                               | targets for the    |
        |                                               | install phase      |
        +-----------------------------------------------+--------------------+
        | :py:meth:`~.AutotoolsPackage.check`           | Run  build time    |
        |                                               | tests if required  |
        +-----------------------------------------------+--------------------+

    """
    #: Phases of a GNU Autotools package
    phases = ['autoreconf', 'configure', 'build', 'install']
    #: This attribute is used in UI queries that need to know the build
    #: system base class
    build_system_class = 'AutotoolsPackage'
    #: Whether or not to update ``config.guess`` on old architectures
    patch_config_guess = True

    #: Targets for ``make`` during the :py:meth:`~.AutotoolsPackage.build`
    #: phase
    build_targets = []
    #: Targets for ``make`` during the :py:meth:`~.AutotoolsPackage.install`
    #: phase
    install_targets = ['install']

    #: Callback names for build-time test
    build_time_test_callbacks = ['check']

    #: Callback names for install-time test
    install_time_test_callbacks = ['installcheck']

    #: Set to true to force the autoreconf step even if configure is present
    force_autoreconf = False
    #: Options to be passed to autoreconf when using the default implementation
    autoreconf_extra_args = []
        setattr(self, 'configure_flag_args', [])
        for flag, values in flags.items():
            if values:
                values_str = '{0}={1}'.format(flag.upper(), ' '.join(values))
                self.configure_flag_args.append(values_str)

    def configure(self, spec, prefix):
        """Runs configure with the arguments specified in
        :py:meth:`~.AutotoolsPackage.configure_args`

Important to note are the highlighted lines. These properties allow the packager to set what build targets and install targets they want for their package. If, for example, we wanted to add as our build target foo then we can append to our build_targets property:

build_targets = ["foo"]

Which is similiar to invoking make in our Package

make("foo")

This is useful if we have packages that ignore environment variables and need a command-line argument.

Another thing to take note of is in the configure() method. Here we see that the prefix argument is already included since it is a common pattern amongst packages using Autotools. We then only have to override configure_args(), which will then return it’s output to to configure(). Then, configure() will append the common arguments

Packagers also have the option to run autoreconf in case a package needs to update the build system and generate a new configure. Though, for the most part this will be unnecessary.

Let’s look at the mpileaks package.py file that we worked on earlier:

$ spack edit mpileaks

Notice that mpileaks is a Package class but uses the Autotools build system. Although this package is acceptable let’s make this into an AutotoolsPackage class and simplify it further.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Mpileaks(AutotoolsPackage):
    """Tool to detect and report leaked MPI objects like MPI_Requests and
       MPI_Datatypes."""

    homepage = "https://github.com/hpc/mpileaks"
    url      = "https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz"

    version('1.0', '8838c574b39202a57d7c2d68692718aa')

    depends_on("mpi")
    depends_on("adept-utils")
    depends_on("callpath")

    def install(self, spec, prefix):
        configure("--prefix=" + prefix,
                  "--with-adept-utils=" + spec['adept-utils'].prefix,
                  "--with-callpath=" + spec['callpath'].prefix)
        make()
        make("install")

We first inherit from the AutotoolsPackage class.

Although we could keep the install() method, most of it can be handled by the AutotoolsPackage base class. In fact, the only thing that needs to be overridden is configure_args().

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Mpileaks(AutotoolsPackage):
    """Tool to detect and report leaked MPI objects like MPI_Requests and
       MPI_Datatypes."""

    homepage = "https://github.com/hpc/mpileaks"
    url      = "https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz"

    version('1.0', '8838c574b39202a57d7c2d68692718aa')

    variant("stackstart", values=int, default=0,
            description="Specify the number of stack frames to truncate")

    depends_on("mpi")
    depends_on("adept-utils")
    depends_on("callpath")

    def configure_args(self):
        stackstart = int(self.spec.variants['stackstart'].value)
        args = ["--with-adept-utils=" + spec['adept-utils'].prefix,
                "--with-callpath=" + spec['callpath'].prefix]
        if stackstart:
            args.extend(['--with-stack-start-c=%s' % stackstart,
                         '--with-stack-start-fortran=%s' % stackstart])
        return args

Since Spack takes care of setting the prefix for us we can exclude that as an argument to configure. Our packages look simpler, and the packager does not need to worry about whether they have properly included configure and make.

This version of the mpileaks package installs the same as the previous, but the AutotoolsPackage class lets us do it with a cleaner looking package file.

Makefile

Packages that utilize Make or a Makefile usually require you to edit a Makefile to set up platform and compiler specific variables. These packages are handled by the Makefile subclass which provides convenience methods to help write these types of packages.

A MakefilePackage class has three phases that can be overridden. These include:

  1. edit()
  2. build()
  3. install()

Packagers then have the ability to control how a Makefile is edited, and what targets to include for the build phase or install phase.

Let’s also take a look inside the MakefilePackage class:

$ spack edit --build-system makefile

Take note of the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
class MakefilePackage(PackageBase):
    #: Phases of a package that is built with an hand-written Makefile
    phases = ['edit', 'build', 'install']
    #: This attribute is used in UI queries that need to know the build
    #: system base class
    build_system_class = 'MakefilePackage'

    #: Targets for ``make`` during the :py:meth:`~.MakefilePackage.build`
    #: phase
    build_targets = []
    #: Targets for ``make`` during the :py:meth:`~.MakefilePackage.install`
    #: phase
    install_targets = ['install']

    #: Callback names for build-time test
    build_time_test_callbacks = ['check']

    #: Callback names for install-time test
    install_time_test_callbacks = ['installcheck']

    def edit(self, spec, prefix):
        """Edits the Makefile before calling make. This phase cannot
        be defaulted.
        """
        tty.msg('Using default implementation: skipping edit phase.')

    def build(self, spec, prefix):
        """Calls make, passing :py:attr:`~.MakefilePackage.build_targets`
        as targets.
        """
        with working_dir(self.build_directory):
            inspect.getmodule(self).make(*self.build_targets)

    def install(self, spec, prefix):
        """Calls make, passing :py:attr:`~.MakefilePackage.install_targets`
        as targets.
        """
        with working_dir(self.build_directory):
            inspect.getmodule(self).make(*self.install_targets)

Similar to Autotools, MakefilePackage class has properties that can be set by the packager. We can also override the different methods highlighted.

Let’s try to recreate the Bowtie package:

$ spack create -f https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip
==> This looks like a URL for bowtie
==> Found 1 version of bowtie:

1.2.1.1  https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip

==> How many would you like to checksum? (default is 1, q to abort) 1
==> Downloading...
==> Fetching https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip
######################################################################## 100.0%
==> Checksummed 1 version of bowtie
==> This package looks like it uses the makefile build system
==> Created template for bowtie package
==> Created package file: /Users/mamelara/spack/var/spack/repos/builtin/packages/bowtie/package.py

Once the fetching is completed, Spack will open up your text editor in the usual fashion and create a template of a MakefilePackage package.py.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Bowtie(MakefilePackage):
    """FIXME: Put a proper description of your package here."""

    # FIXME: Add a proper url for your package's homepage here.
    homepage = "http://www.example.com"
    url      = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip"

    version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf')

    # FIXME: Add dependencies if required.
    # depends_on('foo')

    def edit(self, spec, prefix):
        # FIXME: Edit the Makefile if necessary
        # FIXME: If not needed delete this function
        # makefile = FileFilter('Makefile')
        # makefile.filter('CC = .*', 'CC = cc')
        return

Spack was successfully able to detect that Bowtie uses Make. Let’s add in the rest of our details for our package:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Bowtie(MakefilePackage):
    """Bowtie is an ultrafast, memory efficient short read aligner
    for short DNA sequences (reads) from next-gen sequencers."""

    homepage = "https://sourceforge.net/projects/bowtie-bio/"
    url      = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip"

    version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf')

    variant("tbb", default=False, description="Use Intel thread building block")

    depends_on("tbb", when="+tbb")

    def edit(self, spec, prefix):
        # FIXME: Edit the Makefile if necessary
        # FIXME: If not needed delete this function
        # makefile = FileFilter('Makefile')
        # makefile.filter('CC = .*', 'CC = cc')
        return

As we mentioned earlier, most packages using a Makefile have hard-coded variables that must be edited. These variables are fine if you happen to not care about setup or types of compilers used but Spack is designed to work with any compiler. The MakefilePackage subclass makes it easy to edit these Makefiles by having an edit() method that can be overridden.

Let’s take a look at the default Makefile that Bowtie provides. If we look inside, we see that CC and CXX point to our GNU compiler:

$ spack stage bowtie

Note

As usual make sure you have shell support activated with spack:
source /path/to/spack_root/spack/share/spack/setup-env.sh
$ spack cd -s bowtie
$ cd bowtie-1.2
$ vim Makefile
CPP = g++ -w
CXX = $(CPP)
CC = gcc
LIBS = $(LDFLAGS) -lz
HEADERS = $(wildcard *.h)

To fix this, we need to use the edit() method to write our custom Makefile.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Bowtie(MakefilePackage):
    """Bowtie is an ultrafast, memory efficient short read aligner
    for short DNA sequences (reads) from next-gen sequencers."""

    homepage = "https://sourceforge.net/projects/bowtie-bio/"
    url      = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip"

    version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf')

    variant("tbb", default=False, description="Use Intel thread building block")

    depends_on("tbb", when="+tbb")

    def edit(self, spec, prefix):
        makefile = FileFilter("Makefile")
        makefile.filter('CC= .*', 'CC = ' + env['CC'])
        makefile.filter('CXX = .*', 'CXX = ' + env['CXX'])

Here we use a FileFilter object to edit our Makefile. It takes in a regular expression and then replaces CC and CXX to whatever Spack sets CC and CXX environment variables to. This allows us to build Bowtie with whatever compiler we specify through Spack’s spec syntax.

Let’s change the build and install phases of our package:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Bowtie(MakefilePackage):
    """Bowtie is an ultrafast, memory efficient short read aligner
    for short DNA sequences (reads) from next-gen sequencers."""

    homepage = "https://sourceforge.net/projects/bowtie-bio/"
    url      = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip"

    version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf')

    variant("tbb", default=False, description="Use Intel thread building block")

    depends_on("tbb", when="+tbb")

    def edit(self, spec, prefix):
        makefile = FileFilter("Makefile")
        makefile.filter('CC= .*', 'CC = ' + env['CC'])
        makefile.filter('CXX = .*', 'CXX = ' + env['CXX'])

    @property
    def build_targets(self):
        if "+tbb" in spec:
            return []
        else:
            return ["NO_TBB=1"]

    @property
    def install_targets(self):
        return ['prefix={0}'.format(self.prefix), 'install']

Here demonstrate another strategy that we can use to manipulate our package We can provide command-line arguments to make(). Since Bowtie can use tbb we can either add NO_TBB=1 as a argument to prevent tbb support or we can just invoke make with no arguments.

Bowtie requires our install_target to provide a path to the install directory. We can do this by providing prefix= as a command line argument to make().

Let’s look at a couple of other examples and go through them:

$ spack edit esmf

Some packages allow environment variables to be set and will honor them. Packages that use ?= for assignment in their Makefile can be set using environment variables. In our esmf example we set two environment variables in our edit() method:

def edit(self, spec, prefix):
    for var in os.environ:
        if var.startswith('ESMF_'):
            os.environ.pop(var)

    # More code ...

    if self.compiler.name == 'gcc':
        os.environ['ESMF_COMPILER'] = 'gfortran'
    elif self.compiler.name == 'intel':
        os.environ['ESMF_COMPILER'] = 'intel'
    elif self.compiler.name == 'clang':
        os.environ['ESMF_COMPILER'] = 'gfortranclang'
    elif self.compiler.name == 'nag':
        os.environ['ESMF_COMPILER'] = 'nag'
    elif self.compiler.name == 'pgi':
        os.environ['ESMF_COMPILER'] = 'pgi'
    else:
        msg  = "The compiler you are building with, "
        msg += "'{0}', is not supported by ESMF."
        raise InstallError(msg.format(self.compiler.name))

As you may have noticed, we didn’t really write anything to the Makefile but rather we set environment variables that will override variables set in the Makefile.

Some packages include a configuration file that sets certain compiler variables, platform specific variables, and the location of dependencies or libraries. If the file is simple and only requires a couple of changes, we can overwrite those entries with our FileFilter object. If the configuration involves complex changes, we can write a new configuration file from scratch.

Let’s look at an example of this in the elk package:

$ spack edit elk
def edit(self, spec, prefix):
    # Dictionary of configuration options
    config = {
        'MAKE': 'make',
        'AR':   'ar'
    }

    # Compiler-specific flags
    flags = ''
    if self.compiler.name == 'intel':
        flags = '-O3 -ip -unroll -no-prec-div'
    elif self.compiler.name == 'gcc':
        flags = '-O3 -ffast-math -funroll-loops'
    elif self.compiler.name == 'pgi':
        flags = '-O3 -lpthread'
    elif self.compiler.name == 'g95':
        flags = '-O3 -fno-second-underscore'
    elif self.compiler.name == 'nag':
        flags = '-O4 -kind=byte -dusty -dcfuns'
    elif self.compiler.name == 'xl':
        flags = '-O3'
    config['F90_OPTS'] = flags
    config['F77_OPTS'] = flags

    # BLAS/LAPACK support
    # Note: BLAS/LAPACK must be compiled with OpenMP support
    # if the +openmp variant is chosen
    blas = 'blas.a'
    lapack = 'lapack.a'
    if '+blas' in spec:
        blas = spec['blas'].libs.joined()
    if '+lapack' in spec:
        lapack = spec['lapack'].libs.joined()
    # lapack must come before blas
    config['LIB_LPK'] = ' '.join([lapack, blas])

    # FFT support
    if '+fft' in spec:
        config['LIB_FFT'] = join_path(spec['fftw'].prefix.lib,
                                    'libfftw3.so')
        config['SRC_FFT'] = 'zfftifc_fftw.f90'
    else:
        config['LIB_FFT'] = 'fftlib.a'
        config['SRC_FFT'] = 'zfftifc.f90'

    # MPI support
    if '+mpi' in spec:
        config['F90'] = spec['mpi'].mpifc
        config['F77'] = spec['mpi'].mpif77
    else:
        config['F90'] = spack_fc
        config['F77'] = spack_f77
        config['SRC_MPI'] = 'mpi_stub.f90'

    # OpenMP support
    if '+openmp' in spec:
        config['F90_OPTS'] += ' ' + self.compiler.openmp_flag
        config['F77_OPTS'] += ' ' + self.compiler.openmp_flag
    else:
        config['SRC_OMP'] = 'omp_stub.f90'

    # Libxc support
    if '+libxc' in spec:
        config['LIB_libxc'] = ' '.join([
            join_path(spec['libxc'].prefix.lib, 'libxcf90.so'),
            join_path(spec['libxc'].prefix.lib, 'libxc.so')
        ])
        config['SRC_libxc'] = ' '.join([
            'libxc_funcs.f90',
            'libxc.f90',
            'libxcifc.f90'
        ])
    else:
        config['SRC_libxc'] = 'libxcifc_stub.f90'

    # Write configuration options to include file
    with open('make.inc', 'w') as inc:
        for key in config:
            inc.write('{0} = {1}\n'.format(key, config[key]))

config is just a dictionary that we can add key-value pairs to. By the end of the edit() method we write the contents of our dictionary to make.inc.

CMake

CMake is another common build system that has been gaining popularity. It works in a similar manner to Autotools but with differences in variable names, the number of configuration options available, and the handling of shared libraries. Typical build incantations look like this:

def install(self, spec, prefix):
    cmake("-DCMAKE_INSTALL_PREFIX:PATH=/path/to/install_dir ..")
    make()
    make("install")

As you can see from the example above, it’s very similar to invoking configure and make in an Autotools build system. However, the variable names and options differ. Most options in CMake are prefixed with a '-D' flag to indicate a configuration setting.

In the CMakePackage class we can override the following phases:

  1. cmake()
  2. build()
  3. install()

The CMakePackage class also provides sensible defaults so we only need to override cmake_args().

Let’s look at these defaults in the CMakePackage class in the _std_args() method:

$ spack edit --build-system cmake
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
    @staticmethod
    def _std_args(pkg):
        """Computes the standard cmake arguments for a generic package"""
        try:
            generator = pkg.generator
        except AttributeError:
            generator = 'Unix Makefiles'

        # Make sure a valid generator was chosen
        valid_generators = ['Unix Makefiles', 'Ninja']
        if generator not in valid_generators:
            msg  = "Invalid CMake generator: '{0}'\n".format(generator)
            msg += "CMakePackage currently supports the following "
            msg += "generators: '{0}'".format("', '".join(valid_generators))
            raise InstallError(msg)

        try:
            build_type = pkg.spec.variants['build_type'].value
        except KeyError:
            build_type = 'RelWithDebInfo'

        args = [
            '-G', generator,
            '-DCMAKE_INSTALL_PREFIX:PATH={0}'.format(pkg.prefix),
            '-DCMAKE_BUILD_TYPE:STRING={0}'.format(build_type),
            '-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON'
        ]

        if platform.mac_ver()[0]:
            args.extend([
                '-DCMAKE_FIND_FRAMEWORK:STRING=LAST',
                '-DCMAKE_FIND_APPBUNDLE:STRING=LAST'
            ])

        # Set up CMake rpath
        args.append('-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=FALSE')
        rpaths = ';'.join(spack.build_environment.get_rpaths(pkg))
        args.append('-DCMAKE_INSTALL_RPATH:STRING={0}'.format(rpaths))
        # CMake's find_package() looks in CMAKE_PREFIX_PATH first, help CMake
        # to find immediate link dependencies in right places:
        deps = [d.prefix for d in
                pkg.spec.dependencies(deptype=('build', 'link'))]
        deps = filter_system_paths(deps)
        args.append('-DCMAKE_PREFIX_PATH:STRING={0}'.format(';'.join(deps)))
        return args

Some CMake packages use different generators. Spack is able to support Unix-Makefile generators as well as Ninja generators.

If no generator is specified Spack will default to Unix Makefiles.

Next we setup the build type. In CMake you can specify the build type that you want. Options include:

  1. empty
  2. Debug
  3. Release
  4. RelWithDebInfo
  5. MinSizeRel

With these options you can specify whether you want your executable to have the debug version only, release version or the release with debug information. Release executables tend to be more optimized than Debug. In Spack, we set the default as RelWithDebInfo unless otherwise specified through a variant.

Spack then automatically sets up the -DCMAKE_INSTALL_PREFIX path, appends the build type (RelWithDebInfo default), and then specifies a verbose Makefile.

Next we add the rpaths to -DCMAKE_INSTALL_RPATH:STRING.

Finally we add to -DCMAKE_PREFIX_PATH:STRING the locations of all our dependencies so that CMake can find them.

In the end our cmake line will look like this (example is xrootd):

$ cmake $HOME/spack/var/spack/stage/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk/xrootd-4.6.0 -G Unix Makefiles -DCMAKE_INSTALL_PREFIX:PATH=$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_FIND_FRAMEWORK:STRING=LAST -DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=FALSE -DCMAKE_INSTALL_RPATH:STRING=$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk/lib:$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk/lib64 -DCMAKE_PREFIX_PATH:STRING=$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/cmake-3.9.4-hally3vnbzydiwl3skxcxcbzsscaasx5

We can see now how CMake takes care of a lot of the boilerplate code that would have to be otherwise typed in.

Let’s try to recreate callpath:

$ spack create -f https://github.com/llnl/callpath/archive/v1.0.3.tar.gz
==> This looks like a URL for callpath
==> Found 4 versions of callpath:

1.0.3  https://github.com/LLNL/callpath/archive/v1.0.3.tar.gz
1.0.2  https://github.com/LLNL/callpath/archive/v1.0.2.tar.gz
1.0.1  https://github.com/LLNL/callpath/archive/v1.0.1.tar.gz
1.0    https://github.com/LLNL/callpath/archive/v1.0.tar.gz

==> How many would you like to checksum? (default is 1, q to abort) 1
==> Downloading...
==> Fetching https://github.com/LLNL/callpath/archive/v1.0.3.tar.gz
######################################################################## 100.0%
==> Checksummed 1 version of callpath
==> This package looks like it uses the cmake build system
==> Created template for callpath package
==> Created package file: /Users/mamelara/spack/var/spack/repos/builtin/packages/callpath/package.py

which then produces the following template:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

#
# This is a template package file for Spack.  We've put "FIXME"
# next to all the things you'll want to change. Once you've handled
# them, you can save this file and test your package like this:
#
#     spack install callpath
#
# You can edit this file again by typing:
#
#     spack edit callpath
#
# See the Spack documentation for more information on packaging.
# If you submit this package back to Spack as a pull request,
# please first remove this boilerplate and all FIXME comments.
#
from spack import *


class Callpath(CMakePackage):
    """FIXME: Put a proper description of your package here."""

    # FIXME: Add a proper url for your package's homepage here.
    homepage = "http://www.example.com"
    url      = "https://github.com/llnl/callpath/archive/v1.0.1.tar.gz"

    version('1.0.3', 'c89089b3f1c1ba47b09b8508a574294a')

    # FIXME: Add dependencies if required.
    # depends_on('foo')

    def cmake_args(self):
        # FIXME: Add arguments other than
        # FIXME: CMAKE_INSTALL_PREFIX and CMAKE_BUILD_TYPE
        # FIXME: If not needed delete this function
        args = []
        return args

Again we fill in the details:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Callpath(CMakePackage):
    """Library for representing callpaths consistently in
       distributed-memory performance tools."""

    homepage = "https://github.com/llnl/callpath"
    url      = "https://github.com/llnl/callpath/archive/v1.0.3.tar.gz"

    version('1.0.3', 'c89089b3f1c1ba47b09b8508a574294a')

    depends_on("elf", type="link")
    depends_on("libdwarf")
    depends_on("dyninst")
    depends_on("adept-utils")
    depends_on("mpi")
    depends_on("cmake@2.8:", type="build")

As mentioned earlier, Spack will use sensible defaults to prevent repeated code and to make writing CMake package files simpler.

In callpath, we want to add options to CALLPATH_WALKER as well as add compiler flags. We add the following options like so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class Callpath(CMakePackage):
    """Library for representing callpaths consistently in
       distributed-memory performance tools."""

    homepage = "https://github.com/llnl/callpath"
    url      = "https://github.com/llnl/callpath/archive/v1.0.3.tar.gz"

    version('1.0.3', 'c89089b3f1c1ba47b09b8508a574294a')

    depends_on("elf", type="link")
    depends_on("libdwarf")
    depends_on("dyninst")
    depends_on("adept-utils")
    depends_on("mpi")
    depends_on("cmake@2.8:", type="build")

    def cmake_args(self):
        args = ["-DCALLPATH_WALKER=dyninst"]

        if self.spec.satisfies("^dyninst@9.3.0:"):
            std.flag = self.compiler.cxx_flag
            args.append("-DCMAKE_CXX_FLAGS='{0}' -fpermissive'".format(
                std_flag))

        return args

Now we can control our build options using cmake_args(). If defaults are sufficient enough for the package, we can leave this method out.

CMakePackage classes allow for control of other features in the build system. For example, you can specify the path to the “out of source” build directory and also point to the root of the CMakeLists.txt file if it is placed in a non-standard location.

A good example of a package that has its CMakeLists.txt file located at a different location is found in spades.

$ spack edit spades
root_cmakelists_dir = "src"

Here root_cmakelists_dir will tell Spack where to find the location of CMakeLists.txt. In this example, it is located a directory level below in the src directory.

Some CMake packages also require the install phase to be overridden. For example, let’s take a look at sniffles.

$ spack edit sniffles

In the install() method, we have to manually install our targets so we override the install() method to do it for us:

# the build process doesn't actually install anything, do it by hand
def install(self, spec, prefix):
    mkdir(prefix.bin)
    src = "bin/sniffles-core-{0}".format(spec.version.dotted)
    binaries = ['sniffles', 'sniffles-debug']
    for b in binaries:
        install(join_path(src, b), join_path(prefix.bin, b))

PythonPackage

Python extensions and modules are built differently from source than most applications. Python uses a setup.py script to install Python modules. The script consists of a call to setup() which provides the information required to build a module to Distutils. If you’re familiar with pip or easy_install, setup.py does the same thing.

These modules are usually installed using the following line:

$ python setup.py install

There are also a list of commands and phases that you can call. To see the full list you can run:

$ python setup.py --help-commands
Standard commands:
    build             build everything needed to install
    build_py          "build" pure Python modules (copy to build directory)
    build_ext         build C/C++ extensions (compile/link to build directory)
    build_clib        build C/C++ libraries used by Python extensions
    build_scripts     "build" scripts (copy and fixup #! line)
    clean             (no description available)
    install           install everything from build directory
    install_lib       install all Python modules (extensions and pure Python)
    install_headers   install C/C++ header files
    install_scripts   install scripts (Python or otherwise)
    install_data      install data files
    sdist             create a source distribution (tarball, zip file, etc.)
    register          register the distribution with the Python package index
    bdist             create a built (binary) distribution
    bdist_dumb        create a "dumb" built distribution
    bdist_rpm         create an RPM distribution
    bdist_wininst     create an executable installer for MS Windows
    upload            upload binary package to PyPI
    check             perform some checks on the package

We can write package files for Python packages using the Package class, but the class brings with it a lot of methods that are useless for Python packages. Instead, Spack has a PythonPackage subclass that allows packagers of Python modules to be able to invoke setup.py and use Distutils, which is much more familiar to a typical python user.

To see the defaults that Spack has for each a methods, we will take a look at the PythonPackage class:

$ spack edit --build-system python

We see the following:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
class PythonPackage(PackageBase):

    # Standard commands

    def build(self, spec, prefix):
        """Build everything needed to install."""
        args = self.build_args(spec, prefix)

        self.setup_py('build', *args)

    def build_args(self, spec, prefix):
        """Arguments to pass to build."""
        return []

    def build_py(self, spec, prefix):
        '''"Build" pure Python modules (copy to build directory).'''
        args = self.build_py_args(spec, prefix)

        self.setup_py('build_py', *args)

    def build_py_args(self, spec, prefix):
        """Arguments to pass to build_py."""
        return []

    def build_ext(self, spec, prefix):
        """Build C/C++ extensions (compile/link to build directory)."""
        args = self.build_ext_args(spec, prefix)

        self.setup_py('build_ext', *args)

    def build_ext_args(self, spec, prefix):
        """Arguments to pass to build_ext."""
        return []

    def build_clib(self, spec, prefix):
        """Build C/C++ libraries used by Python extensions."""
        args = self.build_clib_args(spec, prefix)

        self.setup_py('build_clib', *args)

    def build_clib_args(self, spec, prefix):
        """Arguments to pass to build_clib."""
        return []

    def build_scripts(self, spec, prefix):
        '''"Build" scripts (copy and fixup #! line).'''
        args = self.build_scripts_args(spec, prefix)

        self.setup_py('build_scripts', *args)

    def clean(self, spec, prefix):
        """Clean up temporary files from 'build' command."""
        args = self.clean_args(spec, prefix)

        self.setup_py('clean', *args)

    def clean_args(self, spec, prefix):
        """Arguments to pass to clean."""
        return []

    def install(self, spec, prefix):
        """Install everything from build directory."""
        args = self.install_args(spec, prefix)

        self.setup_py('install', *args)

    def install_args(self, spec, prefix):
        """Arguments to pass to install."""
        args = ['--prefix={0}'.format(prefix)]

        # This option causes python packages (including setuptools) NOT
        # to create eggs or easy-install.pth files.  Instead, they
        # install naturally into $prefix/pythonX.Y/site-packages.
        #
        # Eggs add an extra level of indirection to sys.path, slowing
        # down large HPC runs.  They are also deprecated in favor of
        # wheels, which use a normal layout when installed.
        #
        # Spack manages the package directory on its own by symlinking
        # extensions into the site-packages directory, so we don't really
        # need the .pth files or egg directories, anyway.
        #
        # We need to make sure this is only for build dependencies. A package
        # such as py-basemap will not build properly with this flag since
        # it does not use setuptools to build and those does not recognize
        # the --single-version-externally-managed flag
        if ('py-setuptools' == spec.name or          # this is setuptools, or
            'py-setuptools' in spec._dependencies and  # it's an immediate dep
            'build' in spec._dependencies['py-setuptools'].deptypes):
                args += ['--single-version-externally-managed', '--root=/']

        return args

    def install_lib(self, spec, prefix):
        """Install all Python modules (extensions and pure Python)."""
        args = self.install_lib_args(spec, prefix)

        self.setup_py('install_lib', *args)

    def install_lib_args(self, spec, prefix):
        """Arguments to pass to install_lib."""
        return []

    def install_headers(self, spec, prefix):
        """Install C/C++ header files."""
        args = self.install_headers_args(spec, prefix)

        self.setup_py('install_headers', *args)

    def install_headers_args(self, spec, prefix):
        """Arguments to pass to install_headers."""
        return []

    def install_scripts(self, spec, prefix):
        """Install scripts (Python or otherwise)."""
        args = self.install_scripts_args(spec, prefix)

        self.setup_py('install_scripts', *args)

    def install_scripts_args(self, spec, prefix):
        """Arguments to pass to install_scripts."""
        return []

    def install_data(self, spec, prefix):
        """Install data files."""
        args = self.install_data_args(spec, prefix)

        self.setup_py('install_data', *args)

    def install_data_args(self, spec, prefix):
        """Arguments to pass to install_data."""
        return []

    def sdist(self, spec, prefix):
        """Create a source distribution (tarball, zip file, etc.)."""
        args = self.sdist_args(spec, prefix)

        self.setup_py('sdist', *args)

    def sdist_args(self, spec, prefix):
        """Arguments to pass to sdist."""
        return []

    def register(self, spec, prefix):
        """Register the distribution with the Python package index."""
        args = self.register_args(spec, prefix)

        self.setup_py('register', *args)

    def register_args(self, spec, prefix):
        """Arguments to pass to register."""
        return []

    def bdist(self, spec, prefix):
        """Create a built (binary) distribution."""
        args = self.bdist_args(spec, prefix)

        self.setup_py('bdist', *args)

    def bdist_args(self, spec, prefix):
        """Arguments to pass to bdist."""
        return []

    def bdist_dumb(self, spec, prefix):
        '''Create a "dumb" built distribution.'''
        args = self.bdist_dumb_args(spec, prefix)

        self.setup_py('bdist_dumb', *args)

    def bdist_dumb_args(self, spec, prefix):
        """Arguments to pass to bdist_dumb."""
        return []

    def bdist_rpm(self, spec, prefix):
        """Create an RPM distribution."""
        args = self.bdist_rpm(spec, prefix)

        self.setup_py('bdist_rpm', *args)

    def bdist_rpm_args(self, spec, prefix):
        """Arguments to pass to bdist_rpm."""
        return []

    def bdist_wininst(self, spec, prefix):
        """Create an executable installer for MS Windows."""
        args = self.bdist_wininst_args(spec, prefix)

        self.setup_py('bdist_wininst', *args)

    def bdist_wininst_args(self, spec, prefix):
        """Arguments to pass to bdist_wininst."""
        return []

    def upload(self, spec, prefix):
        """Upload binary package to PyPI."""
        args = self.upload_args(spec, prefix)

        self.setup_py('upload', *args)

    def upload_args(self, spec, prefix):
        """Arguments to pass to upload."""
        return []

    def check(self, spec, prefix):
        """Perform some checks on the package."""
        args = self.check_args(spec, prefix)

        self.setup_py('check', *args)

    def check_args(self, spec, prefix):
        """Arguments to pass to check."""
        return []

Each of these methods have sensible defaults or they can be overridden.

We will write a package file for Pandas:

$ spack create -f https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz
==> This looks like a URL for pandas
==> Warning: Spack was unable to fetch url list due to a certificate verification problem. You can try running spack -k, which will not check SSL certificates. Use this at your own risk.
==> Found 1 version of pandas:

0.19.0  https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz

==> How many would you like to checksum? (default is 1, q to abort) 1
==> Downloading...
==> Fetching https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz
######################################################################## 100.0%
==> Checksummed 1 version of pandas
==> This package looks like it uses the python build system
==> Changing package name from pandas to py-pandas
==> Created template for py-pandas package
==> Created package file: /Users/mamelara/spack/var/spack/repos/builtin/packages/py-pandas/package.py

And we are left with the following template:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

#
# This is a template package file for Spack.  We've put "FIXME"
# next to all the things you'll want to change. Once you've handled
# them, you can save this file and test your package like this:
#
#     spack install py-pandas
#
# You can edit this file again by typing:
#
#     spack edit py-pandas
#
# See the Spack documentation for more information on packaging.
# If you submit this package back to Spack as a pull request,
# please first remove this boilerplate and all FIXME comments.
#
from spack import *


class PyPandas(PythonPackage):
    """FIXME: Put a proper description of your package here."""

    # FIXME: Add a proper url for your package's homepage here.
    homepage = "http://www.example.com"
    url      = "https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz"

    version('0.19.0', 'bc9bb7188e510b5d44fbdd249698a2c3')

    # FIXME: Add dependencies if required.
    # depends_on('py-setuptools', type='build')
    # depends_on('py-foo',        type=('build', 'run'))

    def build_args(self, spec, prefix):
        # FIXME: Add arguments other than --prefix
        # FIXME: If not needed delete this function
        args = []
        return args

As you can see this is not any different than any package template that we have written. We have the choice of providing build options or using the sensible defaults

Luckily for us, there is no need to provide build args.

Next we need to find the dependencies of a package. Dependencies are usually listed in setup.py. You can find the dependencies by searching for install_requires keyword in that file. Here it is for Pandas:

# ... code
if sys.version_info[0] >= 3:

setuptools_kwargs = {
                     'zip_safe': False,
                     'install_requires': ['python-dateutil >= 2',
                                          'pytz >= 2011k',
                                          'numpy >= %s' % min_numpy_ver],
                     'setup_requires': ['numpy >= %s' % min_numpy_ver],
                     }
if not _have_setuptools:
    sys.exit("need setuptools/distribute for Py3k"
             "\n$ pip install distribute")

# ... more code

You can find a more comprehensive list at the Pandas documentation.

By reading the documentation and setup.py we found that Pandas depends on python-dateutil, pytz, and numpy, numexpr, and finally bottleneck.

Here is the completed Pandas script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# Copyright 2013-2018 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack import *


class PyPandas(PythonPackage):
    """pandas is a Python package providing fast, flexible, and expressive
       data structures designed to make working with relational or
       labeled data both easy and intuitive. It aims to be the
       fundamental high-level building block for doing practical, real
       world data analysis in Python. Additionally, it has the broader
       goal of becoming the most powerful and flexible open source data
       analysis / manipulation tool available in any language.
    """
    homepage = "http://pandas.pydata.org/"
    url = "https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz"

    version('0.19.0', 'bc9bb7188e510b5d44fbdd249698a2c3')
    version('0.18.0', 'f143762cd7a59815e348adf4308d2cf6')
    version('0.16.1', 'fac4f25748f9610a3e00e765474bdea8')
    version('0.16.0', 'bfe311f05dc0c351f8955fbd1e296e73')

    depends_on('py-dateutil', type=('build', 'run'))
    depends_on('py-numpy', type=('build', 'run'))
    depends_on('py-setuptools', type='build')
    depends_on('py-cython', type='build')
    depends_on('py-pytz', type=('build', 'run'))
    depends_on('py-numexpr', type=('build', 'run'))
    depends_on('py-bottleneck', type=('build', 'run'))

It is quite important to declare all the dependencies of a Python package. Spack can “activate” Python packages to prevent the user from having to load each dependency module explictly. If a dependency is missed, Spack will be unable to properly activate the package and it will cause an issue. To learn more about extensions go to spack extensions.

From this example, you can see that building Python modules is made easy through the PythonPackage class.

Other Build Systems

Although we won’t get in depth with any of the other build systems that Spack supports, it is worth mentioning that Spack does provide subclasses for the following build systems:

  1. IntelPackage
  2. SconsPackage
  3. WafPackage
  4. RPackage
  5. PerlPackage
  6. QMakePackage

Each of these classes have their own abstractions to help assist in writing package files. For whatever doesn’t fit nicely into the other build-systems, you can use the Package class.

Hopefully by now you can see how we aim to make packaging simple and robust through these classes. If you want to learn more about these build systems, check out Implementing the installation procedure in the Packaging Guide.