~vijaykumar

We use an example project to demonstrate the SCons build system. The project creates a PDF document, from an asciidoc text file. Asciidoc is a tool that can convert a text based wiki-like markup to various other output formats. The asciidoc text file in our example refers to 3 PNG image files. The images are themselves drawn in Dia and are generated from corresponding Dia files.

The dependency graph is shown below.

The PDF can be created from the asciidoc text file using the following asciidoc command.

The PNG image files can be created from the corresponding .dia files using the following command.

The Makefile to build the above project is given below.

The input file to SCons is SConstruct, which is similar to a Makefile. The SConstruct is a Python script, that is invoked by SCons to specify the dependency graph, and the commands to build the target from the dependencies. The SConstruct file to build the above project is given below.

The following command is used to invoke SCons to build all the targets. The -Q option specifies quiet mode, where diagnostic messages are not printed.

The SConstruct file has a list of invocations to the Command() function. The target, the source and the command to build the target from the source is passed to Command() function. With this information the SConstruct file has a one-to-one mapping with the Makefile.

It must be noted that the Command() function does not cause the commands to be executed immediately. The function invocation only causes the in-memory dependency graph to be updated. After the SConstruct file is completely read, SCons determines which files have to be rebuilt and only those commands are executed.

The SConstruct file is deliberately written to mimic the Makefile. We will gradually improve the SConstruct file, to use the advanced features provides by SCons.

Sometimes it is required to customize a command. For example, to pass additional options and to specify a different value for an option’s argument. In make, we have make variables, that can be assigned a value and then used within the command. SCons has what is called a Construction Environment. A Construction Environment is bundle of variables associated with values and set of rules to build targets from dependencies.

An environment object is created by invoking the Environment() constructor. Variables are set in the environment using the Replace() method, and command rules can be added using Command() method. The previous example re-written using the construction environment is shown below. Here we specify the size of the image to be created using the variable SIZE, and we use that variable in the dia command to create a PNG file with required size.

The Replace() accepts a single argument. The method uses Python’s named arguments style invocation, to specify the name of the variable and the value for the variable. Just as with the $TARGET and $SOURCE special variables, user define variables can also be expanded using $varname notation.

Note that, in the version 1 of the SConstruct file, there was no environment defined, and Command() was invoked as a function. A default environment was created by SCons, and rules were associated with the default environment.

In the previous example, the dia command to build the PNG file is repeated 3 times, once for every PNG file to be created. This is a violation of the DRY (Don’t Repeat Yourself) rule. The problem with such repetition is that, it does scale well. Where there are 100s of PNG files to be created, the same rule has repeated 100 times. Apart from the issue of having to re-type the same command over and over again, if the command is to be modified it has to be done in 100 locations, and if one of them is missed, its a bug.

Enter builders. Builders are SCons way of creating a generic rule, that specifies how to build a specific kind of target from source files. The above example re-written using builders is shown below. Two builders are created one to create a PNG from DIA files. And another to create PDF from TXT and PNG files. And then the builders are invoked instead of the Command() method.

The Builder() constructor accepts an action argument, that defines generically how to build the target from the source, using the $TARGET and $SOURCE variables. The environment variable BUILDERS is a dictionary that maps builder names to builder objects. To be able to use the newly created builders, the builder objects should be added to the BUILDERS environment variable. This is done using the Append() method.

Once added to the BUILDERS environment variable, the builders can be invoked as methods of the environment. Only the target and the dependencies need to be passed to the builders.

There is another level of violation of the DRY rule in the previous example. The asciidoc TXT file is listed below.

The asciidoc TXT file, already contains the list of images to be included. But this information is repeated with the SConstruct file. Whenever a new PNG file is referred to in the TXT file, the file has be added to the SConstruct file. It would be great if SCons can scan the file add these dependencies automatically. That is exactly what scanners are for.

Scanners are functions attached to specific types of files. When a file is specified as a dependency, SCons invokes the scanners associated with the file type. Each scanner, reads the file and looks for additional dependencies, and returns the list of files. SCons add these files to the list of dependencies.

The above example is modified to use scanners to automatically add the PNG dependencies. Regular expressions are used to search for the pattern image::<filename>[, and the filename is extracted.

The Scanner() constructor is used to create a scanner object. The constructor accepts the scanner function and the file types as arguments. The environment variables SCANNERS is a list that contains the available scanner objects. To be able to use a scanner, the scanner object has to added to this list. The scanner object is appended using the Append() method.

The scanner function is passed three arguments

The file to be scanned is passed as a Node object. A Node object represents file or a directory. The contents of the file can be retrieved using get_text_contents() method.

The scanner returns a list of Node objects, that are to be added to the dependencies. The File() function accepts a list of filenames, and returns a list of Node objects.

Many projects consist of a hierarchy of directories. In such cases, it would be convenient if the targets to built can be specified by a separate configuration file in each directory. In SCons, this can be achieved by creating a toplevel SConstruct file, and one SConscript for each directory. The environment is created in the toplevel SConstruct file and is exported to the SConscript files. And hence all the variables, builders, scanners, and rules are part of the same environment. This allows SCons build a single dependency graph, which can correctly track inter-directory dependencies, and allow for accurate parallel builds.

The above example is modified to have two sub-dirs, and the same files are copied to both the sub-directories. The resulting SConstruct file and SConscript in the sub-directories is shown below.

The builders and scanners are defined in the top level SConstruct file. The environment stored in Python variable env is exported to the SConscript files using the Export() function. The SConscript() function is then used to invoke the sub-directory SConscript files.

The SConscript files import the Python variable env using the Import function. Then the builders are invoked to specify the targets and their dependencies.

When invoking SCons, the key value pairs can be specified in the command line as arguments. The general syntax is shown below.

The key value pairs are then available through the ARGUMENTS object, in the SConstruct file.

The above example is modified to accept the SIZE from the SCons command line. The resulting SConstruct file is shown below.

The ARGUMENTS object has a get() method. The get() method accepts two arguments the key and the default value, if the user did not specify a value.

SCons has many built-in builders and scanners for many common tasks. For example, the Program() builder can build an executable from C files. And scanners are available to scan C files and automatically add the header files as dependency. So to build an executable from 4 files, the following 1-line SConstruct is sufficient.

Since SCons has the list of targets to be built, it can clean up the generated files, with no further information. SCons can be invoked with the -c option, to cleanup files generated during the build.

Another interesting aspect of SCons is that it uses the MD5 signature of the file, to see if file has changed, instead of the modification time. This can be changed by invoking the Decider() function.

The SCons documentation though complete, approaches SCons from a newbie perspective. For people who are familiar with make, the documentation does not help to apply their existing knowledge of build systems. I hope with the background provided by this article, a make user can easily transition to the world of SCons.