Chapter 1. Getting Started

To start using Aap you must have two applications:

• Python version 1.5 or later

• Aap

Python is often installed already. Try this:

    python -V

For obtaining and installing Aap look here: www.a-a-p.org/download.html.

To check if your Aap program is working, type this command:

    aap --help

Chapter 2. Compiling a Program

A "Hello world" (of sorts)

Most programming languages start with a short example that prints a "hello world" message. With Aap, this is also possible. In a file called main.aap, enter the following:

    :print Hello, World!

Now run Aap by entering aap at the command line. Aap will respond something like this:

    % aap
    Hello, World!
    Aap: No target on the command line and no $TARGET, build rules or "all" target in a recipe

Using Aap to compile "hello.c"

Suppose you have written a "hello world" program in C, and the sources are stored in a file called hello.c. Aap already knows about the C language (and several others), so the instructions to Aap about how to compile this program are very short. Instructions for Aap are stored in a file with the extension .aap. Such a file is called a recipe.

This is the recipe for compiling such a program with Aap:

    :program hello : hello.c

Write this text in a file main.aap, in the same directory as hello.c. Now invoke Aap to compile hello.c into the program hello:

    % ls
    hello.c             main.aap
    % aap
1   Aap: Creating directory "/home/mool/tmp/build-FreeBSD4_5_RELEASE"
2   Aap: cc -I/usr/local/include -g -O2 -E -MM hello.c > build-FreeBSD4_5_RELEASE/hello.c.aap
3   Aap: cc -I/usr/local/include -g -O2 -c -o build-FreeBSD4_5_RELEASE/hello.o hello.c
4   Aap: cc -L/usr/local/lib -g -O2 -o hello build-FreeBSD4_5_RELEASE/hello.o

Other things to do with "hello world"

The same simple recipe not only specifies how to build the "hello" program, it can also be used to install the program:

    % aap install PREFIX=try
    Aap: Creating directory "try/bin/"
    Aap: Copied "test/hello" to "try/bin/hello"
    Aap: /usr/bin/strip 'try/bin/hello'

See the reference manual for details about :program.

Several Source Files

When you have several files with source code you can specify them as a list:

    :program myprogram : main.c
                         version.c
                         help.c

There are three source files: main.c, version.c and help.c. Notice that it is not necessary to use a line continuation character, as you would have to do in a Makefile. The list ends at a line where the indent is equal to or less than what the assignment started with. The amount of indent for the continuation lines is irrelevant, so long as it's more than the indent of the first line.

The Makefile-style line continuation with a backslash just before the line break can also be used, by the way.

Indents are very important, just like in a Python script. Make sure your tabstop is always set to the standard value of eight, otherwise you might run into trouble when mixing tabs and spaces!

When you give a list of files to :program, Aap will determine dependencies and compile each of the source files in turn, and then link them all together into an executable.

Variables and Assignments

Sometimes it is convenient to have an abbreviation for a long list of files. Aap supports this through variables (just like the make command and the shell).

An assignment has the form:

    variablename = expression

    foo = one
    Foo = two
    FOO = three
    :print $foo $Foo $FOO

If you want text directly after the variable's value, for example, to append an extension to the value of a variable, the text may be impossible to distinguish from a variable name. In these cases you must put parenthesis around the variable name, so that Aap knows where it ends:

    all:
      MakeName = Make
      :print $(MakeName)file    # 'f' can be in a variable name
      :print $(MakeName).txt    # '.' can be in a variable name
      :print $MakeName-more     # '-' is not in a variable name

    % aap -f try.aap
    Makefile
    Make.txt
    Make-more
    %

All Aap commands, except the assignment, start with a colon. That makes them easy to recognize.

Some characters in the expression have a special meaning. The :print command also handles a few arguments in a special way. To avoid the special meaning use the $(x) form, where "x" is the special character. For example, to print a literal dollar use $($). See the user manual for a complete list.

Comments

Someone who sees this recipe would like to know what it's for. This requires adding comments. These start with a "#" character and extend until the end of the line (like in a Makefile and Python script).

It is also possible to associate a comment with a specific item:

    # A-A-P recipe for compiling "myprogram"
    :program myprogram { comment = MyProgram is really great } :
            main.c            # startup stuff
            version.c         # just the date stamp
            help.c            # display a help message

Now run Aap with a "comment" argument:

    % aap comment
    target "myprogram": MyProgram is really great
    target "clean": delete generated files that are not distributed
    target "cleanmore": delete all generated files
    target "cleanALL": delete all generated files, AAPDIR and build-* directories
    target "install": install files
    target "uninstall": delete installed files
    %

The text inside curly braces is called an attribute. In this case the attribute name is "comment" and the attribute value is "MyProgram is really great". An attribute can be used to attach extra information to a file name. We will encounter more attributes later on.

Dependencies

Let's go back to the "Hello world" example and find out what happens when you change a source file. Use this hello.c file:

    #include <stdio.h>
    #include "hello.h"
    main()
    {
        printf("Hello %s\n", world);
    }

The included "hello.h" file defines "world":

    #define world "World!"

If you run Aap, the "hello" program will be built as before. If you run Aap again you will notice that nothing happens. Aap remembers that "hello.c" was already compiled. Now try this:

    % touch hello.c
    % aap
    %

If you have been using the "make" program you would expect something to happen. But Aap checks the contents of the file, not the timestamp. A signature of "hello.c" is computed and if it is still the same as before Aap knows that it does not need to be compiled, even though "hello.c" is newer than the "hello" program.

Aap uses the mechanism of dependencies. When you use the :program command Aap knows that the target depends on the sources. When one of the sources changes, the commands to build the target from the sources must be executed. This can also be specified explicitly:

      hello$EXESUF : $BDIR/hello$OBJSUF
          :do build $source

      $BDIR/hello$OBJSUF : hello.c
          :do compile $source

The generic form of a dependency is:

      target : list-of-sources
           build-commands

The colon after the target is important, it separates the target from the sources. It is not required to put a space before it, but there must be a space after it. We mostly put white space before the colon, so that it is easy to spot. There could be several targets, but that is unusual.

There are two dependencies in the example. In the first one the target is "hello$EXESUF", the source file is "$BDIR/hello$OBJSUF" and the build command is ":do build $source". This specifies how to build the "hello$EXESUF" program from the "$BDIR/hello$OBJSUF" object file. The second dependency specifies how to compile "hello.c" into "$BDIR/hello$OBJSUF" with the command ":do compile $source". The "BDIR" variable holds the name of the platform-dependent directory for intermediate results, as mentioned in the first example of this chapter. In case you need it, the $EXESUF variable Aap is empty on Unix and ".exe" on MS-Windows.

The relation between the two dependencies in the example is that the source of the first one is the target in the second one. The logic is that Aap follows the dependencies and executes the associated build commands. In this case "hello$EXESUF" depends on "$BDIR/hello$OBJSUF", which then depends on "hello.c". The last dependency is handled first, thus first hello.c is compiled by the build command of the second dependency, and then linked into "hello$EXESUF" by the build command of the first dependency.

Now change the "hello.h" file by replacing "World" with 'Universe":

    #define world "Universe!"

If you now run Aap with "aap hello" or "aap hello.exe" the "hello" program will be built. But you never mentioned the "hello.h" file in the recipe. How did Aap find out the change in this file matters? When Aap is run to update the "hello" program, this is what will happen:

1. The first dependency with "hello$EXESUF" as the target is found, it depends on "$BDIR/hello$OBJSUF".

2. The second dependency with "$BDIR/hello$OBJSUF" as the target is found. The source file "hello.c" is recognized as a C program file. It is inspected for included files. This finds the "hello.h" file. "stdio.h" is ignored, since it is a system file. "hello.h" is added to the list of files that the target depends on.

3. Each file that the target depends on is updated. In this case "hello.c" and "hello.h". No dependency has been specified for them and the files exist, thus nothing happens.

4. Aap computes signatures for "hello.c" and "hello.h". It also computes a signature for the build commands. If one of them changed since the last time the target was built, or the target was never built before, the target is considered "outdated" and the build commands are executed.

5. The second dependency is now finished, "$BDIR/hello$OBJSUF" is up-to-date. Aap goes back to the first dependency.

6. Aap computes a signature for "$BDIR/hello$OBJSUF". Note that this happens after the second dependency was handled, it may have changed the file. It also computes a signature for the build command. If one of them changed since the last time the target was built, or the target was never built before, the target is considered "outdated" and the build commands are executed.

Now try this: Append a comment to one of the lines in the "hello.c" file. This means the file is changed, thus when invoking Aap it will compile "hello.c". But the program is not built, because the produced intermediate file "$BDIR/hello$OBJSUF" is still equal to what it was the last time. When compiling a large program with many dependencies this mechanism avoids that adding a comment may cause a snowball effect. (Note: some compilers include line numbers or a timestamp in the object file, in that case building the program will happen anyway).

Compiling Multiple Programs

Suppose you have a number of sources files that are used to build two programs. You need to specify which files are used for which program. Here is an example:

1.    Common = help.c util.c
2.
3.    all : foo bar
4.
5.    :program foo : $Common foo.c
6.
7.    :program bar : $Common bar.c

This recipe defines three targets: "all", "foo" and "bar". "foo" and "bar are programs that Aap can build from source files. But the "all" target is not a file. This is called a virtual target: A name used for a target that does not exist as a file. Let's list the terminology of the items in a dependency:

Aap knows the target with the name "all" is always used as a virtual target. There are a few other names which Aap knows are virtual, see Table 36-1. For other targets you need to specify it with the "{virtual}" attribute.

The first dependency has no build commands. This only specifies that "all" depends on "foo" and "bar". Thus when Aap updates the "all" target, this dependency specifies that "foo" and "bar" need to be updated. Since the "all" target is the default target, this dependency causes both "foo" and "bar" to be updated when Aap is started without an argument. You can use "aap foo" to build "foo" only. The dependencies for "all" and "bar" will not be used then.

The two files help.c and util.c are used by both the "foo" and the "bar" program. To avoid having to type the file names twice, the "Common" variable is used.

Kinds of things you can build

Not everything you want to build is a program. Your recipe might need too build a library or a libtool archive. In these cases, :lib, :dll or :ltlib provide the same level of automation as :program does for programs. The :produce command is more generic, you can use this to build various kinds of things.

If all else fails, you can use Aap like the make program and explicitly list the commands you need to build your project.

Chapter 3. Publishing a Web Site

If you are maintaining a web site it is often a good idea to edit the files on your local system. After trying out the changes you then need to upload the changed files to the web server. A-A-P can be used to identify the files that changed and upload these files only. This is called publishing.

Uploading The Files

Here is an example of a recipe:

   Files = index.html
           project.html
           links.html
           images/logo.png
   :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $Files

That's all. You just need to specify the files you want to publish and the URL that says how and where to upload them to. Now "aap publish" will find out which files have changed and upload them:

    % aap publish
    Aap: Uploading ['/home/mool/www/foo/index.html'] to scp://user@ftp.foo.org/public_html/index.html
    Aap: scp '/home/mool/www/vim/index.html' 'user@ftp.foo.org:public_html/index.html'
    Aap: Uploaded "/home/mool/www/vim/index.html" to "scp://user@ftp.foo.org/public_html/index.html"
    %

The first time you execute the recipe all files will be uploaded. Aap will create the "images" directory for you. If you had already uploaded the files and want to avoid doing it again, first run the recipe with: "aap publish --touch". Aap will compute the signatures of the files as they are now and remember them. Only files that are changed will be uploaded from now on.

The :attr command uses its first argument as an attribute and further arguments as file names. It will attach the attribute to each of the files. In this case the "publish" attribute is added, which specifies the URL where to upload a file to. In the example the "scp" protocol is used, which is a good method for uploading files to a public server. "ftp" can be used as well, but this means your password will go over the internet, which is not safe. The special item "%file%" is replaced with the name of the file being published.

Generating a HTML File

It is common for HTML files to consist of a standard header, a body with the useful info and a footer. You don't want to manually add the header and footer to each page. When the header changes you would have to make the same change in many different files. Instead, use the recipe to generate the HTML files.

Let's start with a simple example: Generate the index.html file. Put the common header, containing a logo and navigation links, in "header.part". The footer, containing contact info for the maintainer, goes in "footer.part". The useful contents of the page goes in "index_body.part". Now you can use this recipe to generate "index.html" and publish it:

   Files =    index.html
              images/logo.png
   :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $Files

   all: $Files

   publish: $Files
       :publishall

   index.html: header.part index_body.part footer.part
       :cat $source >! $target

Notice that only the published files are put in the "Files" variable. These files get a "publish" attribute, which tells Aap that these are the files that need to be uploaded. The ".part" files are not published, thus they do not get the "publish" attribute.

Three dependencies follow. The "all" target is the virtual target we have seen before. It specifies that the default work for this recipe is to update the files in the "Files" variable. This means you don't accidentally upload the files by running "aap" without arguments. The normal way of use is to run "aap", check if the produced HTML file looks OK, then use "aap publish" to upload the file.

For "index.html" a target is specified with a build command. The :cat command concatenates the source files. "$source" stands for the source files used in the dependency: "header.part, "index_body.part" and "footer.part". The resulting text is written to "$target", which is the target of the dependency, thus "index.html". The ">!" is used to redirect the output of the :cat command and overwrite any existing result. This works just like the Unix "cat" command.

In the dependency with the "publish" target the :publishall command is used. This command goes through all the files which were given a "publish" attribute with the :attr command. Note that this does not work:

   # This won't work.
   Files = index.html {publish = scp://user@ftp.foo.org/public_html/%file%}

Using a "publish" attribute in an assignment will not make it used with the :publishall command.

Using ":rule" to Generate Several HTML Files

Your web site contains several pages, thus you need to specify how to generate each HTML page. This quickly becomes a lot of typing. We would rather specify once how to make a "xxx.html" file from a "xxx_body.part" file, and then give the list of names to use for "xxx" (if you have assocations with the name "xxx_body.part" that is your own imagination! :-). This is how it's done:

   Files =    *.html
              images/*.png
   :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $Files

   all: $Files

   publish: $Files
       :publishall

   :rule %.html : header.part %_body.part footer.part
       :cat $source >! $target

This is very similar to the example that only generates the "index.html" file. The first difference is in the value of "Files": It contains wildcards. These wildcards are expanded when they are used where a file name is expected. The expansion is not done in the assignment! More about that later. In the three places where $Files is used the wildcard expansion results in a list of all "*.html" files in the current directory and all "*.png" files in the "images" directory.

The second difference is that there is no specific dependency for the "index.html" file but a :rule command. It looks very much the same, but the word "index" has been replaced by a percent character. You could read the rule command as a dependency where the "%" stands for "anything". In the example the target is "anything.html" and in the sources we find "anything_body.part". Obviously these two occurrences of "anything" are the same word.

If you have made HTML pages, you know they contain a title. We ignored that until now. The following recipe will handle a title, stored in the file "xxx_title.part". You also need a file "start.part", which contains the HTML code that goes before the title.

   Files =    *.html
              images/*.png
   :attr {publish = scp://user@ftp.foo.org/public_html/%file%} $Files

   all: $Files

   publish: $Files
       :publishall

   :rule %.html : start.part %_title.part header.part %_body.part footer.part
       :cat $source >! $target

Notice that "%" is now used three times in the :rule command. It stands for the same word every time.

After writing this recipe you can forget what changes you made to what file. A-A-P will take care of generating and uploading those HTML files that are affected. For example, if you change "header.part", all the HTML files are generated and uploaded. If you change "index_title.part" only "index.html" will be done.

There is one catch: You must create an (empty) xxx.html file the first time, otherwise it will not be found with "*.html". And you have to be careful not to have other "xxx.html" files in this directory. You might want to explicitly specify all the HTML files instead of using wildcards.

A similar recipe is actually used to update the A-A-P website. It's a bit more complicated, because not all pages use the same header.

Chapter 4. Distributing a Program

Open source software needs to be distributed. This chapter gives a simple example of how you can upload your files and make it easy for others to download and install your program.

Downloading

To make it easy for others to obtain the latest version of your program, you give them a recipe. That is all they need. In the recipe you describe how to download the files and compile the program. Here is an example:

1     Origin = ftp://ftp.mysite.org/pub/theprog
2
3     :recipe {fetch = $Origin/main.aap}
4
5     Source = main.c
6              version.c
7     Header = common.h
8
9     :attr {fetch = $Origin/%file%} $Source $Header
10
11    :program theprog : $Source

The first line specifies the location where all the files can be found. It is good idea to specify this only once. If you would use the text all over the recipe it is more difficult to read and it would be more work when the URL changes.

Line 3 specifies where this recipe can be obtained. After obtaining this recipe once, it can be updated with a simple command:

    % aap refresh
    Aap: Updating recipe "main.aap"
    Aap: Attempting download of "ftp://ftp.mysite.org/pub/theprog/main.aap"
    Aap: Downloaded "ftp://ftp.mysite.org/pub/theprog/main.aap" to "/home/mool/.aap/cache/98092140.aap"
    Aap: Copied file from cache: "main.aap"
    %

The messages from Aap are a bit verbose. This is just in case the downloading is very slow, you will have some idea of what is going on.

Lines 5 to 7 define the source files. This is not different from the examples that were used to compile a program, except that we explicitly mention the header file used.

Line 9 specifies where the files can be fetched from. This is done by giving the source and header files the fetch attribute. The :attr command does not cause the files to be fetched yet. When a file is used somewhere and it has a fetch attribute, then it is fetched. Thus files that are not used will not be fetched.

A user of your program stores this recipe as main.aap and runs aap without arguments. What will happen is:

1. Dependencies will be created by the :program command to build "theprog" from main.c and version.c.

2. The target "theprog" depends on main.c and version.c. Since these files do not exist and they do have a fetch attribute, they are fetched.

3. The main.c file is inspected for dependencies. It includes the common.h file, which is automatically added to the list of dependencies. Since common.h does not exist and has a fetch attribute, it is fetched as well.

4. Now that all the files are present they are compiled and linked into "theprog".

Uploading

You need to upload the files mentioned in the recipe above. This needs to be repeated each time one of the files changes. This is essentially the same as publishing a web site. You will need to upload both the source files and the recipe itself. The {publish} attribute can be used for this. You can add the following two lines to the recipe above in order to upload all the files:

    URL = scp://user@ftp.mysite.org//pub/theprog/%file%
    :attr {publish = $URL} $Source $Header main.aap

Now you can use aap publish to upload your source files as well.

Chapter 5. Building Variants

A-A-P provides a way to build two variants of the same application. You just need to specify what is different about them. A-A-P will then take care of putting the resulting files in a different directory, so that you don't have to recompile everything when you toggle between two variants.

For the details see :variant in the reference manual.

One Choice

Quite often you want to compile an application for release with maximal optimizing. But the optimizer confuses the debugger, thus when stepping through the program to locate a problem, you want to recompile without optimizing. Here is an example:

1    Source = main.c version.c gui.c
2
3    :variant Build
4        release
5            OPTIMIZE = 4
6            Target = myprog
7        debug
8            DEBUG = yes
9            Target = myprogd
10
11   :program $Target : $Source

Write this recipe as "main.aap" and run Aap without arguments. This will build "myprog" and use a directory for the object files that ends in "-release". The release variant is the first one mentioned, that makes it the default choice.

The first argument for the :variant command is Build. This is the name of the variable that specifies what variant will be selected. The names of the alternatives are specified with a bit more indent in lines 4 and 7. For each alternative two commands are given, again with more indent. Note that the indent not only makes it easy for you to see the parts of the :variant command, they are essential for Aap to recognize them.

To select the "debug" variant the Build variable must be set to "debug". A convenient way to do this is by specifying this on the command line:

    % aap Build=debug

This will build the "myprogd" program for debugging instead of for release. The DEBUG variable is recognized by Aap. The object files are stored in a directory ending in "-debug". Once you finished debugging and fixed the problem in, for example, "gui.c", running Aap to build the release variant will only compile the modified file. There is no need to compile all the C files, because the object files for the "release" variant are still in the "-release" directory.

Two Choices

You can extend the Build variant with more items, for example "profile". This is useful for alternatives that exclude each other. Another possibility is to add a second :variant command. Let us extend the example with a selection of the user interface type.

1    Source = main.c version.c gui.c
2
3    :variant Build
4        release
5            OPTIMIZE = 4
6            Target = myprog
7        debug
8            DEBUG = yes
9            Target = myprogd
10    
11   Gui ?= motif
12   :variant Gui
13       console
14       motif
15            Source += motif.c
16       gtk
17            Source += gtk.c
18
19   DEFINE += -DGUI=$Gui
20
21   :program $Target : $Source

The :variant command in line 12 uses the Gui variable to select one of "console", "motif" or "gtk". Together with the earlier :variant command this offers six alternatives: "release" with "console", "debug" with "console", "release" with "motif", etc. To build "debug" with "gtk" use this command:

    % aap Build=debug Gui=gtk

In line 11 an optional assignment "?=" is used. This assignment is skipped if the Gui variable already has a value. Thus if Gui was given a value on the command line, as in the example above, it will keep this value. Otherwise it will get the value "motif".

In line 15, 17 and 19 the append assignment "+=" is used. This appends the argument to an existing variable. A space is inserted if the value was not empty. For the variant "motif" the result of line 15 is that Source becomes "main.c version.c gui.c motif.c".

The "motif" and "gtk" variants each add a source file in line 15 and 17. For the console version no extra file is needed. The object files for each combination of variants end up in a different directory. Ultimately you get object files in each of the six directories ("SYS" stands for the platform being used):

See the user manual for more examples of using variants.

Chapter 6. Using Python

In various places in the recipe Python commands and expressions can be used. Python is a powerful and portable scripting language. In most recipes you will only use a few Python items. But where needed you can do just about anything with it.

Conditionals

When a recipe needs to work both on Unix and on MS-Windows you quickly run into the problem that the compiler does not use the same arguments. Here is an example how you can handle that.

      @if OSTYPE == "posix":
          INCLUDE += -I/usr/local/include
      @else:
          INCLUDE += -Ic:/vc/include

      all:
          :print INCLUDE is "$INCLUDE"

The first and third line start with the "@" character. This means a Python command follows. The other lines are normal recipe lines. You can see how these two kinds of lines can be mixed.

The first line is a simple "if" statement. The OSTYPE variable is compared with the string "posix". If they compare equal, the next line is executed. When the OSTYPE variable has a different value the line below @else: is executed. Executing this recipe on Unix:

    % aap
    INCLUDE is "-I/usr/local/include"
    %

OSTYPE has the value "posix" only on Unix and Unix-like systems. Executing the recipe on MS-Windows, where OSTYPE has the value "mswin":

    C:> aap
    INCLUDE is "-Ic:/vc/include"
    C:>

Note that the Python conditional commands end in a colon. Don't forget to add it, you will get an error message! The indent is used to form blocks, thus you must take care to align the "@if" and "@else" lines.

You can include more lines in a block, without the need for extra characters, such as { } in C:

      @if OSTYPE == "posix":
          INCLUDE += -I/usr/local/include
          LDFLAGS += -L/usr/local
      @else:
          INCLUDE += -Ic:/vc/include
          LDFLAGS += -Lc:/vc/lib

Scope

In Aap commands a variable without a scope is searched for in other scopes. Unfortunately, this does not happen for variables used in Python. To search other scopes you need to prepend "_no." before the variable name. Changing the above example to print the result from Python:

      @if OSTYPE == "posix":
          INCLUDE += -I/usr/local/include
      @else:
          INCLUDE += -Ic:/vc/include

      all:
          @print 'INCLUDE is "%s"' % _no.INCLUDE

Loops

Python has a "for" loop that is very flexible. In a recipe it is often used to go over a list of items. Example:

1      @for name in [ "solaris", "hpux", "linux", "freebsd" ]:
2          fname = README_$name
3          @if os.path.exists(fname):
4              Files += $fname
5      all:
6          :print $Files

The first line contains a list of strings. A Python list uses square brackets. The lines 2 to 4 are executed with the name variable set to each value in the list, thus four times. The indent of line 5 is equal to the @for line, this indicates the "for" loop has ended.

Note how the name and fname variables are used without a dollar in the Python code. This might be a bit confusing at first. Try to remember that you only put a dollar before a variable name in the argument of a recipe command.

In line 2 the fname variable is set to "README_" plus the value of name. The os.path.exists() function in line 3 tests if a file exists. Assuming all four files exist, this is the result of executing this recipe:

    % aap
    README_solaris README_hpux README_linux README_freebsd
    %

Python Block

When the number of Python lines gets longer, the "@" characters become annoying. It is easier to put the lines in a block. Example:

    :python
        Files = ''
        for name in [ "solaris", "hpux", "linux", "freebsd" ]:
            fname = "README_" + name
            if os.path.exists(fname):
                if Files:
                    Files = Files + ' '
                Files = Files + fname
    all:
        :print $Files

This does the same thing as the above recipe, but now using Python commands. As usual, the :python block ends where the indent is equal to or less than that of the :python line.

When using the :python command, make sure you get the assignments right. Up to the "=" character the Python assignment is the same as the recipe assignment, but what comes after it is different.

Expressions

In many places a Python expression can be used. For example, the glob() function can be used to expand wildcards:

   Source = `glob("*.c")`

Python users know that the glob() function returns a list of items. Aap automatically converts the list to a string, because all Aap variables are strings. A space is inserted in between the items and quotes are added around items that contain a space.

Why use glob() when you can use wildcards directly? The difference is that the expansion with glob() takes place immediately, thus $Source will get the expanded value. When using wildcards directly the expansion is done when using the variable, but that depends on where it is used. For example, the :print command does not do wildcard expansion:

   pattern = *.c
   expanded = `glob(pattern)`
   all:
       :print pattern $pattern expands into $expanded

When "foo.c" and "bar.c" exist, the output will be:

    % aap
    pattern *.c expands into foo.c bar.c
    %

The following example turns the list of source files into a list of header files:

    Source = `glob("*.c")`
    Header = `sufreplace(".c", ".h", Source)`
    all:
        :print Source is "$Source"
        :print Header is "$Header"

Running Aap in a directory with "main.c" and "version.c"?

    % aap
    Source is "version.c main.c"
    Header is "version.h main.h"
    %

The sufreplace() function takes three arguments. The first argument is the suffix which is to be replaced. The middle argument is the replacement suffix. The last argument is the name of a variable that is a list of names, or a Python expression. In this example each name in Source ending in ".c" will be changed to end in ".h".

Further Reading

The User manual Chapter 21 has more information. Documentation about Python can be found on its web site: http://www.python.org/doc/

Chapter 7. Version Control with CVS

CVS is often used for development of Open Source Software. A-A-P provides facilities to obtain the latest version of an application and for checking in changes you made.

Downloading (Checkout)

For downloading a whole module you only need to specify the location of the CVS server and the name of the module. Here is an example that obtains the A-A-P Recipe Executive:

   CVSROOT = :pserver:anonymous@cvs.a-a-p.sf.net:/cvsroot/a-a-p
   all:
        :fetch {fetch = cvs://$CVSROOT} Exec

Write this recipe as "main.aap" and run aap. The directory "Exec" will be created and all files in the module obtained from the CVS server:

    % aap
    Aap: CVS checkout for node "Exec"
    Aap: cvs -d:pserver:anonymous@cvs.a-a-p.sf.net:/cvsroot/a-a-p checkout 'Exec'
    cvs server: Updating Exec
    U Exec/Action.py
    U Exec/Args.py
    [....]
    %

If there is a request for a password just hit enter (mostly there is no password).

The :fetch command takes care of obtaining the latest version of the items mentioned as arguments. Usually the argument is one module, in this example it is "Exec". That CVS needs to be used is specified with the fetch attribute. This is a kind of URL, starting with "cvs://" and then the CVS root specification. In the example the CVSROOT variable was used. This is not required, it just makes the recipe easier to understand.

If the software has been updated, you can get the latest version by running "aap" again. CVS will take care of obtaining the changed files.

Note that all this only works when you have the "cvs" command installed. When it cannot be found Aap will ask you want Aap to install it for you. Whether this works depends on your system.

Getting Past A Firewall

Firewalls may block the use of a CVS connection. Some servers have setup another way to connect, so that firewalls will not cause problems. This uses port 80, normally used for http connections. Here is the above example using a different "pserver" address:

   CVSROOT = :pserver:anonymous@cvs-pserver.sf.net:80/cvsroot/a-a-p
   all:
        :fetch {fetch = cvs://$CVSROOT} Exec

This doesn't always work through a proxy though. If you have problems connecting to the CVS server, try reading the information at this link.

Uploading (Checkin)

You are the maintainer of a project and want to distribute your latest changes, so that others can obtain the software with a recipe as used above. This means you need to checkin your files to the CVS server. This is done by listing the files that need to be distributed and giving them a commit attribute. Example:

   CVSUSER_FOO = johndoe
   CVSROOT = :ext:$CVSUSER_FOO@cvs.foo.sf.net:/cvsroot/foo
   Files =  main.c
            common.h
            version.c
   :attr {commit = cvs://$CVSROOT} $Files

Write this as "cvs.aap" and run aap -f cvs.aap revise . What will happen is:

1. Files that you changed since the last checkin will be checked in to the CVS server.

2. Files that you added to the list of files with a commit attribute will be added to the CVS module.

3. Files that you removed from the list of files with a commit attribute will be removed from the CVS module.

Further Reading

The User manual Chapter 18 has more information about version control and Chapter 19 about using CVS.

Chapter 8. Filetypes and Actions

A-A-P can recognize what the type of a file is, either by looking at the file name or by inspecting the contents of the file. The filetype can then be used to decide how to perform an action with the file.

A New Type of File

Suppose you are using the "foo" programming language and want to use A-A-P to compile your programs. Once this is has been setup you can compile "hello.foo" into the "hello" program with a simple recipe:

    :program hello : hello.foo

You need to explain Aap how to deal with "foo" files. This is done with a recipe:

    :filetype
        suffix foo foo

    :action compile foo
        :sys foocomp $?FOOFLAGS $source -o $target

    :route  foo object
        compile

For Unix, write this recipe as "/usr/local/share/aap/startup/foo.aap" or "~/.aap/startup/foo.aap". The recipes in these "startup" directories are always read when Aap starts up.

Now try it out, using the simple recipe at the top as "main.aap":

    % aap
    Aap: foocomp hello.foo -o build-FreeBSD4_5_RELEASE/hello.o
    Aap: cc -L/usr/local/lib -g -O2 -o hello build-FreeBSD4_5_RELEASE/hello.o
    %

The "foo.aap" recipe does three things:

1. The :filetype command is used to tell A-A-P to recognize your "hello.foo" file as being a "foo" file.

2. The :action command is used to specify how the "foocomp" compiler is used to compile a "foo" program into an object file. The user can set the FOOFLAGS variable to options he wants to use. The convention is that the option variable is in uppercase, starts with the filetype and ends in "FLAGS".

3. The :route command is used to specify which actions are to be used to turn a "foo" file into an "object" file.

Defining a Filetype by Suffix

The :filetype command is followed by the line "suffix foo foo". The first word "suffix" means that recognizing is done by the suffix of the file name (the suffix is what comes after the last dot in the name). The second word is the suffix and the third word is the type. Quite often the type is equal to the suffix, but not always. Here are a few more examples of lines used with :filetype:

    :filetype
        suffix fooh foo
        suffix bash sh

It is also possible to recognize a file by matching the name with a pattern, checking the contents of the file or using a Python script. See the user manual.

Defining a Compile Action

The lower half of "foo.aap" specifies the compile action for the "foo" filetype:

    :action compile foo
        :sys foocomp $source -o $target

The :action command has two arguments. The first one specifies the kind of action that is being defined. In this case "compile". This action is used to make an object file from a source file. The second argument specifies the type of source file this action is used for, in this case "foo".

Below the :action line the build commands are specified. In this case just one, there could be more. The :sys command invokes an exteral program, "foocomp", and passes the arguments. In an action $source is expanded to the source of the action and $target to the target. These are obtained from the :do command that invokes the action. Example:

    :do compile {target = `src2obj("main.foo")`} main.foo

This :do command invokes the compile action, specified with its first argument. The target is specified as an attribute to the action, the source is the following argument "main.foo". When executing the :do command the filetype of "main.foo" is detected to be "foo", resulting in the compile action for "foo" to be invoked. In the build command of the action $source and $target are replaced, resulting in:

    :sys foocomp main.foo -o `src2obj("main.foo")`

Note that in many cases $target is passed implicitly from a dependency and does not appear in the :do command argument.

Another Use of Filetypes

When building a program you often want to include the date and time when it was built. A simple way of doing this is creating a source file "version.c" that contains the timestamp. This file needs to be compiled every time your program is built. Here is an example how this can be done:

1   :program prog : main.c work.c
2
3   :attr prog {filetype = myprog}
4
5   :action build myprog object
6        version_obj = `src2obj("version.c")`
7        :do compile {target = $version_obj} version.c
8        :do build {filetype = program} $source $version_obj

The target "prog" is explicitly given a different filetype in line 3. The default filetype for a program is "program", here it is set to "myprog". This allows us to specify a different build action for "prog".

Write the recipe as "main.aap" (without the line numbers) and execute it with aap. The first time all the files will be compiled and linked together. Executing aap again will do nothing. Thus the timestamp used in "version.c" will not be updated if the files were not changed. If you now make a change in "main.c" and run aap you will see that both "main.c" and "version.c" are compiled.

The :action command in line 5 has three arguments. The first one "build" is the kind of action, like before. The second argument "myprog" specifies the target filetype, the third one "object" the source filetype. Thus the template is:

    :action  kind-of-action  target-filetype  source-filetype

For more information about customizing filetype detection and actions see Chapter 28.

Chapter 9. More Than One Recipe

When you are working on a project that is split up in several directories it is convenient to use one recipe for each directory. There are several ways to split up the work and use a recipe from another recipe.

Children

A large program can be split in several parts. This makes it easy for several persons to work in parallel. You then need to allow the files in each part to be compiled separately and also want to build the complete program. A convenient way to do this is putting files in separate directories and creating a recipe in each directory. The recipe at the top level is called the parent. Here is an example that includes two recipes in subdirectories, called the children:

1    :child core/main.aap       # sets Core_obj
2    :child util/main.aap       # sets Util_obj
3
4    :program theprog : core/$*Core_obj util/$*Util_obj

In the first two lines the child recipes are included. These specify how the source files in each directory are to be compiled and assign the list of object files to Core_obj and Util_obj. This parent recipe then defines how the object files are linked together to build the program "theprog".

In line 4 a special mechanism is used. Assume that Core_obj has the value "main.c version.c". Then "core/$*Core_obj" will expand into "core/main.c core/version.c". Thus "core/" is prepended to each item in Core_obj. This is called rc-style expansion. You can remember it by thinking of the "*" to multiply the items.

An important thing to notice is that the parent recipe does not need to know what files are present in the subdirectories. Only the child recipes contain the list of files. Thus when a file is added, only one recipe needs to be changed. The "core/main.aap" recipe contains the list of files in the "core" directory:

1    Source =  main.c
2              version.c
3
4    CPPFLAGS += -I../util
5
6    _top.Core_obj = `src2obj(Source)`
7
8    all: $_top.Core_obj

Variables in a child recipe are local to that recipe. The CPPFLAGS variable that is changed in line 4 will remain unchanged in the parent recipe and other children. That is desired here, since finding header files in "../util" is only needed for source files used in this recipe.

The Core_obj variable we do want to be available in the parent recipe. That is done by prepending the "_top" scope name. The generic way to use a scope is:

         {scopename} . {variablename}

Several scope names are defined, such as "_recipe" for the current recipe and "_top" for the toplevel recipe. The full list of scope names can be found in the reference manual, chapter "Recipe Syntax and Semantics". When a variable is used without a scope name, it is looked up in the local scope and surrounding scopes. Thus the variables from the parent recipe are also available in the child. But when assigning to a variable without a scope, it is always set in the local scope only. To make the variable appear in another scope you must give the scope name.

The value of Core_obj is set with a Python expression. The src2obj() function takes a list of source file names and transforms them into object file names. This takes care of changing the files in Source to prepend $BDIR and change the file suffix to $OBJSUF. It also takes care of using the "var_BDIR" attribute if it is present.

In the last line is specified what happens when running aap without arguments in the "core" directory: The object files are built. There is no specification for how this is done, thus the default rules will be used.

All the files in the child recipe are defined without mentioning the "core" directory. That is because all parent and child recipes are executed with the current directory set to where the recipe is. Note the files in Core_obj are passed to the parent recipe, which is in a different directory. That is why the parent recipe had to prepend "core/" when using Core_obj. This is so that the child recipe doesn't need to know what its directory name is, only the parent recipe contains this directory name.

Sharing Settings

Another mechanism to use a recipe is by including it. This is useful to put common variables and rules in a recipe that is included by several other recipes. Example:

    CPPFLAGS += -DFOOBAR
    :rule %$OBJSUF : %.foo
        :sys foocomp $source -o $target

This recipe adds something to CPPFLAGS and defines a rule to turn a ".foo" file into an object file. Suppose you want to include this recipe in all the recipes in your project. Write the above recipe as "common.aap" in the top directory of the project. Then in "core/main.aap" and "util/main.aap" put this command at the top:

    :include ../common.aap

The :include command works like the commands in the included recipe were typed instead of the :include command. There is no change of directory, like with the :child command and the included recipe uses the same scope.

In the toplevel recipe you need include "common.aap" as well. Suppose you include it in the first line of the recipe, before the :child commands. The children also include "common.aap". The CPPFLAGS variable would first be appended to in the toplevel recipe, then passed to the child and appended to again. That is not what is supposed to happen.

To avoid this, add the {once} option to the :include command. This means that the recipe is only included once and not a second time. The child recipes use:

    :include {once} ../common.aap

And the parent uses:

1    :include {once} common.aap
2    :child core/main.aap       # sets Core_obj
3    :child util/main.aap       # sets Util_obj
4
5    all: theprog$EXESUF
6
7    theprog$EXESUF : core/$*Core_obj util/$*Util_obj
8        :do build $source

You might argue that another way would be to put the :include command at the top of the parent recipe, so that the children don't have to include "common.aap". You could do this, but then it is no longer possible to execute a child recipe by itself.

Note that using :include like this will always use the _top scope for the variables set in the included recipe. Be careful that the _recipe scope isn't used in one of the child recipes.

Executing a Recipe

Besides :child and :include there is a third way to use another recipe: :execute. This command executes a recipe. This works as if Aap was run as a separate program with this recipe, except that it is possible to access variables in the recipe that has the :execute command. Here is an example:

    :program prog : main.c common.c

    test:
        :execute test.aap test
        :print $TestResult

This recipe uses the :program command as we have seen before. This takes care of building the "prog" program. For testing a separate recipe is used, called "test.aap". The first argument of the :execute command is the recipe name. Further arguments are handled like the arguments of the aap command. In this case the target "test" is used.

The "test.aap" recipe sets the TestResult variable to a message that summarizes the test results. To get this variable back to the recipe that executed "test.aap" the "_parent" scope is used:

    @if all_done:
        _parent.TestResult = All tests completed successfully.
    @else:
        _parent.TestResult = Some tests failed!

It would also be possible to use the :child command to reach the "test" target in it. The main difference is that other targets in "test.aap" could interfere with targets in this recipe. For example, "test.aap" could define a different "prog" target, to compile the program with specific test options. By using :execute we don't need to worry about this. In general, the :child command is useful when splitting up a tree of dependencies in parts, while :execute is useful for two tasks that have no common dependencies.

Fetching a Recipe

So far we assumed the included recipes were stored on the local system. It is also possible to obtain them from elsewhere. The example with children above can be extended like this:

1    Origin = ftp://ftp.foo.org/recipes
2    :include {once} common.aap {fetch = $Origin/common.aap}
3    :child core/main.aap {fetch = $Origin/core.aap}
4    :child util/main.aap {fetch = $Origin/util.aap}
5
6    all: theprog$EXESUF
7
8    theprog$EXESUF : core/$*Core_obj util/$*Util_obj
9        :do build $source

The fetch attribute is used to specify the URL where the recipe can be obtained from. This works just like fetching source files. Notice in the example that the file name in the URL can be different from the local file name. When Aap reads this recipe and discovers that a child or included recipe does not exist, it will use the fetch attribute to download it. The fetch attribute can also be used with the :execute command.

Once a recipe exists locally it will be used, even when the remote version has been updated. If you explicitly want to get the latest version of the recipes used, run aap -R or aap fetch.

Chapter 10. Commands in a Pipe

A selection of commands can be connected together with a pipe. This means the output of one command is the input for the next command. It is useful for filtering text from a variable or file and writing the result in a variable or file.

Changing a timestamp

This example shows how you can change the timestamp in a file. It is done in-place.

    all:
        :print Setting date in foobar.txt.
        :cat foobar.txt
              | :eval re.sub('Last Change: .*\n', 'Last Change: ' + DATESTR + '\n', stdin)
              >! foobar.txt

Lets see how this works:

    % cat foobar.txt
    This is example text for the A-A-P tutorial.
    Last Change: 2002 Feb 29
    The useful contents would start here.
    % aap
    Setting date in foobar.txt.
    % cat foobar.txt
    This is example text for the A-A-P tutorial.
    Last Change: 2002 Oct 21
    The useful contents would start here.
    %

The last command in the example consists of three parts. First comes the :cat command. It reads the "foobar.txt" file and passes it throught the pipe to the next command. "cat" is short for "concatenate". This is one of the good-old Unix commands that actually does much more than the name suggests. In this example nothing is concatenated. Below you will see examples where it does.

The second part of the example is the :eval command. This is used to read the text coming in through the pipe and modify it with a Python expression. In this case the expression is a "re.sub()" function call. This Python function takes three arguments: A pattern, a replacement string and the text to operate on. All occurences of the pattern in the text are changed to the replacement string. The pattern "Last Change: .*\n" matches a line with the date that was inserted previously. The replacement string contains DATESTR, which is an Aap variable that contains today's date as a string, e.g., "2002 Oct 19". The text to operate on is stdin. This is the variable that holds the text that is coming in through the pipe.

The third and last part >! foobar.txt redirects the output of the :eval command back to the file "foobar.txt". Using just ">" would cause an error, since the file already exists.

Note that in a Unix shell command this pipe would not work: The "foobar.txt" would be overwritten before it was read. In Aap this does not happen, the commands in the pipe are executed one by one. That makes it easier to use, but it does mean the text is kept in memory. Don't use pipes for a file that is bigger than half the memory you have available.

Changing a file in-place has the disadvantage that the normal dependencies don't work, since there is no separate source and target file. Often it is better to use a file "foobar.txt.in" as source, change it like in the example above and write it as a new file. The recipe would be:

    foobar.txt: foobar.txt.in
        :print Setting date in $target.
        :cat $source
                | :eval re.sub('Last Change: .*\n', 'Last Change: ' + DATESTR + '\n', stdin)
                >! $target

Creating a file from pieces

Sometimes you need to generate a file from several pieces. Here is an example that concatenates two files and puts a generated text line in between.

    manual.html: body.html footer.html
        @import time
        :eval time.strftime("%A %d %B %Y", time.localtime(time.time()))
                 | :print $(lt)BR$(gt)Last updated: $stdin$BR
                 | :cat body.html - footer.html >! $target

There are quite a few items here that need to be explained. First of all, the "@import time" line. This is a Python command to load the "time" module. So far we used modules that Aap has already loaded for you. This one isn't, and since we use the "time" module in the next :eval command it needs to be loaded explicitly.

The Python function "strftime()" formats the date and time in a specified format. See the Python documentation for the details. In this case the resulting string looks like "Monday 21 October 2002".

The output of the :eval command is piped into a :print command. The variable stdin contains the output of the previous command. Note that "$(lt)" is used instead of "$lt". The meaning is exactly the same: the value of the lt variable. Without the extra parenthesis it would read "$ltBR", which would be the value of the "ltBR" variable.

The resulting text is:

        <BR>Last updated: Monday 21 October 2002\n

Note that the first "BR" is the HTML code for a line break, while the "$Br" at the end is the Aap variable that contains a line break (here displayed as "\n").

Finally, the :cat command concatenates the file "body.html", the output of the :print command and the file "footer.html". Thus the "-" stands for where the pipe input is used. The result is redirected to target, which is "manual.html".

Pipe output in a variable

The generated date in the previous example could be used elsewhere in the recipe. Since we don't want to repeat a complicated expression the result of the :eval command should be redirected to a variable, like this:

    @import time
    :eval time.strftime("%A %d %B %Y", time.localtime(time.time()))
                | :assign Datestamp

    manual.html: body.html footer.html
           :print $(lt)BR$(gt)Last updated: $Datestamp$Br
                 | :cat body.html - footer.html >! $target

The :assign command takes the input from the pipe and puts it in the variable mentioned as its argument, which is "Datestamp" here. Actually, the same can be done with a normal assignment and a Python expression in backticks, but we intentionally wanted to show using a pipe here.

Creating a file from scratch

It is also possible to completely generate a file from scratch. Here is an example that generates a C header file:

1    :include config.aap
2    pathdef.c: config.aap
3       :print Creating $target
4       :print >! $target /* pathdef.c */
5       :print >> $target /* This file is automatically created by main.aap */
6       :print >> $target /* DO NOT EDIT!  Change main.aap only. */
7       :print >> $target $#include "vim.h"
8       :print >> $target char_u *default_vim_dir = (char_u *)"$VIMRCLOC";
9       :print >> $target char_u *all_cflags = (char_u *)"$CC -c -I$srcdir $CFLAGS";

The first :print command displays a message, so that it's clear "pathdef.c" is being generated. The next line contains ">!" to overwrite an existing file. It doesn't matter if the file already existed or not, it now only contains the line "/* pathdef.c */". The third and following lines contain ">>". This will cause each line to be appended to "pathdef.c".

In the example the VIMRCLOC and srcdir variables are defined in the recipe "config.aap". That is why this file is used as a source in the dependency. Also note the use of "$#" in line 7. Since "#" normally starts a comment it cannot be used directly here. "$#" is a special item that results in a "#" in the :print output. This is the resulting file:

       /* pathdef.c */
       /* This file is automatically created by main.aap */
       /* DO NOT EDIT!  Change main.aap only. */
       #include "vim.h"
       char_u *default_vim_dir = (char_u *)"/usr/local/share/vim61";
       char_u *all_cflags = (char_u *)"cc -c -I. -g -O2";

The list of ">>" redirections is quite verbose. Fortunately there is a shorter way:

1    :include config.aap
2    pathdef.c: config.aap
3        :print Creating $target
4        text << EOF
5            /* pathdef.c */
6            /* This file is automatically created by main.aap */
7            /* DO NOT EDIT!  Change main.aap only. */
8            $#include "vim.h"
9            char_u *default_vim_dir = (char_u *)"$VIMRCLOC";
10           char_u *all_cflags = (char_u *)"$CC -c -I$srcdir $CFLAGS";
11               EOF
12       :print $text >! $target

In line 4 "text << EOF" is used. This is called a block assignment. The following lines, up to the matching "EOF" line, are assigned to the variable text. You can use something else than "EOF" if you want to. It must be a word that does not appear inside of the text as a line on its own. White space before and after the word is ignored.

The indent of the text in the block assignment is removed. The indent of the first line is used, the same amount of indent is removed from the following lines. Thus if the second line has two more spaces worth of indent than the first line, it will have an indent of two spaces in the result. Half a tab is replace with four spaces when necessary (a tab always counts for up to eight spaces).

Chapter 11. A Ported Application

When an application already exists but for your system it requires a few tweaks, a port recipe can do the work. This can also be used for applications that work fine but you want to apply a number of patches or to add a feature. The recipe can be distributed, so that others can install the application without knowing the details. This works very much like the FreeBSD ports system.

This chapter is specifically for doing the port. If you are only interested in another kind of building you might want to skip this chapter.

The Port Recipe

Since A-A-P is prepared for doing all the work, usually you only need to specify the relevant information, such as where to find the files involved. Here is an example:

 1    # A-A-P port recipe for Vim 6.1 plus a few patches.
 2    RECIPEVERSION =     1.0
 3
 4    PORTNAME =          vim
 5    LASTPATCH =         003
 6    PORTVERSION =       6.1.$LASTPATCH
 7    MAINTAINER =        Bram@vim.org
 8
 9    CATEGORIES =        editors
10    PORTCOMMENT =       Vim - Vi IMproved, the text editor
11    PORTDESCR << EOF
12    This is the description for the Vim package.
13    A very nice editor, backwards compatible to Vi.
14    You can find all info on http://www.vim.org.
15            EOF
16
17    :recipe {fetch = http://www.a-a-p.org/ports/vim/main.aap}
18
19    WRKSRC =            vim61
20    BUILDCMD =          make
21    TESTCMD =           make test
22    INSTALLCMD =        make install DESTDIR=$PKGDIR
23    PREFIX =            /usr/local
24
25    MASTER_SITES ?=     ftp://ftp.vim.org/pub/vim
26                        ftp://ftp.us.vim.org/pub/vim
27    PATCH_SITES =       $*MASTER_SITES/patches
28
29    DISTFILES =         unix/vim-6.1.tar.bz2
30
31    version1 =          `range(1, int(LASTPATCH) + 1)`
32    PATCHFILES =        6.1.00$*version1
33
34    #>>> automatically inserted by "aap makesum" <<<
35    do-checksum:
36            :checksum $DISTDIR/vim-6.1.tar.bz2 {md5 = 7fd0f915adc7c0dab89772884268b030}
37            :checksum $PATCHDISTDIR/6.1.001 {md5 = 97bdbe371953b9d25f006f8b58b53532}
38            :checksum $PATCHDISTDIR/6.1.002 {md5 = f56455248658f019dcf3e2a56a470080}
39            :checksum $PATCHDISTDIR/6.1.003 {md5 = 0e000edba66562473a5f1e9b5b269bb8}
40    #>>> end <<<

Well, that is the longest example we have had so far. Let's go through it from top to bottom.

 1    # A-A-P port recipe for Vim 6.1 plus a few patches.
 2    RECIPEVERSION =     1.0

RECIPEVERSION tells Aap what version of Aap this recipe was written for. If in the future the recipe format changes, this line causes Aap to interpret it as Aap version 1.0 would do.

 4    PORTNAME =          vim

Setting PORTNAME to the name of the port is what actually triggers Aap to read this recipe as a port recipe. It makes the other settings to be used to set up a whole range of targets and build commands. The result is that you can do aap install to install the application, for example. Note that PORTNAME does not include the version number.

 5    LASTPATCH =         003
 6    PORTVERSION =       6.1.$LASTPATCH
 7    MAINTAINER =        Bram@vim.org
 8
 9    CATEGORIES =        editors
10    PORTCOMMENT =       Vim - Vi IMproved, the text editor
11    PORTDESCR << EOF
12    This is the description for the Vim package.
13    A very nice editor, backwards compatible to Vi.
14    You can find all info on http://www.vim.org.
15            EOF

In lines 5 to 15 a number of informative items about the port are specified. These are used in various places. LASTPATCH is not a standard item, it is used here to only have to define the patchlevel in one place.

17    :recipe {fetch = http://www.a-a-p.org/ports/vim/main.aap}

The :recipe command specifies where to obtain the recipe itself from. We have seen this before, nothing special here.

19    WRKSRC =            vim61
20    BUILDCMD =          make
21    TESTCMD =           make test

The assignments in lines 19 to 21 specify how building is to be done. WRKSRC is the directory below which the source files are unpacked. The default is "$PORTNAME-$PORTVERSION". The archive used for Vim uses "vim61" instead, thus this needs to be specified. The "CMD" variables set the commands to be used to build the application. The default is to use Aap. Since Vim uses "make" this needs to be specified.

22    INSTALLCMD =        make install DESTDIR=$PKGDIR
23    PREFIX =            /usr/local

Installing a port is done by creating a binary package and installing that package. This makes it possible to copy the package to another system and install it there without the need to compile from sources. Lines 22 and 23 specify how to do a "fake install" with Vim. This copies all the files that are to be installed to a specific directory, so that it is easy to include them in the package. PREFIX specifies below which directory Vim installs its files.

25    MASTER_SITES ?=     ftp://ftp.vim.org/pub/vim
26                        ftp://ftp.us.vim.org/pub/vim
27    PATCH_SITES =       $*MASTER_SITES/patches

MASTER_SITES and PATCH_SITES specify the sites where the Vim files can be downloaded from. The first is for the archives, the second for the patches. Note the use of "$*" in line 27, this causes "/patches" to be appended to each item in MASTER_SITES instead of appending it once at the end of the whole list.

29    DISTFILES =         unix/vim-6.1.tar.bz2

DISTFILES is set to the name of the archive to download. This is appended to items in MASTER_SITES to form the URL.

31    version1 =          `range(1, int(LASTPATCH) + 1)`
32    PATCHFILES =        6.1.00$*version1

Lines 32 and 33 specify the list of patch file names. The Python function "range()" is used, this returns a list of numbers in the specified range (up to and excluding the upper number). Note the user of "int()" to turn the patch number in LASTPATCH into an int type, all Aap variables are strings.

for three patch files this could also have been typed, but when the number of patches grows this mechanism is easier. The example only works up to patch number 009. To make it work for numbers from 100 up to 999:

      version1 =          `range(1, 10)`
      version2 =          `range(10, 100)`
      version3 =          `range(100, int(LASTPATCH) + 1)`
      PATCHFILES =        6.1.00$*version1  6.1.0$*version2  6.1.$*version3

34    #>>> automatically inserted by "aap makesum" <<<
35    do-checksum:
36            :checksum $DISTDIR/vim-6.1.tar.bz2 {md5 = 7fd0f915adc7c0dab89772884268b030}
37            :checksum $PATCHDISTDIR/6.1.001 {md5 = 97bdbe371953b9d25f006f8b58b53532}
38            :checksum $PATCHDISTDIR/6.1.002 {md5 = f56455248658f019dcf3e2a56a470080}
39            :checksum $PATCHDISTDIR/6.1.003 {md5 = 0e000edba66562473a5f1e9b5b269bb8}
40    #>>> end <<<

Finally the "do-checksum" target is defined. This part was not typed, but added to the recipe with aap makesum. This is done by the port recipe maintainer, when he has verified that the files are correct. When a user later uses the recipe Aap will check that the checksums match, so that problems with downloading or a cracked distribution file are found and reported.

Using CVS

The port recipe specifies which source files and patches to download, thus it has to be adjusted for each version. This is good for a stable release, but when you are releasing a new version every day it is a lot of work. Another method is possible when the files are available from a CVS server. Adding these lines to the recipe will do it:

      CVSROOT ?=        :pserver:anonymous@cvs.vim.sf.net:/cvsroot/vim
      CVSMODULES =      vim
      CVSTAG =          vim-6-1-$LASTPATCH

The first line specifies the cvsroot to use. This is specific for the cvs program. CVSMODULES is the name of the module to checkout. Mostly it is just one name, but you can specify several. Specifying CVSTAG is optional. If it is defined, like here, a specific version of the application is obtained. When it is omitted the latest version is obtained.

Much more about the port recipe can be found in Chapter 22.

Chapter 12. How it all works

How Recipes Are Executed

Executing recipes is a two step process:

1. recipe processing

   Read and parse the toplevel recipe, child recipes and included recipes. Commands at the recipe level are executed. Build commands (commands for dependencies, rules, actions, etc.) are stored.

2. target building

   Build each of the specified targets, following dependencies. Build commands are executed.

      :print executed during the first step

      target1 : source1 source2
            :print executed during the second step

Common Recipe Structure

A recipe used for building an application often has these parts:

1. global settings, include recipes with project and/or user settings

2. automatic configuration

3. specify variants (e.g., debug/release)

4. build rules and actions

5. explicit dependencies

6. high level build commands ( :program, :dll, etc.)

Now let us look into each part in more detail.

1. global settings, include recipes with project and/or user settings

   When the recipe is part of a project, it's often useful to move settings (and rules) that apply to the whole project to one file. Then use the :include command in every recipe that can be used to build something.

   User preferences (e.g. configuration choices) should be in a separate file that the user edits (using a template).

2. automatic configuration

   Find out properties of the system and handle user preferences. This may result in building the application in a different way. See Chapter 24.

3. specify variants

   Usually debug and release, but can include many more choices (type of GUI, small or big builds, etc.). This changes the value of BDIR. See Chapter 14.

4. build rules and actions

   Rules that define dependencies and build commands that apply to several files, defined with :rule commands. Actions can be defined for what is not included in the default actions or to overrule the defaults actions to do a different way of building.

5. explicit dependencies

   Dependencies and build commands that apply to specific files. Use these where the automatic dependency checking doesn't work and for exceptions.

6. high level build commands

   :program, :dll, etc. can be used for standard programs, libraries, etc. This comes last, so that explicitly defined dependencies for building some of the items can be used.

For larger projects sections can be moved to other recipes. How you want to do this depends on whether these sub-recipes need to be executed by themselves and who is going to maintain each recipe. More about that below.

Building A Target In The First Step

Since commands at the recipe level are executed in the first step, some building may already be done. Especially the :update command gives you a powerful mechanism. This means you can already build a target halfway the first step. Note that only dependencies that have already been encountered will be used then.

A good use for the :update command at the recipe level is to generate a recipe that you want to include. Useful for automatic configuration. You would do something like this:

      config.aap : config.aap.in
          :print executing the configuration script...
          :sys ./conf.sh < $source > $target

      :update config.aap
      :include config.aap

First a dependency is specified with build commands for the included recipe. In this case the "config.aap.in" file is used as a template. The command :update config.aap invokes building "config.aap". If it is outdated (config.aap.in was changed since config.aap was last build) the build commands are executed. If "config.aap" is up-to-date nothing happens. Then the :include config.aap includes the up-to-date "config.aap" recipe.

Nesting The Steps

In the second step commands of dependencies are executed. One of these commands may be :execute. This means another recipe is read and targets are build. These are again the first and second step mentioned before, but now nested inside the second step. Here is an example that executes a recipe when "docfile.html" is to be build:

      docfile.html :
          :execute docs/main.aap $target

This construction is useful when you do not want to read the other recipe in the first step. Either because it is a large recipe that is not always needed, because the recipe does not always exist, or because the recipe must first be build by other commands. Here is an example of using a depencency on a recipe:

      docfile.html : docs/main.aap
          :execute docs/main.aap $target

      docs/main.aap: docs/main.aap.in
          :cd docs
          :sys ./conf.sh < main.aap.in > main.aap

The :execute command can also be used at the recipe level. This means another recipe is executed during the first step. A good example for this is building an application in different variants:

      # build the GTK version
      :execute main.aap Gui=GTK myprog
      :move myprog myprog-GTK

      # build the Motif version
      :execute main.aap Gui=Motif myprog
      :move myprog myprog-Motif

Using Multiple Recipes

There are many ways to split up a project into multiple recipes. If you are building one application, you mostly build the whole application, using a toplevel recipe. This recipe specifies the configuration, specifies variants and sets variables for choices. Separate recipes are used to handle specific tasks. For example, you can move related sources to a sub-directory and put a recipe in that directory to build those sources. For this situation you use the :child command.

When a project gets bigger, and especially when working together with several people, you may want to be able to split the project up in smaller pieces, which each can be build separately. To avoid replicating commands, you should put the configuration, variants and setting variables in a separate recipe. Each recipe can use the :include command to use this recipe. You need to take care that the recipe is not included twice, because commands like :route give an error when repeated and appending to variables must only be done once. Aap will read a recipe only the first time it is included when you add the {once} argument to the :include command.

Recipe Execution Details

The two-step processing of recipes is part of all the work that Aap does. There are a few other steps. This is what happens when Aap is run:

1. Read the startup recipes, these define default rules and variables. These recipes are used:

   default.aap from the distribution

   all recipes matching /usr/local/share/aap/startup/*.aap

   all recipes matching ~/.aap/startup/*.aap

2. Recipe processing: Read the recipe main.aap or the one specified with the "-f" argument and check for obvious errors. Then execute the toplevel items in the recipe. Dependencies and rules are stored. Also read included and child recipes and execute the toplevel items in them.

3. Apply the clever stuff to add missing dependencies and rules. This adds a "clean" rule only if the recipe didn't specify one, for example.

4. Target building. The first of the following that exists is used:

   targets specified on the command line

   items specified with :program, :dll and :lib

   the "all" target

5. If the "finally" target is specified, execute its build commands. Each recipe can have its own "finally" target, they are all executed.

Use Of Variables

Variables with uppercase letters are generally used to pass choices and options to actions. For example, $CC is the name of the C compiler and $CFLAGS optional arguments for the C compiler. The list of predefined variables is in the reference manual here.

To avoid clashing with an existing or future variable that is defined by Aap, use one or more lower case letters or prepend "MY". Examples:

        $n
        $sources
        $FooFlags
        $MYPROG

Also be careful with chosing a name for a user scope, it must be different from all variables used in recipes! Prepending "s_" is recommended. Examples:

        $s_debug.CFLAGS
        $s_ovr.msg

Special Characters

Some characters in expressions have a special meaning. And a command like :print also handles a few arguments in a special way. This table gives an overview of which characters you need to watch out for when using the :print command:

Example:

    all:
        :print tie $(#)2 $(`)green$(`) $(|) price: $($) 13 $(<) incl vat $(>)

Write this in the file "try.aap". Executing it results in:

    % aap -f try.aap
    tie #2 `green` | price: $ 13 < incl vat >
    %

Line Syntax

Aap parses the recipe into a sequence of lines. A line is a sequence of characters terminated by a newline. You can escape the newline with a backslash to continue a logical line over more than one physical line, as follows:

1 One line 2 A longer line \ 3 that continues \ 4 over three physical lines.

You can always use backslash continuations to continue lines in Aap. Indentation does not matter.

In many constructions, Aap also supports Python-style line continuations, where a line is continued by increasing the indentation of subsequent physical lines. The above example would look different with Python-style continuation:

1 One line 2 A longer line 3 that continues 4 over three physical lines.

As you can see, the "block" of lines with an increased amount of indentation is considered to belong to the line above it.

Python-style line continuations are supported in all Aap constructions except when the command cannot be recognized if the linebreak comes early. For example, in dependencies the colon separating the targets from the sources cannot be in a continuation line. This does not work:

    myprog
            : mysource
        :print This Does Not Work!

It is also not possible to split a dependency by indent when it does not have build commands:

    myprog :
          mysource
    this = Does Not Work

You must use a backslash in this situation:

    myprog : \
          mysource
    this = OK

Chapter 13. Dependencies, Rules and Actions

Build Commands

There are several methods to specify build commands to update a target:

1. A dependency

   This is more or less the same as how this is used in a Makefile: One or more targets, a colon and any number of sources. This specifies that the target(s) depends on the source(s). When build commands are given these are the commands to build the target(s) from the source(s). Without build commands the dependency is only used to check if the target is outdated and needs to be build.

2. A rule

   Specified with a :rule command. A "%" in the target(s) and source(s) stands for any string. This is used to specify a dependency that is to be used for files that match the pattern.

3. An action

   Specified with a :action command. Unlike dependencies and rules an action does not specify a build dependency. It must be invoked by other build commands with the :do command.

Nearly all recipe commands can be used in the build commands. But these are not allowed, they can only be used at the recipe level:

The Production Commands

The commands :program, :lib, :dll and :ltlib are called production commands because they explicitly state what things Aap should produce and what sources are involved. Everything the production commands can do, can be done by hand with dependencies as well, but the automation the production commands provide is quite useful. This section discusses how the production commands can be used and the variables that affect them.

The form of each of the production commands is :command targets : sources. It is unusual to have more than one target, since both targets would be built from the same sources, but it is allowed. The list of sources should list the actual, original sources, i.e. only files that are actually written by the programmer and that exist on disk. It is these sources that will be packaged together for distributing the program or library in source form.

Each production command transforms all of the sources into objects using compile actions. The sources are transformed into object files of a particular type — e.g. libraries use files with type "libobject". Once all of the sources have been compiled, a build action is invoked to turn the object files into the target. The table below lists the production commands and the actions used.

Some of the production commands can use different programs to produce the final product, depending on settings in the recipe. In particular, you may need to chose to link a program with the compiler or through libtool, depending on whether your program links to any libtool libraries or not. The alternatives are listed in the table below as well. To select an alternative form to build the final product, set the filetype of the target to a specific value, e.g. 

:program myProgram { filetype=ltprogram } : source.c

This example uses the ltprogram alternative build command to build the program "myProgram."

In case you do want to have Aap figure out how to turn source files in to objects and then combine them into a target, but the target is not one of the types mentioned above, you can use the :produce command.

Rules And Dependencies

When a target is to be build Aap first searches for an explicit dependency with build commands that produces the target. This dependency may come from a high level build command such as :program. When such a dependency is not found then the rules defined with :rule are checked:

1. All the matching rules without commands are used, but only if the source already exists. Thus this cannot be used to depend on a file that is still to be created.

2. One rule with commands will be selected, in this order of preference:

   A rule for which the sources exist.

   A rule for which one of the sources does not exist and was not defined with the {sourceexists} option.

   If there are multiple matches, the rule with the longest pattern is used. Thus if you have these two rules:

   :rule test/%.html : test/%.in :do something :rule %.html : %.in :do something-else

   The first one will be used for a file "test/foo.html", the second one for a file "./foo.html". If there are two with an equally long pattern, this is an error.

        :rule %.jpg : path/%.jpg
                :copy $source $target
   

Multiple targets

When a dependency with build commands has more than one target, this means that the build commands will produce all these targets. This makes it possible to specify build commands that produce several files at the same time. Here is an example that compiles a file and at the same time produces a documentation file:

   foo.o foo.html : foo.src
        :sys srcit $source -o $(target[0]) --html $(target[1])

People used to "make" must be careful, they might expect the build commands to be executed once for each target. Aap doesn't work that way, because the above example would be impossible. To run commands on each target this must be explicitly specified. Example:

   dir1 dir2 dir3 :
        @for item in target_list:
             :mkdir $item

The variable "target_list" is a Python list of the target items. Another such variable is "source_list", it is the list of source files (this excludes virtual items; "depend_list" also has the virtual items). An extreme example of executing build commands for each combination of sources and targets:

   $OutputFiles : $InputFiles
        @for trg in target_list:
            :print start of file  >! $trg
            @for src in source_list:
                :sys foofilter -D$trg $src >> $trg

When multiple targets are used and there are no build commands, this works as if each target depends on the list of sources. Thus this dependency:

t1 t2 : s1 s2 s3

Is equivalent to:

t1 : s1 s2 s3 t2 : s1 s2 s3

Thus when t1 is outdated to s1, s2 or s3, this has no consequence for t2.

Automatic dependency checking

When a source file includes other files, the targets that depend on the source file also depend on the included files. Thus when "foo.c" includes "foo.h" and "foo.h" is changed, the build commands to produce "foo.o" from "foo.c" must be executed, even though "foo.c" itself didn't change.

Aap detects these implied dependencies automatically for the types it knows about. Currently that is C and C++. Either by using gcc or a Python function the "#include" statements are found in the source code and turned into a dependency without build commands.

This works recursively. Thus when "foo.c" includes "foo.h" and "foo.h" includes "common.h", the dependency will look like this:

        foo.c : foo.h common.h

For other types of files than C and C++ you can add your own dependency checker. For example, this is how to define a checker for the "tt" filetype:

      :action depend tt
          :sys tt_checker $source > $target

The "tt_checker" command reads the file "$source" and writes a dependency line in the file "$target". This is a dependency like it is used in a recipe. In a Makefile this has the same syntax, thus tools that produce dependencies for "make" will work. Here is an example:

   foo.o : foo.tt  foo.hh  include/common.hh

This is interpreted as a dependency on "foo.hh" and "include/common.hh". Note that "foo.o" and "foo.tt" are ignored. Tools designed for "make" produce these but they are irrelevant for Aap.

Since the build commands for ":action depend" are ordinary build commands, you can use Python commands, system commands or a mix of both to do the dependency checking.

More about customizing dependency checking in Chapter 29.

Attributes Overruling Variables

Most variables like $CFLAGS and $BDIR are used for all source files. Sometimes it is useful to use a different value for a group of files. This is done with an attribute that starts with "var_". What follows is the name of the variable to be overruled. Thus attribute "var_XYZ" overrules variable "XYZ".

The overruling is done for:

The attributes of all the sources are used. In case the same attribute is used twice, the last one wins.

Another method is to use an "add_" attribute. This works like "var_", but instead of overruling the variable value it is appended. This is useful for variables that are a list of items, such as $DEFINE. Example:

:attr thefile.c {add_DEFINE = -DEXTRA=yes}

Another method is to define a scope name. This scope is then used to find variables before searching other scopes, but after using the local scope. For example, to specify that the "s_opt" scope is to be used when compiling "filter.c":

              OPTIMIZE = 0
              DEBUG = yes
              :program myprog : main.c filter.c version.c
              :attr {scope = s_opt} filter.c
              s_opt.OPTIMIZE = 4
              s_opt.DEBUG = no

Note that you can set the values of the variables in the user scope after adding the scope attribute to "filter.c".

Virtual Targets

A virtual target is a target that is not an actual file. A Virtual target is used to trigger build commands without creating a file with the name of the target. Common virtual targets are "clean", "all", "publish", etc.

When a target is virtual it is always built. Aap does not remember if it was already done a previous time. However, it is only build once for an invocation of Aap. Example:

clean: :del {r}{f} temp/*

To remember the signatures for a virtual target use the "remember" attribute:

version {virtual}{remember} : version.txt.in :print $Version | :cat - $source >! version.txt

Now "aap version" will only execute the :print command if version.txt.in has changed since the last time this was done.

Using {remember} for one of the known virtual targets (e.g., "all" or "fetch") is unusual, except for "publish".

When using {remember} for a virtual target without a dependency, it will only be built once. This can be used to remember the date of the first invocation.

all: firsttime firsttime {virtual}{remember}: :print First build on $DATESTR > firstbuild.txt

The difference with a direct dependency on "firstbuild.txt" is that when this file is deleted, it won't be built again.

Source Path

The sources for a dependency are searched for in the directories specified with $SRCPATH. The default is ". $BDIR", which means that the sources are searched for in the current directory and in the build directory. The current directory usually is the directory in which the recipe is located, but a :cd command may change this.

The "srcpath" attribute overrules using $SRCPATH for an item. Example:

        :attr bar.c {srcpath = ~/src/lib}

To avoid using $SRCPATH for a source, so that it is only found in the current directory, make the "srcpath" attribute empty:

        foo.o : src/foo.c {srcpath=}

When setting $SRCPATH to include the value of other variables, you may want to use "$=", so that the value of the variable is not expanded right away but when $SRCPATH is used. This is especially important when appending to $SRCPATH before a :variant command, since it changes $BDIR. Example:

        SRCPATH $+= include

Warning: Using the search path means that the first encountered file will be used. When old files are lying around the wrong file may be picked up. Use the full path to avoid this.

Depending On A Directory

When a target depends on the existence of a directory, it can be specified this way:

foodir/foo : foodir {directory} :print >$target this is foo

The directory will be created if it doesn't exist. The normal mode will be used (0777 with umask applied). When a different mode is required specify it with an octal value: {directory = 0700}. The number must start with a zero.

Build Command Signature

A special kind of signature is used to check if the build commands have changed. An example:

foo.o : {buildcheck = $CFLAGS} foo.c :sys $CC $CFLAGS -c $source -o $target

This defines a check for the value of $CFLAGS. When this value changes, the target is considered outdated. When something else in the build command changes, e.g., $CC, this does not cause the target to become outdated.

The default buildcheck is made from the build commands themselves. This is with variables expanded before the commands have been executed. Thus when one of the commands is ":sys $CC $CFLAGS $source" and $CC or $CFLAGS changes, the buildcheck signature changes. The :do commands are also expanded into the commands for the action specified. However, this only works when the action and filetype can be estimated. The action must be specified plain, not with a variable, and the filetype used is the first of:

1. a filetype attribute specified after action

2. if the first argument doesn't contain a "$", the filetype of this argument

3. the filetype of the first source argument of the dependency.

To add something to the default check for the build commands the $commands variable can be used. Example:

Version = 1.4 foo.txt : {buildcheck = $commands $Version} :del {force} $target :print >$target this is $target :print >>$target version number: $Version

If you now change the value of $Version, change one of the :print commands or add one, "foo.txt" will be rebuilt.

To simplify this, $xcommands can be used to check the build commands after expanding variables, thus you don't need to specify $Version:

foo.txt : {buildcheck = $xcommands}

However, this only works when all $VAR in the commands can be expanded and variables used in Python commands are not expanded.

To avoid checking the build commands, use an empty buildcheck. This is useful when you only want the target to exist and don't care about the command used to create it:

objects : {buildcheck = } :print "empty" > objects

Sometimes you might change the build commands in a recipe, which would normally mean the target should be updated, but you are sure that this isn't necessary and want to avoid executing the build commands. You can tell Aap to ignore the buildcheck once with the --contents option.

Chapter 14. Variants

You might first want to read the tutorial for a few examples of using variants.

Here is an example how build variants can be specified. This will be used to explain how it works.

        :variant Opt
            some
                OPTIMIZE = 2
            much
                OPTIMIZE = 6        
            *
                OPTIMIZE = 1

"Opt" is the name of a variable. It is used to select one of the variants. Each possible value is listed in the following line and further lines with the same indent. In the example these are "some" and "much". "*" is used to accept any value, it must be the last one. The first value mentioned is the default when the variable isn't set.

You can now start Aap with various arguments to specify the kind of optimizing you want to use:

        aap Opt=some      will set OPTIMIZE to 2
        aap Opt=much      will set OPTIMIZE to 6
        aap Opt=other    will set OPTIMIZE to 1
        aap                      will set OPTIMIZE to 2

Note that when "Opt" is not given a value the first entry is used, resulting in OPTIMIZE being set to 2. But when it is set to a value that isn't mentioned the last entry "*" is used.

The BDIR Variable

The $BDIR variable will be adjusted for the variant used. CAREFUL: this means that using $BDIR before :variant commands will use a different value, that might not always be what you want.

Inside the :variant command the value of $BDIR has already been adjusted.

When a target that is being build starts with $BDIR and $BDIR doesn't exist, it is created. (Actually, this happens when an item in the path is "build" or starts with "build-".

$BDIR is relative to the recipe. When using ":child dir/main.aap" the child recipe will use a different build directory dir/$BDIR. Note that when building the same source file twice from recipes that are in different directories, you will get two results. Best is to always build a target from the same recipe (that makes it easier to understand the recipe anyway).

Compile only when needed

This continues the last example of the tutorial.

We happen to know that the main.c file does not depend on the GUI used. With the recipe above it will nevertheless be compiled again for every GUI version. Although this is a small thing in this example, in a bigger project it becomes more important to skip compilation when it is not needed. Here is the modified recipe:

1    Source = main.c version.c gui.c
2
3    :variant Build
4        release
5            OPTIMIZE = 4
6            Target = myprog
7        debug
8            DEBUG = yes
9            Target = myprogd
10
11   :attr {var_DEFINE = $DEFINE} {var_BDIR = $BDIR} main.c
12    
13   Gui ?= motif
14   :variant Gui
15       console
16       motif
17            Source += motif.c
18       gtk
19            Source += gtk.c
20
21   DEFINE += -DGUI=$Gui
22
23   :program $Target : $Source

The only new line is line 11. The "main.c" file is given two extra attributes: var_DEFINE and var_BDIR. What happens is that when "main.c" is being build, Aap will check for attributes of this source file that start with "var_". The values will be used to set variables with the following name to the value of the attribute. Thus DEFINE gets the value of var_DEFINE. This means that the variable is overruled by the attribute while building "main.c".

The var_BDIR attribute is set to "$BDIR" before the second :variant command. It does not yet have the selected GUI appended there. The list of directories used is now:

Building multiple variants at once

If you want to build all the variants that are possible, use a few lines of Python code. Here is an example:

 1   :variant license
 2       trial
 3           DEFINE += -DTRIAL
 4       demo
 5           DEFINE += -DDEMO
 6       full
 7           DEFINE += -DFULL
 8   
 9   :variant language
10       chinese
11           DEFINE += -DCHINESE
12       bulgarian
13           DEFINE += -DBULGARIAN
14       *
15           DEFINE += -DENGLISH
16    
17   build:
18       :print Building with $license license for language $language.
19       :print DEFINE=$DEFINE
10    
21   all:
22       @for a in ['trial', 'demo', 'full']:                    #license
23       @   for c in ['chinese', 'bulgarian', 'english']:       #language
24               :execute main.aap build license=$a language=$c

Invoking Aap without arguments builds the "all" target, which loops over all possible variants and invokes the :execute command with the "build" target. The reason to use the "build" target is that without it the "all" target would be built again and result in an endless loop.

This is the resulting output:

     Building with trial license for language chinese.
     DEFINE=-DTRIAL -DCHINESE
     Building with trial license for language bulgarian.
     DEFINE=-DTRIAL -DBULGARIAN
     Building with trial license for language english.
     DEFINE=-DTRIAL -DENGLISH
     Building with demo license for language chinese.
     DEFINE=-DDEMO -DCHINESE
     Building with demo license for language bulgarian.
     DEFINE=-DDEMO -DBULGARIAN
     Building with demo license for language english.
     DEFINE=-DDEMO -DENGLISH
     Building with full license for language chinese.
     DEFINE=-DFULL -DCHINESE
     Building with full license for language bulgarian.
     DEFINE=-DFULL -DBULGARIAN
     Building with full license for language english.
     DEFINE=-DFULL -DENGLISH

Chapter 15. Publishing

Publishing means distributing the files of your project. This is a generic mechanism. You can use it to maintain a web site or to release a new version of your application.

The most straightforward way to publish a file is with the :publish command:

        :publish file ...

This uses the "publish" attribute on each of the files. When the "publish" attribute is missing the "commit" attribute is used. If both are missing this is an error.

When a file didn't change since the last time it was published, it won't be published again. This works with signatures, like building a target. The remote file is the target in this case. But Aap won't read the remote file to compute the signature, it will remember the signature from when the file was last uploaded (otherwise checking for outdated files would be slow).

To publish all files with a "publish" attribute start Aap like this:

      aap publish

If the "publish' target is defined explicitly it will be executed. Otherwise, all files with the "publish" attribute are given to the :publish command, just like using the :publishall command.

The "publish" attribute may consist of several items. Publishing will use all these items. This means a file can be distributed to several sites at once. This is unlike fetching, which stops as soon as the file could be obtained from one of the items.

When publishing fails for one of the sites, e.g., because the server is down, this is remembered. When you publish again, uploading is done only for that site. The destinations for which pusblishing worked will be skipped then.

Using Secure Copy

Uploading files requires write access to the server. There are several methods for this, but some have the disadvantage that your password is sent as normal text over the internet. Or someone in between can change the files that you send out. There is one that provides sufficient security: scp or secure copy.

To publish a file to a server through secure copy use a URL in this form:

        :publish file {publish = scp://myname@the.server.com/dir/file}

"myname" is your login name on the server "the.server.com". The "file" will be written there as "dir/file", relative to your login directory.

This requires the "scp" command, which is not a standard item. If it does not appear to exist then Aap will offer you to install it for you. If the automatic mechanism fails you will have to install it yourself. You might also want to do this if you have specific preferences for how the scp command is to be installed (versions of scp exist with different kinds of encryption).

To avoid having to type a password each time you need to use public keys. For MS-Windows you can find information here:

       http://the.earth.li/~sgtatham/putty/0.53b/htmldoc/Chapter8.html

You can specify the command to be executed with the $SCP variable, see the reference manual. Example:

          SCP = scp -i c:/private/keyfile

Using Another Method

Using "rsync" has an advantage if you have files with only a few changes. The rsync program will only transfer the differences. This adds a bit of overhead to find out what changed, thus it's a bit slower when copying whole files.

Aap uses the same secure channel as with "scp" by default. This can be changed with the $RSYNC variable, see the reference manual.

If your server does not support "scp", you might want to use "rcp". However, this is insecure, information is transferred unencoded over the internet and making a connection is not secure. Only use this when "scp" is not possible.

Using "rcp" means changing the "scp://" part of the URL to "rcp://". The $RCP variable holds the name of the command, see the reference manual.

You can also use "ftp". Some web servers require this, even though ftp sends your password as plain text over the internet, thus it is insecure. Aap uses the Python ftp library, thus an external command is not needed and there is no variable to specify the command.

Changing A Url

When you switch to another server you need to change the "publish" attribute. The next time you invoke Aap for publishing it will automatically upload all the files to the new server.

If you didn't actually change to another server, but the URL used for the server changed, invoke Aap once with the "--contents" argument:

      aap publish --contents

This will cause only files that changed to be published. It avoids that all the files with a changed "publish" attribute are published again. It will still upload files that you changed since the last upload and newly added files. But be careful: files that you once uploaded, deleted from the list of files and now added will NOT be uploaded!

Distributing Generated Files

When publishing a new version of an application, you might want to include a number of generated files. This reduces the number of tools someone needs to use when installing the application. For example, the "configure" script that "autoconf" produces from "configure.in" can be included.

To avoid these generated results to be generated again when the user runs aap, explicitly specify a signature file to be used and distribute this signature file together with the generated files. For example, suppose you have a directory with these files you created:

        main.aap
        prog.c

Additionally there is the file "version.c" that is generated by the "main.aap" recipe. It contains the date of last modification. To avoid it being generated again, include the "mysign" file in the distribution. The files to be distributed are:

        mysign
        main.aap
        prog.c
        version.c

In the "main.aap" recipe the "signfile" attribute is used for the dependency that generates version.c:

        version.c {signfile = mysign} : prog.c
                :print char *timestamp = "$TIMESTR"; >! $target

The result is that "version.c" will only be generated when "prog.c" has changed. When the "signfile" attribute would not have been used, "version.c" would have been generated after unpacking the distributed files.

Copying All Files At Once

The "finally" target is always executed after the other targets have been successfully built. Here is an example that uses the "finally" target to copy all files that need to be uploaded with one command.

        Source = *.html
        all: remote/$*Source {virtual} {remember}

        CFile =
        :rule remote/% : %
                _recipe.CFile += $source

        finally:
                @if _recipe.CFile:
                        :copy $_recipe.CFile ftp://my.org/www

Warning: When the :copy command fails, aap doesn't know the targets were not made properly and won't do it again next time. Using the "publish" attribute works better.

Chapter 16. Fetching

Fetching And Updating

A convention about using the "update" and "fetch" targets makes it easy for users to know how to use a recipe. The main recipe for a project should be able to be used in three ways:

1. Without specifying a target.

   This should build the program in the usual way. Files with a "fetch" attribute are obtained when they are missing.

2. With the "fetch" target.

   This should obtain the latest version of all the files for the program, without building the program.

3. With the "update" target.

   This should fetch all the files for the program and then build it. It's like the previous two ways combined.

        Status = status.txt
        Source = main.c version.c
        Header = common.h
        Target = myprog

        $Target : $Source $Status
            :cat $Status
            :do build $source

        # specify where to fetch the files from
        :attr {fetch = cvs://:pserver:anonymous@cvs.myproject.sf.net:/cvsroot/myproject} $Source $Header
        :attr {fetch = ftp://ftp.myproject.org/pub/%file%} $Status

        PATCH = patches/fix-1.034.diff {fetch = $FTPDIR} {constant}

        fetch:
                :fetch $Source $Header $Status

Or use the :fetchall command.

NOTE: When any child recipe defines a "fetch" target no automatic fetching is done for any of the recipes. This may not be what you expect.

When there is no "update" target it is automatically generated. It will invoke the "fetch" target and the default target(s) of the recipe. To do something similar manually:

        update: fetch $Target

The Fetch Attribute

The "fetch" attribute is used to specify a list of locations where the file can be fetched from. The word at the start defines the method used to fetch the file:

        ftp://ftp.server.name//full/path/file
        ftp://ftp.server.name/relative/path/file
        http://www.server.name/path/file
        scp://host.name/path:path/file
        rcp://host.name/path:path/file
        rsync://host.name/path:path/file
        cvs://:METHOD:[[USER][:PASSWORD]@]HOSTNAME[:[PORT]]/path/to/repository
        file:~user/dir/file
        file:///etc/fstab

Note that for an absolute path, relative to the root of the file system, you use either one or three slashes, but not two. Thus "file:/etc/fstab" and "file:///etc/fstab" are the file "/etc/fstab". A relative path has two or no slashes, but keep in mind that moving the recipe will make it invalid. You can also use "file:~/file" or "file://~/file" for a file in your own home directory, and "file:~jan/file" or "file://~jan/file" for a file in the home directory of user "jan".

In the "fetch" attribute the string "%file%" can be used where the path of the local target is to be inserted. This is useful when several files have a common directory. Similarly "%basename%" can be used when the last item in the path is to be used. This removes the path from the local file name, thus can be used when the remote directory is called differently and only the file name is the same. Examples:

        :attr {fetch = ftp://ftp.foo.org/pub/foo/%file%} src/include/bar.h

Gets the file "src/include/bar.h" from "ftp://ftp.foo.org/pub/foo/src/include/bar.h".

        :attr {fetch = ftp://ftp.foo.org/pub/foo/src-2.0/include/%basename%}
                          src/include/bar.h

Gets the file "src/include/bar.h" from "ftp://ftp.foo.org/pub/foo/src-2.0/include/bar.h".

Defining Your Own Method

To add a new fetch method, define a Python function with the name "fetch_method", where "method" is the word at the start. The function will be called with four arguments:

    :python
        def fetch_foo(dict, machine, path, fname):
            from foolib import foo_the_file, FooError
            try:
                foo_the_file(machine, path, fname)
            except FooError, e:
                raise IOError, 'fetch_foo() failed: %s' % str(e)
            return 1

Note that a version control function overrules a fetch function. Thus if "foo_command()" is defined "fetch_foo" will not be called.

Caching

Remote files are downloaded when used. This can take quite a bit of time. Therefore downloaded files are cached and only downloaded again when outdated.

The cache can be spread over several directories. The list is specified with the $CACHE variable.

NOTE: Using a global, writable directory makes it possible to share the cache with other users, but only do this when you trust everybody who can login to the system! Someone who wants to do harm or make a practical joke could put a bogus file in the cache.

A cached file becomes outdated as specified with the "cache_update" attribute or the $CACHEUPDATE variable. The value is a number and a name. Possible values for the name:

When a file becomes outdated, its timestamp is obtained. When it differs from when the file was last downloaded, the file is downloaded again. When the file changes but doesn't get a new timestamp this will not be noticed.

When fetching files the cached files are not used (but may be updated).

Chapter 17. Installing

This section contains details about the installation of the produced programs and other items. Those other items can be libraries (produced by the :lib, :dll, or :ltlib commands), header files for the API of a library, documentation (like manpages or info files), and as a catch-all, "data."

Usually installing is done with aap install. If you do not define an "install" target in the recipe, Aap will add one for you. The default install target invokes up to 15 other install targets, one for each kind of item you can install. This makes it easy to customize the installation of some particular kind of item (e.g. libtool archives).

The default install target invokes two or three other targets: install-platform, install-shared, and (optionally, only if you define it in the recipe) install-local. Each of these invokes other install targets for specific kinds of files, as follows:

Each of these dependencies is only added automatically if you do not define it yourself. In other words, if you do not define a dependency with install-data as a target, Aap will add such a dependency internally. Unless you need special processing for specific kinds of items, you should rarely need to define any of the install targets yourself. The exceptions are install-platform-local, install-shared-local and install-local, which you can define without disturbing Aap's normal mechanisms for installing the programs and libraries you create.

All these dependencies that Aap adds are at the toplevel (unlike "clean" and "cleanmore", which are done for each parent and child recipe).

All of Aap's default install targets operate in roughly the same fashion: specific actions are invoked for each install target. The default actions all use top-level variables named INSTALL_target which collect filenames to install. Other toplevel variables control where those files are installed (targetDIR) and what file mode is used (targetMODE). This table shows the specific settings for each of the default install targets:

The :program command adds its target to the $INSTALL_EXEC variable. The :lib command adds its target to the $INSTALL_LIB variable. The :dll command adds its target to the $INSTALL_DLL variable. The :ltlib command adds its target to the $INSTALL_LTLIB variable.

The "installexec" action will strip the program by default, if the "strip" program can be found. If you don't want this add the {nostrip} attribute to the program or set $STRIP to an empty value.

You can also overrule the default actions by one of your own. The install_files() function can be useful then. See the default.aap recipe for examples.

Destination Directories

All the install targets prepend a path to the directory they install into. The directory mentioned above is appended.

The variables are concatenated. For example, programs are installed in $DESTDIR$PREFIX$EXECDIR. Slashes are added in between where needed.

The directories that are used are automatically created when needed. Note that "uninstall" does not delete the directories!

When installing the path to a file is normally removed. Thus when you produced a program "results/myprog" it will be installed as "myprog". If you need to keep the path use the "keepdir" attribute on the file name.

          INSTALL_INCLUDE += sys/myheader.h {keepdir}

As an alternative to {keepdir}, there is the {installdir} attribute, which explicitly sets the relative path of the file to be installed. Files with an {installdir} attribute are installed in $DESTDIR$PREFIX$targetDIR$installdir. The above setting could also be done as:

INSTALL_INCLUDE += sys/myheader.h {installdir=sys}

The advantage of {installdir} over {keepdir} is that the relative paths (from toplevel recipe to file and from install directory to the desired install location) need not be the same. For instance:

INSTALL_INCLUDE += api/2.2/c/myheader.h {installdir=sys}

The above mentioned mode variable is used to set the mode bits of the file after installing. If this is not wanted, use the {keepmode} attribute. Example:

          INSTALL_DATA += myscript.sh {keepmode}

To use another mode for a specific file add the {mode = 0555} attribute:

          INSTALL_DATA += myscript.sh {mode = 0750}

Installing to a remote machine should work, although setting the file mode may not always work properly, depending on the transfer method used.

Keep in mind that installation is done from the top directory. In a child recipe that is located in another directory you need to specify the path to the file to install relative to the top directory. Using the $TOPDIR variable and rc-style expansion should work. Example:

    # Filenames relative to the child directory
    child_INSTALL_DATA = myscript.sh myicon.png

    # Now add those filenames, relative to the top
    INSTALL_DATA += $TOPDIR/$*child_INSTALL_DATA

If you hard code the paths from the parent to the files to install, say by writing INSTALL_DATA += child/myscript.sh, then you cannot execute the child recipe by itself (as if it were a toplevel recipe), since the paths will be wrong. Using $TOPDIR, or equivalently the topdir function, is the safe way to do so.

Uninstall

"aap uninstall" deletes the file that "aap install" has installed. All the targets and actions have the same name with "install" replaced with "uninstall". The same variables are used.

Files that do not exist are silently skipped. Files that do exist but cannot be deleted will cause a warning message.

Sometimes your recipe offers installing optional files. You probably want to uninstall those optional files as well, without requiring the user the specify the same options again. For this you can set the $UNINSTALL_* variables. For example, if you install either the "foo" or "bar" program:

        :variant What
             foo
                  Target = foo
                  UNINSTALL_EXEC = bar$EXESUF
             bar
                  Target = bar
                  UNINSTALL_EXEC = foo$EXESUF

        :program $Target : $Sources

Chapter 18. Version Control

This is about using Aap with a Version Control System (VCS)

The generic form of version control commands is:

:command file ...

Or:

:command {attr = val} ... file ...

The commands use the "commit" attribute of a file to obtain the kind of version control system and its location. For example:

:attr foo.c {commit = cvs://:ext:$CVSUSER_AAP@cvs.a-a-p.sf.net:/cvsroot/a-a-p}

For CVS it is also possible to only specify the method. CVS will then use the same specification for the repository as used when checking the files out.

:attr foo.c {commit = cvs://}

These commands can be used:

Additionally, there is the generic command:

        :verscont action {attr = val} ... file ...

This calls the Version control module as specified in the "commit" attribute for "action" with the supplied arguments. What happens is specific for the VCS.

Operating On All Files

These commands work on all the files mentioned in the recipe and child recipes that have the "commit" attribute:

Related to these commands are targets that are handled automatically when not defined explicitly. When defining a target for these, it would be highly unexpected if it works differently.

For the above the "-l" or "--local" command line option can be used to restrict the operation to the directory of the recipe and not recurse into subdirectories.

A variable can be used to set the default change log entries:

LOGENTRY=message

This variable is used for new, changed and deleted files that don't have a {logentry} attribute.

When it's desired to commit one directory at a time the following construct can be used:

        source_files = *.c
        include_files = include/*.h
        commit-src {virtual}:
            :commit $source_files
            :removeall .
        commit-include {virtual}:
            :commit $include_files
            :removeall include

Note that this is not possible when the sources and include files are in one directory, :removeall only works per directory.

Using Subversion

Subversion is a new version control system that is going to replace CVS. It has many advantages, such as atomic commits. But version 1.0 is not ready yet (although the current versions appear to be very stable and usable).

Subversion support is not implemented yet. For the time being you can retrieve files from a Subversion repository by using a URL. That works, because subversion is using an Apache server. You can obtain a copy of single files by specifying the URL in the fetch attribute. Obviously you can't commit changed files this way.

Using Another Version Control System

To add support for a new version control system, define a Python function with the name "method_command", where "method" is the word at the start of the commit attribute. The function will be called with five arguments:

For an example look at cvs_command() in the VersContCvs.py file of the Aap sources.

A second function that is to be defined is "method_list". It should return a list of the files that are currently in a specified directory in the repository. Return an empty list if there are no files. The function will be called with these arguments:

For an example look at cvs_list() in the VersContCvs.py file of the Aap sources.

Chapter 19. Using CVS

A common way to distribute sources and working together on them is using CVS. This requires a certain way of working. The basics are explained here. For more information on CVS see http://www.cvshome.org.

Obtaining A Module

The idea is to hide the details from a user that wants to obtain the module. This requires making a toplevel recipe that contains the instructions. Here is an example:

        CVSROOT = :pserver:anonymous@cvs.myproject.sf.net:/cvsroot/myproject
        :child mymodule/main.aap {fetch = cvs://$CVSROOT}
        all fetch:
            :fetch {fetch = cvs://$CVSROOT} mymodule

Executing this recipe will use the "fetch" target. The :fetch command takes care of checking out the whole module "mymodule" from the CVS repository with the specified name.

Note that this toplevel recipe cannot be obtained from CVS itself, that has a chicken-egg problem.

Fetching

The child recipe "mymodule/main.aap" may be totally unaware of coming from a CVS repository. If this is the case, you can build and install with the recipe, but not fetch the files or send updates back into CVS. You need to use the toplevel recipe above to obtain the latest updates of the files. This will then update all the files in the module. However, the toplevel recipe itself will never be fetched.

To be able to fetch only some of the files of the module, the recipe must be made aware of which files are coming from CVS. This is done by using an "fetch" attribute with a URL-like specification for the CVS server: {fetch = cvs://servername/dir}. Since CVS remembers the name of the server, leaving out the server name and just using "cvs://" is sufficient. Example:

        source = foo.c version.c
        header = common.h
        :attr {fetch = cvs://} $source $header
        :program myprogram : $source

If you now do "aap fetch" with this recipe, the files foo.c, version.c and common.h will be updated from the CVS repository. The target myprogram isn't updated, of course.

Note: When none of the used recipes specifies a "fetch" target, one will be generated automatically. This will go through all the nodes used in the recipe and fetch the ones that have an "fetch" attribute.

The recipe itself may also be fetched from the CVS repository:

        :recipe {fetch = cvs://}

To update a whole directory, omit the "fetch" attribute from individual files and use it on the directory. Example:

        source = main.c version.c
        :attr {fetch = cvs://} .
        :program myprog : $source

Alternatively, a specific "fetch" target may be specified. The automatic updates are not used then. You can specify the "fetch" attribute right there.

        fetch:
            :fetch {fetch = cvs://} $source

If you decided to checkout only part of a module, and want to be able to get the rest later, you need to tell where in the module of the file can be found. This is done by adding a "path" attribute to the cvs:// item in the fetch attribute. Example:

        fetch:
            :fetch {fetch = $CVSROOT {path = mymodule/foo}} foo.aap

What will happen is that aap will checkout "mymodule/foo/foo.aap", while standing in two directories upwards. That's required for CVS to checkout the file correctly. Note: this only works as expected if the recipe is located in the directory "mymodule/foo"!

If the "path" attribute is omitted, A-A-P will obtain the information from the "CVS/Repository" file. This only works when something in the same directory was already checked out from CVS.

Checking In

When you have made changes to your local project files and want to upload them all into the CVS repository, you can use this command:

        :reviseall

You must make sure that _ALL_ files in the current directory and below that you want to appear in CVS have the "commit" attribute, and no others! Files that were previously not in CVS will be added ("cvs add file") and that exist in CVS but don't have a "commit" attribute are removed ("cvs remove file"). Then all files are committed ("cvs commit file").

To be able to commit changes you made into the CVS repository, you need to specify the server name and your user name on that server. Since the user name is different for everybody, you must specify it in a recipe in your ~/.aap/startup/ directory. For example:

        CVSUSER_AAP = foobar

The name of the variable starts with "CVSUSER" and is followed by the name of the project. That is because you might have a different user name for each project.

The method to access the server also needs to be specified. For example, on SourceForge the "ext" method is used, which sends passwords over an SSH connection for security. The name used for the server then becomes:

        :ext:$CVSUSER_AAP@cvs.a-a-p.sf.net:/cvsroot/a-a-p

You can see why this is specified in the recipe, you wouldn't want to type this for commiting each change!

Distributing Your Project With CVS

This is a short how-to that explains how to start distributing a set of files (and directories) using CVS.

1. Copy the files you want to distribute to a separate directory

   Mostly you have various files in your project for your own use that you don't want to distribute. These can be backup files and snippets of code that you want to keep for later. Since the cvs command below imports all files it can find, you need to have a directory tree with exactly those files you want to store in CVS. Best is to to make a copy of the project. On Unix:

           cp -r projectdir tempdir

   Then delete all files you don't want to distribute. Be especially careful to delete "AAPDIR" directories and hidden files (starting with a dot). It's better to delete too much than too few: you can always add files later.

2. Import the project to the CVS repository

   Move to the newly created directory ("tempdir" in the example above). Import the whole tree into CVS with a single command. Example:

           cd tempdir
           cvs -d:ext:myname@cvs.myproject.sf.net:/cvsroot/myproject import mymodule myproject start

   Careful: This will create at least one new directory "mymodule", which you can't delete with CVS commands. This will create the module "mymodule" and put all the files and directories in it. If there are any problems, read the documentation available for your CVS server.

3. Checkout a copy from CVS and merge

   Move to a directory where you want to get your project back. Create the directory "myproject" with this example:

           cvs -d:ext:myname@cvs.myproject.sf.net:/cvsroot/myproject checkout mymodule

   You get back the files you imported in step 2, plus a bunch of "CVS" directories. These contain the administration for the cvs program. Move each of these directories back to your original project. Example:

           mv myproject/CVS projectdir/CVS
           mv myproject/include/CVS projectdir/include/CVS

   If you have many directories, one complicated command does them all:

           cd myproject
           find . -name CVS -exec mv {} ../projectdir/{} \;

   This is a bit tricky. Another method is to copy all the files from your original project into the newly created directory. But then you have to be careful not to change relevant file attributes, which is tricky as well. Obviously, the best solution is to have all files you need in CVS, so that you don't have to copy any files.

4. Commit changes

   After making changes to your project and testing them, it's time to check them in. In the recipe you use for building, add a "commit" attribute to all the files that should be in CVS. The :reviseall command then does the work for you (see above). Example:

   Files = $source $header main.aap :attr {commit = cvs://:ext:$CVSUSER_MYPROJECT@cvs.myproject.sf.net:/cvsroot/myproject} $Files :reviseall

   Careful: $Files must contain all files that you want to publish in this directory and below. If $Files has extra files they will be added in CVS. Files missing from $Files will be removed from CVS.

   You must assign $CVSUSER_MYPROJECT your user name on the CVS server. Usually you do this in one of your personal A-A-P startup files, for example "~/.aap/startup/main.aap".

Using Sourceforge

If you are making open source software and need to find a place to distribute it, you might consider using SourceForge. It's free and relatively stable. They provide http access for your web pages, a CVS repository and a server for downloading files. There are news groups and maillists to support communication. Read more about it at http://sf.net.

Since you never know what happens with a free service, it's a good idea to keep all your precious work on a local system and update the files on SourceForge from there. If several people are updating the SourceForge site, either make sure everyone keeps copies, or make backup copies (at least weekly).

You can use A-A-P recipes to upload your files to the SourceForge servers. To avoid having to type passwords each time, use an ssh client and put your public keys in your home directory (for the web pages) or on your account page (for the CVS server). Read the SourceForge documentation for how to do this.

For uploading web pages you can use a recipe like this:

        Files = index.html
                download.html
                news.html
                images/logo.gif
        :attr {publish = rsync://myname@myproject.sf.net//home/groups/m/my/myproject/htdocs/%file%}
        $Files

Start this recipe with the "publish" target. If you don't have the "rsync" command you might want to use "scp" instead. The effect is the same, but "rsync" works more efficient.

For sourceforge, set environment variable CVS_RSH to "ssh". Otherwise you won't be able to login. Do "touch ~/.cvspass" to be able to use "cvs login" Upload your ssh keys to your account to avoid having to type your password each time.

Chapter 20. Issue Tracking

Bug Reporting

A recipe used to install an application should offer the "report" target. This is the standard way for a user to report a problem. The recipe should then help the user with reporting a problem as much as possible.

An example is to send the developer an e-mail. Commands in the recipe are used to put useful information in the message, so that the user only has to fill in his specific problem. Example:

        report:
            tmpfile = `tempfname()`
            :syseval foobar --version | :assign Version
            :print >$tmpfile  Using foobar version: $Version
            :print >>$tmpfile system type: `os.name`
            @if os.name == "posix":
                :print >>$tmpfile system details: `os.uname()`
            :print >>$tmpfile
            :print >>$tmpfile State your problem here
            :do email {subject = 'problem in FOOBAR'}
                            {to = bugs@foobar.org}
                            {edit}
                            {remove}
                            $tmpfile

The "foobar --version" command is used to obtain the actual version of the "foobar" program being used. Replace "foobar" with the actual name of your program.

When a web form is to be filled in, give the user hints about what information to fill in certain fields and start a browser on the right location. Example:

        report:
            :do view {async} http://www.foo.org/bugreport/
            tmpfile = `tempfname()`
            :print >$tmpfile   use this information in the bug report:
            :print >>$tmpfile  program version: $VERSION
            :print >>$tmpfile  system type: `os.name`
            :do view {remove} $tmpfile

Obviously this is a bit primitive, the user has to copy text from the text viewer to the browser. Try using a better method, filling fields of the form directly if you can.

Bug Fixing

Once a bug has been fixed, the developer needs to update the related bug report. The "tracker" target is the standard way for a developer to get to the place where the status of the bug report can be changed.

Since trackers work in many different ways the recipe has to specify the commands. Example:

        tracker:
            :do view {async} http://www.foo.org/tracker?assigned_to=$USER

This is very primitive. The developer still has to locate the bug report and change the status and add remarks. The above example at least lists the bug reports for the current user.

Chapter 21. Using Python

Python commands can be used where Aap commands are not sufficient. This includes flow control, selecting the commands to be executed and repeating commands.

Using Python Lines

Single lines of Python code can be included in the recipe by prepending "@". This is most often used for flow control:

        @if os.path.isdir("/usr/local/bin"):
            :copy $File /usr/local/bin

You can write multiple Python commands, just prefix a "@" to every line. Do remember that the amount of indent is used to form command blocks. The indent that is used excludes the "@" character. When there is a non-white character after the "@", the "@" is removed. When there is white space after the "@" it is replaced with a space. Generally you don't need to worry about this, if the indenting looks right it probably is.

The main advantage of using single Python lines is that they can be mixed freely with Aap recipe command lines. You can use Python lines both at the recipe level and in build commands. Again, just make sure the indent indicates command blocks.

To learn using Python start at the Python web site: http://www.python.org/doc/

Using Python Expressions

In an assignment, command arguments and most other places a Python expression can be used in backticks. Expanding this is done before expanding $VAR items, because this allows the possibility to use the Python expression to result in the name of a variable. Example:

        foovaridx = 5
        Foo = $Src`foovaridx`

Is equivalent to:

        Foo = $Src5

The result of the Python expression in backticks should be a string or a list of strings. A list is automatically converted to a white-separated string of all list items.

A Python expression cannot be used for the variable name in an assignment. This doesn't work:

        `varname` = value

If you really need this, use a Python command instead:

        @eval(varname + ' = "value"')

When using a function from a module, it must be imported first. Example:

        @from httplib import HTTP
        Connection = `HTTP('www.microsoft.com')`

For your convenience these things are imported for you already:

        from glob import glob
        from RecPython import *

The RecPython module defines the Python functions listed in Chapter 38>.

A backtick in the Python expression has to be escaped to avoid it being recognized as the end of the expression:

        CMD = `my_func("$(`)grep -l foo *.c$(`)")`

contains the Python expression:

        my_func("`grep -l foo *.c`")

In the result of the Python expression $ characters are doubled, to avoid them being interpreted as the start of a variable reference. Otherwise Python expressions with arbitrary results would always have to be filtered explicitly. When the resulting string is used the $$ will be reduced to a single $ again.

Aap variables can be

The result of the expression is used as a string in place of the expression and the backticks. Example:

        Foo = foo/`glob("*.tmp")`

Would be equivalent to:

        Foo = foo/one.tmp two.tmp

Note that "foo/" is only prepended to the whole result, not each white-separated item. If you do want rc-style expansion, use two commands:

        TT = `glob("*.tmp")`
        Foo = foo/$*TT

Equivalent to:

        Foo = foo/one.tmp foo/two.tmp

Note that a following attribute is only attached to the last item resulting from the Python expression.

        Source = `glob('*.c')` {check = md5}

Can be equivalent to:

        Source = foo.c bar.c {check = md5}

To apply it to all items, use the :attr command:

        Source = `glob('*.c')`
        :attr {check = md5} $Source

Watch out for unexpected results when rc-style expansion is done for $*VAR. Example:

        VAR = one two
        Foo = $*VAR/`glob("*.tmp")`

Would result in:

        Foo = one/one.tmp two/one.tmp two.tmp

because the `` part is expanded first, thus the assignment is executed like:

        Foo = $*VAR/one.tmp two.tmp

The backticks for a Python expression are also recognized inside quotes. Thus you need to escape the special meaning there:

        Foo = "this$(`)file" that$(`)file

Backtick expressions can be used inside a string if you really need this:

        DIR = /home/foo /home/bar
        :print "`DIR + "/fi le"`"

results in:

        "/home/foo /home/bar/fi le"

Compare this to:

        :print "$*DIR/fi le"

which results in:

        "/home/foo/fi le" "/home/bar/fi le"

Python Block

A block of Python commands is started with a :python command. If no terminator string is specified the python code ends where the indent is equal to or less than the :python command:

    Source = foo.c bar.c
    :python
        for i in items:
            Source = Source + " " + i
    Target = foo

Optionally a terminator string may be specified. The indent of the Python code may then drop below the indent of the :python command.

The terminator cannot contain white space. A comment may follow. The Python block continues until the terminator string is found in a line by itself. It may be preceded and followed by white space and a comment may follow. Example:

@if ok: :print finding include files :python EOF # start of the Python block include = glob("include/*.c") EOF # end of the Python block

Useful Python Items

A list of Python functions defined by Aap can be found in the reference manual, Chapter 38.

        VAR = `os.environ.get('VAR', 'default')`

Obtain environment variable $VAR. If it is not set use "default".

        @os.environ["PATH"] = mypath

Set an environment variable.

        files = `glob("*.ext")`

Obtain a list of files matching "*.ext". Aap will take care of turning the list that glob() returns into a string, using quotes where needed.

The difference with using "*.ext" directly is that the expansion is done right here, not later when $files is used. The catch with using glob() here is that when a file name contains a wildcard character it may be expanded again. So long as that expansion fails or matches the same file name it is still OK, but it becomes rather unpredictable. Use wildescape() when needed.

        choice = `raw_input("Enter the value: ")`

Prompt the user to enter a value.

        tempfile = `tempfname()`

Get a file name to use for temporary files. The file will not exist. See tempfname().

If you create it you need to make sure it is deleted afterwards. Example:

        tempfile = `tempfname()`
        @try:
          :print >$tempfile  start of file
          :print >>$tempfile $this variable may not exist and cause an error
          :cat $tempfile
        @finally:
          # this is executed whether there is an error or not
          :del $tempfile

Chapter 22. Porting an Application

Porting an application means starting with the original sources and changing them a little bit to make the application compile and install on a specific system. An Aap port recipe offers a simple way to create a port, because all the standard work does not need to be specified.

NOTE: not all features mentioned here fully work. Make sure you test your port recipe before publishing it.

The Port Recipe

The basic idea is that you assign values to a number of predefined variables. Aap will then generate the steps for building and installing the package, using the values you have specified. The presence of the "PORTNAME" variable triggers Aap to handle the recipe as a port recipe. The list of variables is in the next section.

This is thee list of steps performed when executing Aap without arguments, in this order:

You can do part of the work by starting Aap with one of the step names. The steps before it will also be executed if necessary. For example, "aap package" will do all the steps necessary to generate the binary package but not install it.

If your port requires non-standard stuff, you can specify your own build commands. You can replace the normal step, prepend something to it and append something to it:

Example:

       post-test:
          # delete the directory used for testing
          :del {r}{f} $WRKDIR/testdir

Variables

Various variables need to be set to specify properties of the port.

Variables that can be used by the port recipe:

Variables that may be set by the port recipe, defaults are set only after reading the recipe:

Dependency Format

Dependencies on other modules are specified with the various DEPEND_ variables. The format of these variables is a list of items.

NOTE: Although you can currently specify dependencies, the code for checking them has not been implemented yet! Thus the user will have to figure it out by himself...

Items are normally white space separated, which means there is an "and" relation between them. Alternatively "|" can be used to indicate an "or" relation.

        DEPENDS = perl python           # require both perl and python
        DEPENDS = perl | python         # require perl or python

Parenthesis can be used to group items. Parenthesis must be used when combining "or" and "and" relations. Example:

        DEPENDS = (foo bar) | foobar    # Either both "foo" and "bar" or "foobar"
        DEPENDS = foo bar | foobar      # Illegal
        DEPENDS = foo (bar | foobar)    # "foo" and either "bar" or "foobar"

When a dependency is not met the first alternative will be installed, thus the order of "or" alternatives is significant.

Each item is in one of these formats:

In the above "_revision" can be left out to ignore the revision number. It actually works as if there is a "*" wildcard at the end of each item.

"name" can contain wildcards. When a part is following it is appended at the position of the wildcard (or at -9 if it comes first).

The version specifications can be concatenated, this creates an "and" relation. Example:

The "!" can be followed by parenthesis containing a list of specifications. This excludes the versions matching the specifications in the parenthesis. Example:

When a dependency is not met the newest version that meets the description is used.

For systems that do not allow specifying dependencies like this in a binary pacakge, the specific package version that exists when generating the package is used.

Dependencies For Various Steps

The various variables used to specify dependencies:

Only the dependencies specified with DEPEND_RUN will end up in the generated binary package. When using a shared library, it is recommended to put a dependency on the developer version (includes header files) in DEPEND_BUILD and a dependency on the library itself in DEPEND_RUN. The result is that when installing binary packages the header files for the library don't need to be installed.

The "CONFLICTS" variable should be set to specify modules with which this one conflits. That means only one of the two packages can be installed in the same location. It should still be possible to install the packages in different locations. The format of CONFLICTS is identical to that of the DEPENDS_ variables.

NOTE: Although you can currently specify dependencies and conflicts, the code for checking them has not been implemented yet! Thus the user will have to figure it out by himself...

Dependencies are automatically installed, unless "AUTODEPEND" is "no". The dependencies are normally satisfied by installing a port. When a satisfying port can not be found a binary package is installed. The ports and packages are first searched for on the local system. When not found the internet is searched.

The order of searching can be changed with "AUTODEPEND":

Steps

These are the individual steps for installing a ported application. Each step up to "install" depends on the previous one. Thus "aap install" will do all the preceding steps. But the steps that have already been successfully done in a previous invocation of Aap will be skipped. The "rundepend", "installtest", "clean", etc. targets do not depend on previous steps, they can be used separately.

Port Description

The text to describe the port is usually a page full of plain text. Here is an example:

        PORTDESCR << EOF
        This is the description of the port.
        A very important application indeed.

        See our website http://myport.org.
        EOF

In the rare situation that "EOF" actually appears in the text you can use anything else, such as "THEEND".

Chapter 23. Automatic Configuration

The :conf command is used to discover properties of the compiler and the system. With this information a program can be compiled to work on many different systems without the user to manually specify the properties. For Unix systems an alternative is using Autoconf, see Chapter 24.

TODO The ":conf" command is still under development.

  :conf init        Clean the configuration environment.  Only needed when
                    configure checks were done before.

  :conf language lang
                    Use "lang" as the language for tests that do not have a
                    {language} attribute.
                    Only "C" and "C++" are currently supported. "C" is the
                    default.
                    A test will be done if the compiler for the language
                    works.  If not all tests for the language will fail.
                    Example:
                           :conf language C++

  :conf header [mainoptions] headername [itemoptions] ...
                    Check for C or C++ header file(s) "headername".
                    Uses current value of INCLUDE.
                    Defines HAVE_headername if "headername" is available.
                    mainptions:
                        oneof        first of the items that works is used, at
                                     least one is required
                        required     all items are required to exist
                    itemoptions:
                        header       text included in the test program
                        language     C or C++
                    Examples:
                        :conf header X11/Shell.h
                        :conf header {oneof} sys/time.h times.h

  :conf function [mainoptions] functionname [itemoptions] ...
                    Check for C or C++ function(s) "functionname".
                    Uses the current value of $LIBS.
                    Defines HAVE_functionname if "functionname" is available.
                    Options: see ":conf header"
                    Example:
                        :conf function bcopy

  :conf type [mainoptions] typename [itemoptions] ...
                    Check for C or C++ type(s) "typename".
                    Uses the current value of $LIBS.
                    Defines HAVE_typename if type "typename" is available.
                    Options: see ":conf header".
                    Additional itemoption:
                        fallback     type definition to use for the type when
                                     it is not available
                    Example:
                           :conf type size_t {fallback = unsigned long}

  :conf lib [mainoptions] libname,funcname [itemoptions] ...
                    Check for library/libraries by testing if "funcname"
                    exists when using the "libname" library.
                    Uses the current value of $LIBS and appends "-llibname"
                    if the test succeeds.  Also appends to $_conf.LIBS.
                    Defines HAVE_libname if library "libname" is available.
                    Options: see ":conf header".
                    Example:
                           :conf lib iconv,iconv_open

  :conf write header filename
                    Write the HAVE_ and other symbols collected so far into
                    "filename".
                    Example:
                           :conf write header $BDIR/config.h

  :conf write recipe filename
                    Write the settings in the _conf scope into "filename".
                    Example:
                           :conf write recipe $BDIR/config.aap


The _conf scope stores variables set by configure tests.  For example,
$_conf.LIBS the libraries that ":conf lib" found.  The _conf scope is used in
the tree of scopes just before the toplevel scope, after all callstack and
recipe tree scopes.

Special characters in header, function and type names are changed to an
underscore before defining the HAVE_ symbol.  Lowercase characters are changed
to uppercase.

The preprocessor symbols that are gathered from the tests are available in
the $_conf.have dictionary.  Example:
  	:conf header sys/time.h
	@if _conf.have["HAVE_SYS_TIME_H"]:
	   :print we have time
	@else:
	   :print sorry, no time.

":conf write recipe" writes the variables in the _conf scope.  You can remove
variables you do not want to be written.  Example:
  	saveLIBS = $?_conf.LIBS
	_conf.LIBS =
	:conf lib foo,foobar
	_conf.FOOLIB = $_conf.LIBS
	_conf.LIBS = $saveLIBS

Chapter 24. Using Autoconf

The autoconf system is often used to configure C programs to be able to compile them on any Unix system. This section explains how to use autoconf with Aap in a nice way. An alternative is to use the :conf command of Aap. It is much easier to use and also works on non-Unix systems. See Chapter 23.

Running The Configure Script

A recipe that uses the configure script that autoconf generates can start like this:

        $BDIR/config.h $BDIR/config.aap :
                                configure config.arg config.h.in config.aap.in
            :sys ./configure `file2string("config.arg")`
            :move {force} config.h config.aap config.log
                                              config.cache config.status $BDIR
        config.arg:
            :touch {exist} config.arg

        :update $BDIR/config.aap
        :include $BDIR/config.aap

What happens here is that the "config.aap" target is updated before any of the building is done. This is required, because running the configure script will generate or update the "config.aap" file that influences how the building is done.

Remembering Configure Arguments

The arguments for configure are stored in the "config.arg" file. This makes it easy to run configure again with the same arguments. The file is read with the file2string() function. There should be a "config.txt" file that explains all the possible configure arguments, with examples that can be copied into "config.arg". Example:

        # Select the library to be used for terminal access.  When omitted a
        # series of libraries will be tried.  Useful values:
            --with-tlib=curses
            --with-tlib=termcap
            --with-tlib=termlib

The user can now copy one of the example lines to his "config.arg" file. Example:

        # select specific terminal library
            --with-tlib=termcap

Comment lines can be used, they must start with a "#". Note: a comment after an argument doesn't work, it will be seen as an argument.

When updating to a new version of the program, the same "config.arg" file can still be used. A "diff" between the old and the new "config.txt" will show what configure arguments have changed.

Variants And Configure

"config.aap" and "config.h" are put in $BDIR, because they depend on the current system. They might also depend on the variant to be built. In that case the :variant statement must be before the use of $BDIR. However, if the variant is selected by running configure, the variant must come later. "config.aap" and "config.h" are then updated when selecting another variant.

For the program to find "config.h" in $BDIR you must add an option to the C compiler. And you have to notify the compiler that the file exists, so that it will be included:

        INCLUDE += -I$BDIR
        DEFINE += -DHAVE_CONFIG_H

The "config.cache", "config.log" and "config.status" files are also moved to $BDIR. This means they are not available when running "./configure" again., This may be a bit slower, since the cache isn't used, but it is much more reliable. And you can view the log of each variant that was build.

Running Autoconf

For a developer there also needs to be a method to generate the configure script from configure.in. This needs to be done even before configure is run. Prepending this to the example above should work:

        configure {signfile = mysign} : configure.in
            :sys autoconf

Normally the "configure" script is distributed with the program, so that a user does not need to install and run autoconf. The "{signfile = mysign}" attribute on the target is used to avoid running autoconf when the user builds the program and the "configure" and "configure.in" files are still as they were distributed. The signatures in the "mysign" file, which you must include in the distribution, will match and Aap knows that "configure" is up-to-date. If using the "mysign" file was omitted, there would be no signature for the "configure" target and Aap would decide to run autoconf. When you change "configure.in" its signature will be different from what is stored in "mysign" and autoconf will be run.

Using A Distributed Configure Script

If you are porting an application that already has a configure script you can filter it to make it work with Aap. This means you can use the unmodified configure.in.

        configure_aap : configure
	:cat configure
		| :eval re.sub("Makefile", "config.aap", stdin)
		>! configure_aap
	:chmod 755 configure_aap

Now you need to execute "configure_aap" instead of "configure" in the first example above.

Skipping Configuration

Running configure can take quite a bit of time. And when you are not going to build anything that can be annoying. For example, "aap comment" doesn't require configure to run.

Also, configure doesn't work on a non-Unix system. When you have taken care in your code to handle this you can simply skip configure. This line above all the configure code should take care of this:

        @if osname() == "posix" and has_build_target():

The has_build_target() function checks for a target that will do some kind of building, which means configure must be run.

A Complete Example

Using all the parts mentioned above together we have a fairly complete method to handle running autoconf and configure. This code is used in the recipe that builds the Exuberant Ctags program.

    #
    # On Unix we run configure to generate config.h and config.aap.
    # This is skipped if there is no building to be done (e.g., for "clean").
    #
    @if osname() == "posix" and has_build_target():

        # "config.h" and "config.aap" are generated in $BDIR, because the are
        # different for each system.
        # Tell the compiler to find config.h in $BDIR.
        INCLUDE += -I$BDIR
        DEFINE += -DHAVE_CONFIG_H

        # Run autoconf when needed, but avoid doing this always, not everybody has
        # autoconf installed.  Include "mysign" in the distribution, it stores the
        # signature of the distributed configure script.
        configure {signfile = mysign} : configure.in
            @if not program_path("autoconf"):
                :print Can't find autoconf, using existing configure script.
            @else:
                :sys autoconf

        # Filter the configure script created by autoconf to generate config.aap
        # instead of Makefile.  This means we can use the unmodified configure.in
        # distributed with ctags.
        configure_aap : configure
            :cat configure
                    | :eval re.sub("Makefile", "config.aap", stdin)
                    >! configure_aap
            :chmod 755 configure_aap

        # Dependency to create config.aap by running the configure script.
        # The "config.arg" file is used for configure arguments.
        :attr config {virtual} {comment = Do configuration only}

        config $BDIR/config.h $BDIR/config.aap :
                    configure_aap config.arg config.h.in config.aap.in
            :sys ./configure_aap `file2string("config.arg")`
            # Move the results into $BDIR.  This also means the cache isn't used
            # the next time, it is unreliable.
            :move {force} config.h config.aap config.log config.cache
                                                               config.status $BDIR

        # Create an empty config.arg when it's missing.
        config.arg:
            :touch {exist} config.arg

        # Update config.aap before including it.  Forcefully when the "reconfig"
        # target is used.
        @if "reconfig" in var2list(_no.TARGETARG):
            :del {force} config.cache config.status
            :update {force} $BDIR/config.aap
        @else:
            :update $BDIR/config.aap

        :include $BDIR/config.aap

Chapter 25. Automatic Package Install

Aap provides a very powerful feature: When a program is required and it does not exist on the system, it can be installed automatically.

Suppose a recipe specifies that a file is to be uploaded to a server with secure copy. This requires the "scp" command. When Aap cannot find this command it offers you the choice to install it. Aap will check what kind of system you have and find a recipe for it. This recipe is executed and it will install the package for you. Either by obtaining an executable program or by fetching the sources and building them. Then the original recipe continues and uses the "scp" command that has just been installed.

Note: This requires an internet connection!

How Does It Work?

The first step is to detect if a required program is present on the system. This is done internally when "scp" or "cvs" is to be used. You can also check explicitly in a recipe with the :assertpkg command. This uses the $PATH environment variable. Aap will also look in its own directory, because this is where previously installed programs are sometimes located. Also see the :progsearch command and the program_path() function.

When the program is not found, the user is asked what is to be done:

    Cannot find package "scp"!
    1. Let Aap attempt installing the package
    2. Retry (install it yourself first)
    a. Abort
    Choice: 

When the user types "1" the automatic install will be invoked. That will be explained below. An alternative is to install the package yourself. This is useful if you know how to do this or when you don't think the automatic mechanism will work. After the package has been installed you can enter "2" and Aap continues executing the recipe. The last choice is "a", which means you give up and abort executing the recipe.

In situations where the package name differs from the command name, or there are more complicated requirements, you can write a check yourself. When a package needs to be installed the :installpkg command can be used to have Aap install the package.

To install a package automatically Aap will download a recipe and execute it. To be able to execute the recipe in a proper environment a directory is created. On Unix this will be "~/.aap/packages/{package-name}/". The "boot.aap" recipe is downloaded from the A-A-P web site. This uses a simple PHP script that selects the recipe to use. Example:

    http://www.a-a-p.org/package.php?package=scp&osname=posix

The downloaded recipe contains further instructions for building and/or installing the package. This can be anything, thus it is very flexible. Usually the recipe finds out what kind of system you are using and selects another recipe to be used for it. Or it uses the standard package mechanism of your system.

If you are running a BSD system you probably have the BSD ports system installed (if not, you should install it!). This is a very well maintained system that takes care of installing almost any software you can think of. The only disadvantage is that you need to be super-user to use it. Aap will ask you if you want to do this:

    The devel/templ port appears to exist.
    Do you want to become root and use the port? (y/n) 

If you respond with "y" you will be asked to type the root password and Aap will invoke the commands to build and install the BSD port. If you respond with "n" the generic Unix method will be used (if there is one).

On MS-Windows it will often be possible to download an executable file. This works without asking questions. Since there is no standard directory for executables, they are often placed in the Aap directory. You might want to move them to a directory in $PATH if you want to use them at the command line.

On generic Unix systems (Posix, Linux) the recipe will attempt to download a source archive, unpack it and build and install it. This mostly works, but may fail on some systems. You may have to do some steps manually then, possibly by running "configure" with specific arguments. You can find the downloaded files in "~/.aap/packages/{package-name}/". The Aap message log "AAPDIR/log" may contain hints about what went wrong. If you know the solution, please report this to the maintainer of the recipe, so that it can be made to work automatically.

Adding Support For a Package

To make it possible to automatically install a package at least one recipe needs to be written. This recipe is placed on the A-A-P website, so that every Aap user can find it without typing a URL. You need to e-mail this recipe to the maintainer of the A-A-P website: webmaster AT a-a-p.org. Chose the package name carefully, because it must be unique. Mostly it is the name of the command that was to be executed, such as "scp" or "cvs".

If there is only one recipe it must take care of all systems. This is useful for a Python module, for example. Or when you want to redirect to another site where the recipes for this specific package are stored. Here is an example:

    all install:
        :execute generic.aap {fetch = http://www.foo.org/recipes/%file%} $buildtarget

This recipe redirects everything to another recipe, which is downloaded from the URL specified with the "fetch" attribute. Note that the "all" and "install" targets are supported. "all" is used to build the package (as the current user) and "install" to install it (possibly requiring root privileges).

Installing on MS-Windows

For MS-Windows you often have a different method to install a package. Especially when the command to be installed is available as one executable program. Here is an example for the "scp" command:

      # Package recipe for SCP on MS-Windows.
      # Maintainer:  Bram Moolenaar <Bram@a-a-p.org>
      # Last Update: 2003 May 1

      install:
              # We use the scp command from PuTTY, it appears to work well.
              # For info about PuTTY see:
              #	http://www.chiark.greenend.org.uk/~sgtatham/putty/
              :print This will install SCP on MS-Windows.
              dir = "`Global.aap_bindir`"
              :mkdir {force} $dir
              :fetch {fetch = http://the.earth.li/~sgtatham/putty/latest/x86/pscp.exe
                      ftp://ftp.chiark.greenend.org.uk/users/sgtatham/putty-latest/x86/pscp.exe}
                      $dir/scp.exe
              # That's all, it doesn't need to be installed.

The actual work is done with one :fetch command. It specifies two locations where the program can be downloaded. Specifying several locations is useful, because servers may be down or unreachable from behind a firewall (some companies disable access to ftp servers).

Note the use of "Global.aap_bindir". This is the directory where Aap itself is located with "bin" added. Putting the executable there avoids asking the user to make a choice. This directory is always searched for executable commands, it does not have to be in $PATH. Double quotes are used for the case the directory contains a space (e.g., "C:\Program Files").

Building and Installing on Unix

For Unix things are generally a bit more complicated. The best is to use the package mechanism of the system. Using a BSD port was mentioned above. Most systems have an equivalent mechanism. When this is not available you need to fall back to compiling from sources. Here is an example for "cvs":

      # Package recipe for CVS on Posix.
      # Maintainer:  Bram Moolenaar <Bram@a-a-p.org>
      # Last Update: 2003 May 1

      PREFIX =

      name = cvs-1.11.5
      tarname = $(name).tar.gz

      all install:
          # There is no check for a BSD port here, because BSD systems should have a
          # cvs command already.

          # Ask the prefix before compiling.
          @if not _recipe.PREFIX:
              @_recipe.use_asroot, _recipe.PREFIX = ask_prefix("cvs")
              @if not _recipe.PREFIX:
                  :error No prefix given

          # Should use a port recipe for this...

          @if buildtarget == "all":
              # Get the sources and build the executable.
              # This can be done by an ordinary user.
              :print This will build "cvs" on Posix machines.
              :update get-tar
              :sys tar xfz $tarname
              :cd $name
              :sys ./configure --prefix=$PREFIX

              # The GSSAPI stuff breaks the build for me on FreeBSD.  If we can find
              # a solution it can work on other systems..
              :cat config.h | :eval re.sub('#define HAVE_GSSAPI\\b', '', stdin) >! config.h

              :sys make
          @else:
              # Install the executable.
              # This may need to be done by root.
              :cd $name
              @if _recipe.use_asroot:
                  # Assume the directories already exist...
                  :asroot make $buildtarget
              @else:
                  :sys make $buildtarget

      get-tar {virtual}: $tarname {fetch = http://ftp.cvshome.org/$name/%file%}

Note that this recipe specifically mentions the version "1.11-5". This is a bit unusual. Better is to refer to the latest stable version. Unfortunately, for CVS the proper link is not available. This means the recipe has to be updated every time a new stable version is released.

The user has the choice of installing CVS for himself or for everybody on the system. Aap has a build-in function for this: ask_prefix(). When the user is root it will return "1" and "/usr/local/" without asking, assuming the super-user will want to install for everybody (why else would he be doing this as root?). Normal users will be asked to make a choice:

    Select where to install cvs:
    1. become root and install in "/usr/local/"
    2. become root and install in specified location
    3. install in your home directory "/home/mool/"
    4. install in a specified location
    a. abort
    choice: 

It is obvious what the choices will do. The "cvs" recipe then continues to obtain the sources, using ":update get-tar" to allow using a cached file. The archive is unpacked with "tar" and "configure" is run before invoking "make". This is the standard way how most Unix programs are build.

There is one extra step: While testing the recipe it was discovered that the configuration makes a mistake and defines "HAVE_GSSAPI", but that doesn't work. The recipe modifes "config.h" to fix this. This is not a nice solution, but it makes the building work. This kind of porting would actually better be done in a separate recipe. And by reporting the problem to the maintainers of the CVS configure script.

When the recipe is invoked with the "install" target the choice to install as root or not is used. This was stored in "_recipe.use_asroot" to avoid having to make the choice again when invoked a second time (the recipe is first invoked with "all" to build the program as the current user and then with "install" to do the actuall install, possibly as super-user). The :asroot command passes the command to a separate shell with root privileges. The user is asked confirmation for every executed command for safety.

Installing a Specific Package

This automatic package installation system is a nice way of installing a program without the need to know how it's done. You can also use it to install a package directly:

      % aap --install scp

This will attempt to install the "scp" package on your system. It works just like when Aap discovered that the "scp" command was needed for executing a recipe.

Obviously not just any package is available. Quite a few currently, but hopefully this will grow when people submit their packages. When a package cannot be found you get an error message:

      % aap --install foobar
      Aap: Creating directory "/home/mool/.aap/packages/foobar"
      Aap: Entering directory `/home/mool/.aap/packages/foobar'
      Aap: Attempting download of "http://www.a-a-p.org/package.php?package=foobar&osname=posix"
      Aap: Downloaded "http://www.a-a-p.org/package.php?package=foobar&osname=posix" to "boot.aap"
      Aap: Error in recipe "/home/mool/.aap/packages/foobar/boot.aap" line 3: Sorry, package 'foobar' is not available.
      %

Cleaning Up

Currently Aap does not delete the files downloaded and generated while installing a package. This is useful especially when something fails, so that you can read the log file "AAPDIR/log" and/or do part of the installation manually. But this does mean disk space is used.

In the comments produced while installing the package you can see which directory is used for the files. This depends on the system and environment variables. It should be one of these:

Chapter 26. Debugging a Recipe

The log file shows what happened while A-A-P was executing. Often you can figure out what went wrong by looking at the messages.

The log file is named "AAPDIR/log". It is located in the directory of the main recipe. If you executed aap again and now want to see the previous log, it is named "AAPDIR/log1". Older logs are "AAPDIR/log2", "AAPDIR/log3", etc. This goes up to "AAPDIR/log9".

Messages

The kind of messages given can be changed with the MESSAGE variable. It is a comma separated list of message types for which the message is displayed. Other messages are still written in the log file.

The command line arguments "-v" and "-s" can be used to make the most often used selections:

Other values can be assigned at the command line. For example, to only see error and dependency messages:

        aap MESSAGE=error,depend  (other arguments)

Don't forget that excluding "error" means that no error messages are displayed!

No matter what messages are displayed, all messages are written in the log file. This can be used afterwards to see what actually happened.

Chapter 27. Differences from make

An Aap recipe has the same structure as a Makefile. But there are many differences. The most important and unexpected ones are mentioned here.

Build if file does not exist

In a Makefile a dependency with only a target is not executed if the target exists. With Aap the build commands will be executed in more situations:

• when the build commands were never executed

• when the build commands have changed

    gvim:
        ln -s vim gvim

    gvim:
        :symlink vim gvim

    gvim: {buildcheck=}
        :symlink vim gvim

Use Of Environment Variables

The "make" program uses all environment variables for variables in the Makefile. This can cause unexpected results, because you may have a large number of environment variables and some of them you didn't set yourself thus don't even know about them.

Aap does not use environment variables for recipe variables. A few environment variables are explicitly used. For example, $PATH is used to locate programs. To access an environment variable Python code must be used. The "os.environ" dictionary stores them. Example:

        Home = `os.environ.get("HOME")`

Note that some systems are case sensitive (e.g., Unix), some systems are not (e.g., MS-Windows).

Signatures Instead Of Timestamps

Make checks for outdated files by comparing timestamps. A target file is considered out-of-date when it's older than one of the source files. This means that Make will not notice a source file that was changed back to an older version. And Make has big problems when a source file has a timestamp in the future (happens when the system clock is turned back for some reason). The target will always be older, thus Make will build it every time.

The default check for Aap is to use MD5 signatures. This means a target is considered out-of-date if one of the source files is different from when this target was last build. Additionally, a signature is made for the build commands. If you change the commands that build the target it will also be considered out-of-date. Mostly this means Aap will build the target in many more situations.

If you want Aap to use timestamps like Make does, set the $DEFAULTCHECK variable to "newer". Also see the check attribute, it can be used to change the check for a specific dependency.

Chapter 28. Customizing Filetype Detection and Actions

See Chapter 8 for a simple example how to define a new filetype.

Filetype Detection

A-A-P detects the type of a file automatically. This is used to decide what tools can be used for a certain file.

The detection often uses the name of the file, especially the suffix. Extra suffixes like ".in" and ".gz" are ignored. Sometimes the contents of the file is inspected. for example, on Unix a shell script is recognized by "#!/bin/sh" in the first line.

To manually set the file type of an item add the "filetype" attribute. This overrules the automatic detection. Example:

        foo.o : foo.x {filetype = cpp}

Most detection is already built in. If this is not sufficient for your work, filetype detection instructions can be used to change the way file type detection works. These instructions can either be in a file or directly in the recipe:

        :filetype  filename
        :filetype
            suffix p pascal
            script .*bash$ sh

The advantage of using a file is that it will also be possible to use it when running the filetype detection as a separate program. The advantage of using the instructions directly in the recipe is that you don't have to create another file.

For the syntax of the file see Chapter 37.

It is sometimes desired to handle a group of files in a different way. For example, to use a different optimizer setting when compiling C files. An easy way to do this is to give these files a different filetype. Then define a compile action specifically for this filetype. Example:

        :attr {filetype = c_opt} bar.c foo.c
        :action compile c_opt
            OPTIMIZE = 3
            :do compile $source

When you define an action (or a route), Aap checks that the filetypes you use are known filetypes, i.e. mentioned somewhere in a :filetype command. If you just make up filetypes and use them in actions, Aap will give you a warning. This helps detect misspellings and the like. However, for the "optimized C" filetype above, this leads to a warning where you do not want one: c_opt is a proper filetype in this context. In order to declare a filetype without giving any rules to detect files of that type, use declare in a :filetype command:

:filetype declare c_opt

The detected filetypes never contain an underscore. A-A-P knows that the underscore separates the normal filetype from the additional part. When no action is found for the whole filetype, it is tried by removing the "_" and what comes after it. (This isn't fully implemented yet!)

Care should be taken that an action should not invoke itself. For example, to always compile "version.c" when linking a program:

        :attr {filetype = my_prog} $TARGET
        :action build my_prog object
            :do compile {target = version$OBJSUF} version.c
            :do build {target = $target} {filetype = program}
                                        $source version$OBJSUF

Without "{filetype = program}" on the :do build" command, the action would invoke itself, since the filetype for $TARGET is "my_prog".

Attributes on source arguments of the :do that start with "var_" are passed on to the executed action as variables. Examples:

          :do build foo.c {var_OPTIMIZE = 2}

          :attr {var_OPTIMIZE = 2} foo.c
          :do build foo.c

In the same way an attribute starting with "add_" is added to a variable. Example:

          :do build foo.c {add_DEFINE = -DBIG}

Variable values for the executed action can also be passed by giving an attribute just after the action:

          :do build {OPTIMIZE = 2} foo.c

And yet another method is to define a user scope and use this for the action:

          s_foo.OPTIMIZE = 4
          :do build {scope = s_foo} foo.c

Executing Actions

The user can select how to perform an action for each type of file. For example, the user may specify that the "view" action on a file of type "html" uses Netscape, and the "edit" action on a "html" file uses Vim.

To start an application:

:do actionname filename

This starts the action "actionname" for file "filename". Variables to be used by the commands can be specified after the action as attributes:

        :action convert default
            :sys convertcmd $?arg $source >$target
        ...
        :do convert {arg = -r} filename

The filetype is automatically detected from the file name(s) and possibly its contents. To overrule the automatic filetype detection, specify the filetype as an attribute on the file:

        :do convert filename {filetype = text}

This changes the filetype of the source file, the input of the action. To change the filetype of the output, the target of the action, use a "filetype" attribute on the action name:

  :do convert {filetype = html} filename {filetype = text}

Multiple filename arguments can be used:

        :do action file1 file2 file3

For some actions a target can or must be specified. This is done as an attribute on the action name:

        :do convert {target = outfile} infile1 infile2

Attributes of the input filenames other than "filetype" are not used to select the action that will be executed.

The "async" attribute can be used to start the application without waiting for it to finish. However, this only works for system commands and when multi-tasking is possible. Example:

        :do email {async} {remove} {to = piet} {subject = done building} logmessage

The "remove" attribute specifies that the files are to be deleted after the command is done, also when it fails.

When the filetype contains a "_" and no action can be found for it, the search for an action is done with the part before the "_". This is useful for specifying a variant on a filetype where some commands are different and the rest is the same.

"action" is usually one of these:

Examples:

        :do view COPYRIGHT {filetype = text}
        :do edit configargs.xml
        :do email {edit} bugreport
        :do build {target = myprog} main.c version.c
        :do compile foo.cpp

Default Actions

These are defined in the default.aap recipe.

Specifying Actions

Applications for an action-filetype pair can be specified with this command:

        :action action in-filetype
                commands

This associates the commands with action "action" and filetype "in-filetype". The commands are A-A-P commands, just like what is used in the build commands in a dependency.

The "in-filetype" is used for the source of the action, the type of the target is undefined. It is also possible to specify an action for turning a file of one filetype into another filetype. This is like a :rule command, but using filetypes instead of patterns.

        :action action out-filetype in-filetype
                commands

Actually, specifying an action with one filetype is like using "default" for the out-filetype.

Several variables are set and can be used by the commands:

Furthermore, all attributes of the action are turned into variables. Thus when ":do action {arg = -x} file " is used, $arg is set to "-x". Example for using an optional "arg" attribute:

        :action view foo
                :sys fooviewer $?arg $source

In Python code check if the {edit} attribute is specified:

        :action email default
            # when $subject and/or $to is missing editing is required
            @if not globals().get("subject") or not globals().get("to"):
                edit = 1
            @if globals().get("edit"):
                :do edit $fname
            :sys mail -s $subject $to < $fname

"action" and "ftype" can also be a comma-separated list of filetypes. There can't be any white space though. Examples:

        :action view html,xml,php
            :sys netscape --remote $source
        :action edit,view text,c,cpp
            :sys vim $source

"filetype" can be "default" to specify an action used when there is no action specifically for the filetype.

Note that the "async" attribute of the :do command may cause the :sys command to work asynchronously. That is done because the "async" attribute is turned into the "async" variable for the :action commands. If you don't want a specific :sys command to work asynchronously, reset "async":

        :action view foo
            tt = $?async
            async = 
            :sys foo_prepare
            async = $tt
            :sys foo $source

However, since two consecutive :sys commands are executed together, this should do the same thing:

        :action view foo
            :sys foo_prepare
            :sys foo $source

Quite often the program to be used depends on whether it is available on the system. For example, viewing HTML files can be done with netscape, mozilla, konquerer or another browser. We don't want to search the system for all kinds of programs when starting, it would make the startup time quite long. And we don't want to search each time the action is invoked, the result of the first search can be used. This construct can be used to achieve this:

        BROWSER =
        :action view html
            @if not _recipe.BROWSER:
                :progsearch _recipe.BROWSER netscape mozilla konquerer
            :sys $_recipe.BROWSER $source

The generic form form :progsearch is:

        :progsearch varname progname ...

The "_recipe" scope needs to be used for variables set in the commands and need to be used after the action is finished.

The first argument is the variable name to which to assign the resulting program name. When none of the programs is found an error message is given. Note that shell aliases are not found, only executable programs.

When the action uses a temporary file, make sure it is deleted even when one of the commands fail. The Python try/finally construction is ideal for this. Example:

        :action filter .pdf .txt
            tmp = `tempfname()`
            @try:
                :sys extract_docs $source >$tmp
                :sys docs2pdf $tmp $target
            @finally:
                :delete $tmp
            :print Converted $source to $target

When the ":sys" commands execute without trouble the ":delete" and ":print" commands are executed. When the first ":sys" commands causes an exception, execution continues with the ":delete" command and then the exception is handled. Thus the second ":sys" command is skipped and the ":print" command is not reached.

More examples:

        :action compile c,cpp
            :sys $CC $CPPFLAGS $CFLAGS -c $source
        :action build object
            :sys $CC $LDFLAGS $CFLAGS -o $target $source

Chapter 29. Customizing Automatic Depedencies

For various file types A-A-P can scan a source file for header files it includes. This implies that when a target depends on the source file, it also depends on the header files it includes. For example, a C language source file uses #include lines to include code from header files. The object file generated from this C source needs to be rebuilt when one of the header files changes. Thus the inclusing of the header file has an implied dependency.

Aap defines a series of standard dependency checks. You don't need to do anything to use them.

The filetype used for the dependency check is detected automatically. For files with an ignored suffix like ".in" and ".gz" no dependency checking is done.

To avoid all automatic dependency checks, set the variable "AUTODEPEND" to "off":

        AUTODEPEND = off

To avoid automatic dependencies for a specific file, set the attribute "autodepend" to "off":

        foo.o : foo.c {autodepend = off}

You can add your own dependency checks. This is done with the :action command. Its arguments are "depend" and the file types for which the check works. A block of commands follows, which is expected to inspect $source and produce the detected dependencies in $target, which has the form of a dependency. Example:

        :action depend c,cpp
            :sys $CC $CFLAGS -MM $source > $target

The build commands are expected to generate a file that specifies the dependency:

        foo.o : foo.c foo.h

The first item (before the colon) is ignored. The items after the colon are used as implied dependencies. The source file itself may appear, this is ignored. Thus these results have the same meaning:

        foo.xyz : foo.c foo.h
        foo.o : foo.h

Comments starting with "#" are ignored. Line continuation with "\" is supported. Only the first (continued) line is read.

Aap will take care of executing the dependency check when the source changes or when the command changes (e.g., the value of $CFLAGS). This can be changed with a "buildcheck" attribute after "depend".

        :action depend {buildcheck = $CFLAGS} c
            :sys $CC $CFLAGS -MM $source > $target

Aap expects the dependency checker to only inspect the source file. If it recursively inspects the files the source files include, this must be indicated with a "recursive" attribute. That avoids Aap will take care of this and do much more work than is required. Example:

        :action depend {recursive}  c,cpp
            :sys $CC $CFLAGS -MM $source > $target

Chapter 30. Customizing Default Tools

Tools are used to execute actions. Examples are a compiler and a linker. Each tool must be executed with specific arguments. But the recipe attempts to be portable, thus not include the literal command to be executed. The mechanism described in this chapter takes care of translating the generic action into invoking a specific tool with its arguments.

It all starts with an action. Let us use the compile action as an example. In the default recipe a default compile action is defined. This one works for most compilers that take arguments like the Unix "cc" command. If another kind of compiler is to be used, the $COMPILE_ACTION variable is set to the name of the action to be used, for example "compile_msvc". If $COMPILE_ACTION is set then the default action will invoke that action instead of using its generic compile command. $BUILD_ACTION is used for building in a similar way.

During startup the default recipe will check for existence of tools. A specific sequence of tools is checked for, depending on the platform. Thus in MS-Windows "msvc" will be checked for, while on Unix this doesn't happen. This is implemented with a Python module. For msvc it is "tools/msvc.py".

Each tool module defines an exists() function. It contains a check if the tool can be found. Mostly this is done by looking for a certain executable program. The msvc tool searches for "cl" (the compiler) and "vcvars32" (a command to start using the MSVC command line tools).

If a tool is detected, the actions for it are defined. This is always done, also when another tool was already detected. This allows the user to invoke specific actions or switch toolchain where he wants to. For MSVC the "compile_msvc" action is defined. That is like the normal "compile" action but with MSVC specific arguments.

The first tool that is detected will be used by default. The use_actions() function of the tool is invoked, which sets a number of variables, such as $COMPILE_ACTION, to the name of the action to be used. As mentioned above, the result will be that the generic actions will invoke the tool-specific action.

Adding A New Tool

You need to write a Python module and place it in the "tools" directory. Copy one of the existing tool files to start with. Then make changes to the functions:

Using A Specific Tool

The :usetool command can be used to specify a specific tool to be used in the current scope. When used in the toplevel recipe the tool becomes the default tool. When used in a child recipe the tool will be used in that recipe or by all actions invoked there. It can also be used in build commands, the tool will be used by invoked actions and dependencies.

Example:

:usetool mingw :update prog_A :usetool msvc :update prog_B

This actually works by defining the tool-specific actions and defining variables such as $COMPILE_ACTION in the current scope.

Chapter 31. Aap Command Line Arguments

Three Kinds Of Arguments

aap [option]... [assignment]... [target]...

The order of arguments is irrelevant. Options are explained in the next section.

Assignments take the form "VAR=value". This works just like putting this at the top of the main recipe used. But the shell used to start Aap may require escaping special characters, such as spaces. Putting the argument in double quotes often works (but not always).

Targets specify what needs to be done. If no target is given, one of the targets in the recipe is executed, see Recipe Execution Details.

Options

An option must appear only once, except for the ones that are noted to be allowed multiple times.

Chapter 32. Recipe Syntax

This defines the recipe syntax. It is not very strict, but you should be able to understand what is allowed and what isn't.

The intention is that recipes are always encoded in UTF-8. Currently this is not fully supported yet. US-ASCII works in any case.

A comment starts with "#" and continues until the end-of-line. It continues in the next line if the last character in the line is a backslash. A comment may appear where white space may appear, but not inside quotes.

        # comment \
        continued comment
        # another comment

White space may be inserted in between items. It is often ignored, but does have a meaning in a few places.

Line continuation may be done with a backslash immediately before an end-of-line. It is not needed for items that use extra indent to indicate that it continues on the next line.

The backslash can be also used for line continuation in Python commands. A leading @ char and white space before it is removed. Example:

        @ python command \
        @   continued python command \
               still continued

When lines are joined because a command continues in a line with more indent, the line break and leading white space of the next line are replaced with a single space. An exception is when the line ends in "$BR": The $BR is removed, the line break is inserted in the text and leading white space of the next line is removed.

All indent is computed with a tabstop setting of eight spaces.

For items that have a build_block, the start of the build block is the line with the smallest indent that is larger than the indent of the line that started the item. The lines before this are continuation lines of the command itself. Example:

        mytarget : source1
                      source2   # continuation line of dependency
            :print $source      # first line of the build block
                 something      # continuation line of :print command

Chapter 33. Variables and Scopes

Using Variables

A variable value is normally a string. The meaning of the value depends on where it is used. For example, "*" is interpreted as a wildcard when a variable is used where a file name is expected. Thus the wildcard is not expanded in the assignment. If you need it, use a Python expression to expand wildcards.

When using Python you can assign any type of value to a variable. Only a few types are supported for variables used in Aap commands:

In Python code the ExpandVar object needs to be expanded before you can use the value it contains. Use the var2string() function for that.

Normally using $VAR gets what you want. Aap will use the kind of quoting expected and add attributes when needed. This depends on the place where the variable is used. However, when you want something else, this can be specified:

In most places $var is expanded as $+'var (with attributes, using ' and " for quoting). The exceptions are:

The quoted variables don't handle the backslash as a special character. This is useful for MS-Windows file names. Example:

        prog : "dir\file 1.c"
                :print $'source

Results in: "dir\file 1.c"

Be careful with using "$\var" and quotes, you may not always get what you wanted.

RC-style expansion

RC-style expansion means that each item in a variable is concatenated to the item immediately before and after the variable. Example:

        var = one two three
        :print dir/$*var

Results in:

        dir/one dir/two dir/three

For the expansion the variable is used as a list of white-separated items. Quotes can be used to include white space in an item. Use double quotes around a single quote and single quotes around a double quote. Escaping the meaning of quotes with a backslash is not supported.

When concatenating variables and using rc-style expansion, the attributes of the last variable overrule the identical attributes of a previous one.

        v1 = foo {check = 1}
        v2 = bar {check = 2}
        vv = $*v1$v2

Is equivalent to:

        vv = foobar{check = 1}{check = 2}

When using rc-style expansion, quotes will not be kept as they are, but removed and re-inserted where used or necessary. Example:

        foo: "file 1.c" foo.c
                :print "dir/$*source"

Results in:

         "dir/file 1.c" "dir/foo.c"

Variable Indexing

To get one item out of a variable that is a list of items, use an index number in square brackets. Parenthesis or curly braces must be used around the variable name and the index. The first item is indexed with zero. Example:

        BAR = beer coffee cola
        :print $(BAR[0])

        BAR_ONE = $(BAR[2])
        :print $BAR_ONE

Results in:

        beer
        cola

Using an index for which no item exists gives an empty result. When $MESSAGE includes "warning" a message is printed about this.

Using Scopes

A dot is considered part of the variable name. It separates the scope name from the variable name within that scope. However, a trailing dot is not part of the variable name, so that this works:

          :print $result.     # print the result

In Python code you need to explicitly specify the scope name. When no scope name is given only the local scope is used. To get the equivalent of an Aap command that does not specify a scope, you need to use the "_no" scope in Python. The same example as above but now with a Python expression looks like this:

          :print `_no.result`.      # print the result

Predefined Scopes

Aap defines a scope for each recipe and each block of commands.

A user may also define a specific scope, see below. These scope names must start with an alphabetical name. Scope names starting with an underscore are used for predefined scopes.

Each time a block of commands is executed a new scope is created. Thus when executing the commands for a dependency a second time, its scope will not contain items from the first time.

A variable may exist in several scopes with a different value. To specify which scope is to be used, a scope name is prepended before the variable name, using a dot to separate the two.

These scope specifiers can be used to access a specific scope:

These scope specifiers can be used to search scopes to find a variable. The first scope in which the variable exists is used.

These are the scopes searched for a variable with the "_up" scope when it is used in a build command block:

1. Invoking command blocks The scope of the command block that invoked the current command block with a :do command, :update command or because of a dependency. Then the scope of the command block that invoked that command block, and so on. This excludes the toplevel.

2. The recipe in which the command block was defined.

3. Parent of the recipe in which the command block was defined. This goes on until and including the toplevel.

The "_no" scope is used for a variable in recipe commands without a specified scope. Thus these two are equivalent:

          :print $foobar
          :print $_no.foobar

But in Python commands a variable without a specified scope is always in the local scope. You must use "_no" to get the same effect:

          foo = $bar         # finds "bar" in local or "_up" scope
          @foo = bar         # finds "bar" in local scope only
          @foo = _no.bar     # finds "bar" in local or "_up" scope

When reading a variable with the "_no" scope it is first looked up in the local scope. If it does not exist, the "_stack" and "_tree" scopes are used, as explained above.

When writing a variable without a specified scope it is always put in the local scope. A specific situation where this may lead to an unexpected result is appending:

          foo += something

This is equivalent to:

          foo = $foo something

This obtains the value of "foo" from the first scope where it is defined, but it is set in the current scope. To change the variable where it is defined use the "_no" scope explicitly:

          _no.foo += something

User Scopes

The user can define a new scope by assigning a value to a variable, using the scope name:

        s_debug.foo = xxx

This creates the scope "s_debug" if it didn't exist yet. The variable "foo" within that scope is assigned the value "xxx".

The scope name must start with an alphabetic character. Following characters may be letters, digits and the underscore.

A user defined scope is only used when explicitly specified. The "_no" and "_up" scopes do not use it.

The scope can be accessed from everywhere, except recipes that create a new toplevel scope have their own set of user defined scopes. That is when using :execute or ":child {nopass}". ":execute {pass}" and :child do share the user scopes.

There cannot be a scope name and a variable with the same name. This applies to variables in ALL scopes! Thus when you have a scope "foo" in one place, you cannot use the variable "foo" anywhere else. The only exception is that you can use the variable "foo" in scopes that have been abandoned when the user scope "foo" is created, but that is tricky.

Recommendation: Let user scope names start with "s_".

A user scope can be specified for a dependency:

s_foo.OPTIMIZE = 4 ... foo : {scope = s_foo} foo.c :do build $source

A user scope can be specified for a rule:

:rule %.a : {scope = s_aaa} %.b

A user scope can be specified for an action:

:do foobar {scope = s_some} foo.bar

Variables In Build Commands

A dependency and a rule can have a list of commands. For these commands the following variables are available:

Example:

        doit {virtual}:
            :print building $target
        prog : "main file.c" doit
            :print building $target from $source

Results in:

          building doit{virtual=1}
          building prog from "main file.c"

Note that quoting of expanded $var depends on the command used.

The Python lists $source_list and $target_list can be used to loop over each item. Example:

        $OUT : foo.txt
            @for item in target_list:
                :print $source > $item

Note the difference between $source and $depend: $source only contains real files, $depend also contains virtual dependencies.

The list of dictionaries can be used to access the attributes of each item. Each dictionary has an entry "name", which is the (file) name of the item. Other entries are attributes. Example:

        prog : file.c {check = md5}
            @print sourcelist[0]["name"], sourcelist[0]["check"]

Results in: file.c md5

Chapter 34. Common Variables

This is a complete list of the variables that are currently used inside Aap, except the variables specifically used for porting, see Chapter 22 for that.

This list will be extended when more features are being added. To avoid the problem that your own variables interfere with the use of common Aap variables, do not use variable names with only upper case letters. Suggested scheme:

The following table lists the predefined variables. These types are used:

    CACHEPATH = $CACHEPATH

Chapter 35. Assignments

Assignment

overview:

Assignment with "+=" or "$+=" appends the argument as a separate item. This is actually done by inserting a space. But when the variable wasn't set yet and when it is empty it works like a normal assignment:

        VAR += something

is equal to:

        @if globals().get("_no.VAR"):
        @   VAR = _no.VAR + " " + "something"
        @else:
        @   VAR = "something"

Assignment with "?=" only does the assignment when the variable wasn't set yet. A variable that was set to an empty string also counts as being set. Thus when using "aap VAR=" the empty value overrules the value set with "?=".

        VAR ?= something

is equal to:

        @if not globals().has_key("_no.VAR"):
            VAR = something

When using "$=", "$+=" or "$?=" variables in the argument are not evaluated at the time of assignment, but this is done when the variable is used. The expansion is done in the scope where it is used, thus the result may depend on when and where the variable is used..

VAR = 1 TT $= $VAR VAR = 2 :print $TT

prints "2".

A variable with delayed evaluation cannot be used directly in Python code, because it is set the the class ExpandVar. See the var2string() function for expanding the variable in Python code.

When first setting a variable with "$=" and later appending with "+=" the evaluation is done before the new value is appended:

VAR = 1 TT $= $VAR TT += 2 VAR = 3 :print $TT

prints "1 2"

Note that evaluating a python expressions in `` is not postponed.

Block Assignment

The normal assignment command uses a single line of text. When broken into several lines they are joined together, just like with other commands. $BR can be used to insert a line break. Example:

        foo = first line$BR
                second line$BR
                third line $BR

The block assignment keeps the line breaks as they are. The same example but using a block assignment:

        foo << EOF
          first line
          second line
          third line 
            EOF

The generic format is:

        {var} << {term}
        line1
        ...
        {term}

{term} can be any string without white space. The block ends when {term} is found in a line by itself, optionally preceded by white space and followed by white space and a comment.

The amount of indent to be removed from all the lines is set by the first line. When the first line should start with white space use $( ).

All the variations of the assignment command can be used:

Chapter 36. Attributes

Attributes can be added to an item with the :attr command and by using them in a dependency or rule. Note that an assignment does not directly associate the attribute with a node. This only happens when the variable is used in an :attr command or a dependency.

The form for an attribute is:

        {name = value}

"value" is expanded like other items, with the addition that "}" cannot appear outside of quotes.

This form is also possible and uses the default value of 1:

        {name}

Examples:

        bar : thatfile {check = $MYCHECK}
        foo {virtual} : somefile

The "virtual" attribute is used for targets that don't exist (as file or directory) but are used for selecting the dependency to be built. These targets have the "virtual" attribute set by default:

The targets marked with "(added automatically)" will be added by Aap if they are not present. This is done for the toplevel and each child recipe.

These specific targets may have multiple build commands. They are all executed to update the virtual target. Normally there is up to one target in each (child) recipe.

Note that virtual targets are not related to a specific directory. Make sure no other item in this recipe or any child recipe has the same name as the virtual target to avoid confusion. Specifically using a directory "test" while there also is a virtual target "test". Name the directory "testdir" to avoid confusion.

The "comment" attribute can be used for targets that are to be specified at the command line. "aap comment" will show them.

        % aap comment
        target "all": build everything
        target "foo": link the program

Sticky Attributes

When attributes are used in a rule or dependency, most of them are only used for that dependency. But some attributes are "sticky": Once used for an item they are used everywhere for that item. Sticky attributes are:

The check attribute

The check attribute is used to specify what kind of signature is used for an item.

The default check for a file that was changed is an md5 checksum. Each time a recipe is executed the checksums for the relevant items are computed and stored in the file "AAPDIR/sign". The next time the recipe is executed the current and the old checksums are compared. When they are different, the build commands are executed. This means that when you put back an old version of a file, rebuilding will take place even though the timestamp of the source might be older than the target.

Another check can be specified with {check = name}, where "name" is the kind of check. Example:

        foo.txt : foo.db {check = time}
                :sys db_extract $source >$target

The default check is "md5". This is specified with the $DEFAULTCHECK variable. You can set this variable to "time" or "newer" to use timestamps instead of md5 signatures. The value of $DEFAULTCHECK is used when a node does not have a "check" attribute.

When mixing "newer" with other methods, the build rules are executed if the target is older than the source with the "newer" check, or when one of the signatures for the other items differs.

The "AAPDIR/sign" file is normally stored in the directory of the target. This means it will be found even when using several recipes that produce the same target. But for targets that get installed in system directories (use an absolute path), virtual targets and remote targets this is avoided. For these targets the "AAPDIR/sign" file is stored in the directory of the recipe that specifies how to build the target.

To overrule the directory where the "sign" file is written, use the attribute {signdirectory = name} for the target. To overrule the file where the signatures are written, use the attribute {signfile = name} for the target. "name" cannot end in "sign".

Handling Circular Dependencies

Two attributes can be used to handle circular dependencies:

The use can best be illustrated with an example:

      :attr {recursive = 3} index file.out

      index: file.out {update = no}
              # Get the current checksum for the index file.
              @sum = get_md5("index")

              # Generate the new index file from the output file.
              :system wc file.out >$target

              # Update the output file if the index file changed.
              @if sum != get_md5("index"):
                :update file.out

      file.out: file.in index {update = no}
              # Make sure index exists.
              @if not os.path.exists("index"):
                :print empty > index

              # Generate the output file.
              :cat $source >! $target

              # Need to generate the index file again.
              :update index

      all: file.out

The goal is to produce the file "file.out". It is created from "test.in" and "index". The "index" is created from "file.out", which includes the "index" file, thus a circular dependency exists. The idea is to repeat generating "file.out" until it no longer changes.

The "recursive" attribute is set to 3 for "index" and "file.out". This allows rebuilding "file.out" three times before giving up.

In the first dependency the "{update = no}" attribute is used to avoid updating "file.out". The build commands first update the "index" file before using :update to update "file.out". But this is only done when the index file has changed. That is where the circular dependency stops: When the generated index file no longer changes.

In the second dependency a similar thing is done: The "index" file is not updated before executing the build commands but as part of the build commands.

Chapter 37. Filetype detection

The filetype detection module basically takes a file name and returns the type of the file.

The A-A-P filetype detection is a separate module. You can use the filetype detection in recipes, as a standalone program and from any Python program.

A filetype name is made of lowercase ASCII letters and digits: a-z and 0-9.

The Program

Usage:

    Filetype.py [-I ruledir] ... [-f rulefile] ... filename

This will print the filetype of "filename" on stdout. When the type could not be detected the result is the string "None".

The "-I ruledir" argument can be used to specify a directory to load *.afd (Aap Filetype Detection) files from. These add rules for filetype detection. These are the default directories which are always scanned:

/usr/local/share/aap/afd/
~/.aap/afd/

The "-f rulefile" argument can be used to specify a file to load rules from.

Detection

Detection is done in this order:

1. early Python items

2. check the file name extensions

3. match the regular expressions with the file name

4. check the first line in the file for a matching script name

5. later Python items

When on a non-Posix system, the file name is forced to be lower case, so that case differences are ignored. The rules must use lower case names for this to work properly. Rules with an upper case letter will only match on a Posix system (this can be used for *.H to be recognized as cpp only on systems that make a difference between *.h and *.H).

The Python Module

The "ft_detect" function can be called to detect the type of file "fname":

        from Filetype import ft_detect
	type = ft_detect(fname)

A string with the detected filetype is returned. If the type is not recognized, ft_detect() returns the None value.

To ignore extra suffixes like ".in", ".gz", add an extra non-zero argument:

	type = ft_detect(fname, 1)

To influence the messages given, add an extra "dict" argument. The "MESSAGE" item will be used, see its explanation in the main documenation.

For more info about the Filetype module, see the comments at the start of Filetype.py.

Format Of Filetype Detection Rules

Blank lines and lines starting with "#" (preceded by any amount of white space) are ignored.

These filetype detection lines are supported:

In the above the first argument can be put in quotes to include white space. {type} can only consist of ASCII lowercase letters and digits.

Chapter 38. A-A-P Python functions

These Aap specific functions can be used in Python code:

While developing Aap some functions have been renamed. The old names are still available to keep old recipes from working. But some day these will be removed.

Chapter 39. A-A-P Commands

Commands grouped by functionality