Changes between Version 21 and Version 22 of pm-autotools


Ignore:
Timestamp:
Apr 16, 2007 9:08:05 PM (14 years ago)
Author:
lou
Comment:

Cleanup as part of BuildTools wiki update.

Legend:

Unmodified
Added
Removed
Modified
  • pm-autotools

    v21 v22  
    33= Working With the GNU Autotools =
    44
    5 Most of the COIN projects use the '''GNU autotools''' to manage the automatic configuration of the compilation process on a user's machine.  The goal is that users don't need to edit Makefiles by hand, and to make the code and compilation as portable as possible.
     5Most of the COIN projects use GNU autoconf, automake, and libtool (collectively, the '''GNU autotools''') to automatically configure the source code and the compilation process for a user's computing environment.  The goal is to spare users the chore of editing source and configuration files by hand, and to make the code and compilation as portable as possible.
    66
    7 Because the autotools try to serve many operating systems, compilers, and other environment characteristics, they might seem at the first sight somewhat complicated.  However, we tried to hide most of the details that are required to successully use those tools by encapsulating the difficult issues and provide a COIN project manager friendly layer on top of the autotools.
     7Because the autotools support many operating systems, compilers, and other environment characteristics, they might seem somewhat complicated at first glance.  However, we have tried to hide most of the details that are required to successfully use these tools by encapsulating the difficult issues and provide a COIN project manager friendly layer on top of the autotools.
    88
    99== Introduction ==
     
    1111In COIN, we make use of the three GNU packages [http://www.gnu.org/software/autoconf/ autoconf], [http://www.gnu.org/software/automake/ automake], and [http://www.gnu.org/software/libtool/ libtool].
    1212
    13 A good description of those tools and how they play together can be found in the [http://sources.redhat.com/autobook/ Autobook]; but the versions of the tools discussed there are not the most recent ones.  The documentation for each tool are here for [http://www.gnu.org/software/autoconf/manual/autoconf-2.59/ autoconf], [http://www.gnu.org/software/automake/manual/automake.html automake], and [http://www.gnu.org/software/libtool/manual.html libtool].
     13A good description of those tools and how they play together can be found in the [http://sources.redhat.com/autobook/ Autobook], but the versions of the tools discussed there are not the most recent ones.  The documentation for each tool can be found here for [http://www.gnu.org/software/autoconf/manual/autoconf-2.59/ autoconf], [http://www.gnu.org/software/automake/manual/automake.html automake], and [http://www.gnu.org/software/libtool/manual.html libtool].
    1414
    15 [wiki:pm-autotools-intro Here you find a short description of those tools and how they are used in COIN.]
     15[wiki:pm-autotools-intro Here you can find a short description of the autotools and how they are used in COIN.]
    1616
    17 As a project manager, you will need to write the input files for the Autotools.  Those files are the {{{configure.ac}}} and {{{Makefile.am}}} files.  Once those are generated, you need to run the Autotool programs to generate the final scripts and template Makefiles.  All this is described next.
     17As a project manager, you will need to write the input files for the autotools: the {{{configure.ac}}} and {{{Makefile.am}}} files.  Once they are written, you need to run the autotools programs to generate the {{{configure}}} script and {{{Makefile.in}}} template files.  All this is described in the following sections.
    1818
    19 == The configure.ac Files  ==
     19The autotools will allow you to do much, much more. The goal here is simply to get you started.
    2020
    21 The {{{configure}}} script is generated by {{{autoconf}}} based on the {{{configure.ac}}} input file.  This script usually contains autoconf macros and maybe some {{{/bin/sh}}} commands.  To make the usage of autoconf easier for COIN developers, we define a number of new autoconf macros (in the file {{{BuildTools/coin.m4}}}).
     21== The {{{configure.ac}}} Files  ==
    2222
    23  * '''Basic structure of the {{{configure.ac}}} file''':  The {{{configure.ac}}} files have a certain structure.  Even if you used autoconf before, you should read about [wiki:pm-structure-config the basic structure of COIN configure.ac files], since we are using a number of custom defined autoconf macros.
     23The {{{configure}}} script is generated by {{{autoconf}}} based on the {{{configure.ac}}} input file. {{{Configure.ac}}} usually contains autotools macro invocations and {{{sh}}} commands.  To make the use of autotools easier for COIN developers and to promote consistency across COIN projects, the !BuildTools project defines many new macros that provide many standard features. (You can examine the definitions in the file [source:stable/0.5/coin.m4]).
    2424
    25 In COIN, we distinguish between two types of {{{configure.ac}}} files:
     25In COIN, we distinguish between two types of {{{configure.ac}}} files, [wiki:pm-autotools#BaseDirConfigAc base directory configure.ac files] and [wiki:pm-autotools#ProjDirConfigAc project directory configure.ac files].
    2626
    27  * '''Base directory configure.ac files''': Many packages in COIN have a simple {{{configure}}} script in the base directory of a package (the directory {{{Coin-Clp}}} in the example for the [wiki:user-directories directory structure], which is the {{{trunk}}} directory in the repository).  This configure script usually only verifies which components are available in the subdirectories, and generates the main Makefile.  [wiki:pm-base-config Here we discuss the package base directory configure.ac file].
     27=== Basic structure of the {{{configure.ac}}} file ===
    2828
    29  * '''Project directory configure.ac files''': In the project subdirectories (such as the {{{CoinUtils}}} and {{{Clp}}} subdirectories in [wiki:user-directories directory structure example]) are the real configuration files.  This is where the tests for each separate project are performed.  The {{{configure.ac}}} files here can be more involved.  [wiki:pm-project-config Here we discuss the project main directory configure.ac file].
     29COIN {{{configure.ac}}} files have a certain structure.  Even if you've used {{{autoconf}}} before, you should read about [wiki:pm-structure-config the basic structure of COIN configure.ac files], since we are using a number of custom defined {{{autoconf}}} macros.
     30
     31=== Base directory {{{configure.ac}}} file === #BaseDirConfigAc
     32
     33Many packages in COIN use a simple {{{configure}}} script in the base directory of the package (''e.g.'', the directory {{{Coin-Clp}}} in the [wiki:user-directories directory structure example]).  This {{{configure}}} script usually does only a few tasks: create a {{{libtool}}} for use by the included packages, determine which components are available in subdirectories, and generate the main Makefile.  [wiki:pm-base-config Details of the corresponding configure.ac file can be found here].
     34
     35=== Package directory configure.ac files === #ProjDirConfigAc
     36
     37The real configuration files are contained in the project subdirectories (such as the {{{CoinUtils}}} or {{{Clp}}} subdirectories in the [wiki:user-directories directory structure example]). This is where the tests for each separate project are performed.  The {{{configure.ac}}} files here are usually more involved.  [wiki:pm-project-config Details of package directory configure.ac files can be found here].
    3038
    3139== The Makefile.am Files ==
    3240
    33 The Makefiles are generated by {{{automake}}} based on {{{Makefile.am}}} input files.  Those usually contain information on what is to be compiled (e.g., program, library) and what source files are required for each, where things should be installed. etc.  One can also add regular Makefile rules that will copied verbatim into the final Makefile.
     41{{{Makefiles}}} are generated by the {{{configure}}} script from {{{Makefile.in}}} template files. {{{Makefile.in}}} files are generated by {{{automake}}} from {{{Makefile.am}}} input files.  {{{Makefile.am}}} files contain information on what targets are to be built (''e.g.'', programs or libraries), what source files are required for each target, where libraries and include files should be installed, ''etc.''  One can also write regular Makefile rules that will copied verbatim into the final Makefile. Just remember that if you want to keep portability, you're writing for the 'lowest common denominator' implementation of {{{make}}}.
    3442
    35 In a typical COIN project we have a number of different types of {{{Makefile.am}}}, depending on what the purpose of the corresponding directory is.  Before looking at the descriptions below, you should read the [wiki:pm-automake-intro introduction of Automake concepts].
     43In a typical COIN project there are a number of different types of {{{Makefile.am}}}, depending on the purpose of the corresponding directory.  Before looking at the descriptions below, please read the [wiki:pm-automake-intro introduction to automake concepts].
    3644
    37  * '''Base directory Makefile.am''':   Even though this generates the only Makefile a user will usually work with, it is very simple.  It mainly defines, into which subdirectories {{{make}}} should recurse, and it typically defines additional targets, such as {{{tests}}} which runs {{{make test}}} in all subdirectories that have tests.  [wiki:pm-base-make The Makefile.am for the package base directory is discussed here].
     45 * '''Base directory Makefile.am file''':   Even though this generates the only {{{Makefile}}} a typical user will use directly, it is very simple.  It defines the subdirectories into which {{{make}}} should recurse to build the project, and it typically defines additional recursive targets (''e.g.'', {{{tests}}}, which will run '{{{make test}}}' in all subdirectories that have tests).  [wiki:pm-base-make Details of the package base directory Makefile.am can be found here].
    3846
    39  * '''Project main directory Makefile.am''':  In each project's main directory (the subdirectories under the base directory, such as {{{Cpl}}} and {{{CoinUtils}}} in the [wiki:user-directories directory structure example]), is the Makefile that takes care of project specific things, such defining how {{{make test}}} is run.  It also knows, into which source code directories {{{make}}} should recurse in order to build everything that the project has to provide.  [wiki:pm-project-make The main Makefile.am for a project is discussed here].
     47 * '''Project main directory Makefile.am file''':  Each project's main directory (the subdirectories under the package base directory, such as {{{Clp}}} or {{{CoinUtils}}} in the [wiki:user-directories directory structure example]) contains the {{{Makefile}}} that takes care of project-specific activities, such as defining how '{{{make test}}}' is run.  It also knows the source code directories into which {{{make}}} should recurse in order to build everything that the project must provide.  [wiki:pm-project-make Details of the main Makefile.am for a project can be found here].
    4048
    41  * '''Source code directory Makefile.am files''': This is where the real stuff happens.  Here one specifies what is to be built, what source files are required for this, where (and whether) the compiled files are to be installed.  [wiki:pm-source-make The Makefile.am files for the source code directories are discussed here].
     49 * '''Source code directory Makefile.am files''': This is where the actual build happens.  Here one specifies what is to be built, what source files are required, where (and whether) the compiled files are to be installed, ''etc.''  [wiki:pm-source-make Details of the Makefile.am files for the source code directories can be found here].
    4250
    43  * '''Unit Test Makefile.am files''':  Since those also compile a program, they are similar to source directory {{{Makefile.am}}} files, but usually simpler.  [wiki:pm-test-make The Makefile.am files for the test code directories are discussed here].
     51 * '''Unit Test Makefile.am files''':  These also provide specify how to compile a program and are thus similar to a source directory {{{Makefile.am}}} file, but they are usually simpler.  [wiki:pm-test-make Details of the Makefile.am files for the test code directories can be found here].
    4452
    45  * '''Example Makefile.in files''':  COIN projects that generate a library, should contain some example code that demonstrates how the library can be used.  Here, we also recommend to provide a [wiki:user-examples ''simple'' Makefile] to the user, that is adapted to the user's system and can be easily modified for the user's own application.  Since the Makefiles generated by {{{automake}}} are very complicated, we prefer to provide the template file {{{Makefile.in}}} directly instead one that is produced by {{{automake}}}.  If you want to provide such a Makefile for your example program, have a look at another existing COIN project and adapt it.
     53 * '''Example Makefile.in files''':  COIN projects that generate a library should provide some example code (typically a simple main program) that demonstrates how the library can be used. It is assumed that most users of COIN software will lack the skills or the environment to run the autotools; it follows, therefore, that providing a {{{Makefile.am}}} file for an example is pointless. It's also true that the Makefiles generated by {{{automake}}} are complex and generally unreadable by a human. For these reasons, project developers should provide a {{{Makefile.in}}} file which the {{{configure}}} script can process to generate a {{{Makefile}}}. It is recommended that the project provide a {{{Makefile.in}}} that produces a [wiki:user-examples ''simple'' Makefile] for the user that is configured for the user's system and can be easily modified for the user's own application. If you want to provide such a Makefile for your example program, have a look at existing COIN projects for examples and adapt them to your needs.
    4654
    4755== Running the Autotools ==
    4856
    49 [wiki:pm-get-autotools First, you should make sure that you are using the correct version of the Autotools.]
     57First, you should make sure that you are using the [wiki:pm-get-autotools correct version of the autotools.]
    5058
    51 You can run the Autotools in your project by going into the package's base directory ({{{Coin-Clp}}} in the [wiki:user-directories directory structure example]), and run the '''BuildTools/run_autotools''' script.  This script will copy the required auxilliary files into the base directory, and run the tools for every {{{configure.ac}}} file that is found in the base directory and its subdirectories.  If you run the script without arguments, the autotools will be run in every subdirectory (including the current one), where a {{{configure.ac}}} file is found.  You can also explicitly specify the directories as arguments, in which you want to run the autotools.
     59You can run the autotools on your package by going into the package's base directory ({{{Coin-Clp}}} in the [wiki:user-directories directory structure example]) and running the '''{{{BuildTools/run_autotools}}}''' script. When run with no parameters, the script will examine the current directory and subdirectories for {{{configure.ac}}} files. For each directory found to contain a {{{configure.ac}}} file, the script will copy the required auxiliary files into the  directory and run the autotools. You can also explicitly specify a set of directories as arguments. The {{{run_autotools}}} script also observes the environment variable {{{COIN_SKIP_PROJECTS}}}.
    5260
    53 Once you have a working version of your package and you can run {{{make}}}, you should (re)run the {{{configure}}} script with the '''--enable-maintainer-mode''' flag.  This will activate a number of Makefile rules that make the maintainance of the configuration and Makefiles easy.  When you now edit one of the {{{configure.ac}}} or {{{Makefile.am}}} files, the corresponding autotool program is run. If necessary, the {{{configure}}} script will be rerun, so that everything is always kept up-to-date.  Also, for COIN, when the {{{Externals}}} file, which contains the definitions of the subversion externals, has changed, the Makefile will automatically update the subversion {{{svn:externals}}} property of the base directory.
     61Once you have a working version of your package and you can run {{{make}}}, you should (re)run the {{{configure}}} script with the '''{{{--enable-maintainer-mode}}}''' parameter.  This will activate a number of makefile rules that simplify the maintenance of the configuration files. When you now edit a {{{configure.ac}}} or {{{Makefile.am}}} file, the corresponding autotool program will be run at the next execution of {{{make}}}. If necessary, the {{{configure}}} script will be rerun so that everything is kept up-to-date. Specific to COIN, when the {{{Externals}}} file, which contains the definitions of subversion externals, has changed, the subversion {{{svn:externals}}} property of the base directory will be automatically updated.
    5462
    55 A few notes of caution:
     63A few notes of caution when maintainer mode is enabled:
    5664
    57  * If you made a mistake in the {{{configure.ac}}} or {{{Makefile.am}}} input file, the {{{make}}} for updating everything might fail.  In some cases it is then require to rerun the autotools "by hand" after the error has been corrected, using the {{{BuildTools/run_autotools}}} script.
     65 * If changes to a {{{configure.ac}}} or {{{Makefile.am}}} input file introduce an error, the {{{make}}} for updating everything may fail.  It may be necessary to rerun the autotools "by hand" after the error has been corrected, using the {{{BuildTools/run_autotools}}} script.
    5866
    59  * When you made a change in a {{{configure.ac}}} file, a {{{make}}} will rerun the {{{configure}}} script (if you configured in the maintainer-mode), but not recursively.  If the change you made now requires the recursion into a new directory, you will have to rerun the {{{configure}}} script by hand.  To find out, with which options the {{{configure}}} script was run the last time, you can look at the beginning of the {{{config.log}}} output file.  Make sure that you don't use the {{{--no-create}}} and {{{--no-recursion}}} option when you rerun the script.
     67 * When you make a change in a {{{configure.ac}}} file, {{{make}}} will rerun the {{{configure}}} script, but ''not'' recursively.  If the change you made now requires recursion into a new directory, you will have to rerun the {{{configure}}} script by hand.  To determine the options used in the most recent run of the {{{configure}}} script, look at the beginning of the {{{config.log}}} output file.  Make sure that you don't use the {{{--no-create}}} or {{{--no-recursion}}} options when you rerun the script.
    6068
    61  * The Makefiles will update the {{{svn:externals}}} when the {{{Externals}}} file has changed, but it will not automatically run a {{{svn update}}} to get any new external (to avoid unwanted conflicts).  If you want to get the code for a new external, you need to run {{{svn update}}} by hand.
     69 * The base directory Makefile will update the {{{svn:externals}}} property when the {{{Externals}}} file has been changed, but it will not automatically run a '{{{svn update}}}' command to check out new external directories (to avoid unwanted conflicts).  To check out the code for a new external dependency, run '{{{svn update}}}' by hand.
    6270
    63 == Which Files Should be in the Subversion Repository ==
     71== Which Files Should be in the Subversion Repository? ==
    6472
    65 The following files should be included in the subversion repository:
     73The short answer is "Everything that's input for the autotools, or is created when the {{{run_autotools}}} script is run." Assume that users will not have the ability to run the autotools.
    6674
    67  * Every '''configure.ac''', '''configure''', '''Makefile.am''' and '''Makefile.in''' file, '''but no Makefile''' file.  Make sure, that the {{{svn:keywords}}} property for the {{{configure.ac}}} and {{{Makefile.am}}} files are set to "{{{Author Date Id Revision}}}".
     75More specifically, the following files should be included in the subversion repository:
    6876
    69  * In the base directory of the package, you also need to include: '''config.guess''', '''config.sub''', '''depcomp''', '''install-sh''', '''ltmain.sh''', and '''missing'''.  Those files will be copied into this directory automatically when you run {{{BuildTools/run_autotools}}} there.
     77 * Every '''{{{configure.ac}}}''' and '''{{{Makefile.am}}}''' file. Make sure that the {{{svn:keywords}}} properties are set to "{{{Author Date Id Revision}}}". In other words, treat them like source files.
    7078
    71  * You should also make sure that you have a subversion external dependency defined to {{{BuildTools/trunk}}} into a subdirectory {{{BuildTools}}} of the package's base directory, since otherwise the Autotools will not work.
     79 * Every '''{{{configure}}}''' and '''{{{Makefile.in}}}''' file. These are generated by the autotools and cannot be recreated by the average user. '''Do not''' include {{{Makefile}}}s; these are generated when the user runs the {{{configure}}} script.
     80
     81 * In the base directory of the package, include '''{{{config.guess}}}''', '''{{{config.sub}}}''', '''{{{depcomp}}}''', '''{{{install-sh}}}''', '''{{{ltmain.sh}}}''', and '''{{{missing}}}'''.  These files will be copied into the directory automatically when you run {{{BuildTools/run_autotools}}} from the base directory.
     82
     83 * You should also make sure that you have a subversion external dependency defined in your package's {{{Externals}}} file to check out {{{BuildTools/stable/}}}''current_release'' into a subdirectory named {{{BuildTools}}} in the package's base directory. Otherwise, the autotools will not work.
    7284
    7385== Working On Your Project ==
    7486
    75 If you are working on your project, you should '''run configure with the --enable-maintainer-mode flag'''.  This way, changes in your {{{configure.ac}}} and {{{Makefile.am}}} files will automatically trigger execution of the autotools programs.  If necessary, you can run the autotools "by hand", using the '''BuildTools/run_autotools''' script.
     87If you are working on your project, you should '''run configure with the --enable-maintainer-mode flag'''.  This way, changes in your {{{configure.ac}}} and {{{Makefile.am}}} files will automatically trigger execution of the autotools programs.  If necessary, you can run the autotools "by hand" using the {{{BuildTools/run_autotools}}} script.
    7688
    77 If you want to compile the code with debug information, you should use the '''--enable-debug''' flag when you run the {{{configure}}} script.  This will automatically switch on the debug compiler flag.  It will also switch to the usage of static libraries.  (When you compile only shared libraries, which is otherwise the default behavior, the executables generated will be in the {{{.libs}}} subdirectory, and not run properly, since the shared libraries might not be found.)
     89If you want to compile the package's code with debug information, you should use the '''--enable-debug''' parameter when you run the {{{configure}}} script.  This will add to the compile command the parameters necessary to produce code with debugging information.  It will also force the use of static libraries.  (This last sidesteps a few less-than-obvious consequences of using {{{libtool}}}. When shared libraries are built, the libraries remain hidden in the {{{.libs}}} subdirectories until '{{{make install}}}' is executed. Programs which depend on these libraries will most likely not run because the shared libraries cannot be found.)