1 | % Latex file containing the documentation of ADOL-C |
---|
2 | % |
---|
3 | % Copyright (C) Andrea Walther, Andreas Griewank, Andreas Kowarz, |
---|
4 | % Hristo Mitev, Sebastian Schlenkrich, Jean Utke, Olaf Vogel |
---|
5 | % |
---|
6 | % This file is part of ADOL-C. This software is provided as open source. |
---|
7 | % Any use, reproduction, or distribution of the software constitutes |
---|
8 | % recipient's acceptance of the terms of the accompanying license file. |
---|
9 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
10 | |
---|
11 | \documentclass[11pt,twoside]{article} |
---|
12 | \usepackage{hyperref} |
---|
13 | \usepackage{amsmath,amsthm,amssymb} |
---|
14 | \usepackage{graphicx} |
---|
15 | \usepackage{datetime} |
---|
16 | |
---|
17 | \newdateformat{monthyear}{\monthname\ \THEYEAR} |
---|
18 | |
---|
19 | \usepackage{color} |
---|
20 | |
---|
21 | \pagestyle{headings} |
---|
22 | \bibliographystyle{plain} |
---|
23 | \parskip=6pt |
---|
24 | |
---|
25 | \setlength{\textwidth}{433.6pt} |
---|
26 | \setlength{\oddsidemargin}{23pt} |
---|
27 | \setlength{\evensidemargin}{23pt} |
---|
28 | \setlength{\topmargin}{25.0pt} |
---|
29 | \setlength{\textheight}{580pt} |
---|
30 | \setlength{\baselineskip}{8pt} |
---|
31 | |
---|
32 | \newcommand{\N}{{ {\rm I} \kern -.225em {\rm N} }} |
---|
33 | \newcommand{\R}{{ {\rm I} \kern -.225em {\rm R} }} |
---|
34 | \newcommand{\T}{{ {\rm I} \kern -.425em {\rm T} }} |
---|
35 | |
---|
36 | \renewcommand{\sectionautorefname}{Section} |
---|
37 | \renewcommand{\subsectionautorefname}{Section} |
---|
38 | \renewcommand{\figureautorefname}{Figure} |
---|
39 | \renewcommand{\tableautorefname}{Table} |
---|
40 | |
---|
41 | \setcounter{equation}{0} |
---|
42 | |
---|
43 | \input{version.tex} |
---|
44 | |
---|
45 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
46 | \begin{document} |
---|
47 | |
---|
48 | \begin{titlepage} |
---|
49 | \begin{center} |
---|
50 | {\Large {\bf ADOL-C:}} |
---|
51 | \footnote{The development of earlier versions was supported by the Office of |
---|
52 | Scientific Computing, U.S. Department of Energy, the NSF, and the Deutsche |
---|
53 | Forschungsgemeinschaft. During the development of the current |
---|
54 | version Andrea Walther and Andreas Kowarz were supported by the |
---|
55 | grant Wa 1607/2-1 of the Deutsche Forschungsgemeinschaft} |
---|
56 | \vspace{0.2in} \\ |
---|
57 | % |
---|
58 | {\Large A Package for the Automatic Differentiation}\vspace{0.1in} \\ |
---|
59 | {\Large of Algorithms Written in C/C++}\\ |
---|
60 | \vspace{.2in} |
---|
61 | {\large\bf Version \packageversion, \monthyear\today} \\ |
---|
62 | \bigskip |
---|
63 | \mbox{Andrea Walther}\footnote{Institute of Mathematics, University |
---|
64 | of Paderborn, 33098 Paderborn, Germany} and |
---|
65 | \mbox{Andreas Griewank}\footnote{Department of Mathematics, |
---|
66 | Humboldt-Universit\"at zu Berlin, 10099 Berlin, Germany} |
---|
67 | \end{center} |
---|
68 | % |
---|
69 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
70 | \begin{abstract} |
---|
71 | The C++ package ADOL-C described here facilitates the evaluation of |
---|
72 | first and higher derivatives of vector functions that are defined |
---|
73 | by computer programs written in C or C++. The resulting derivative |
---|
74 | evaluation routines may be called from C, C++, Fortran, or any other |
---|
75 | language that can be linked with C. |
---|
76 | |
---|
77 | The numerical values of derivative vectors are obtained free |
---|
78 | of truncation errors at a small multiple of the run time and |
---|
79 | random access memory required by the given function evaluation program. |
---|
80 | Derivative matrices are obtained by columns, by rows or in sparse format. |
---|
81 | For solution curves defined by ordinary differential equations, |
---|
82 | special routines are provided that evaluate the Taylor coefficient vectors |
---|
83 | and their Jacobians with respect to the current state vector. |
---|
84 | For explicitly or implicitly defined functions derivative tensors are |
---|
85 | obtained with a complexity that grows only quadratically in their |
---|
86 | degree. The derivative calculations involve a possibly substantial but |
---|
87 | always predictable amount of data. Since the data is accessed strictly sequentially |
---|
88 | it can be automatically paged out to external files. |
---|
89 | \end{abstract} |
---|
90 | % |
---|
91 | {\bf Keywords}: Computational Differentiation, Automatic |
---|
92 | Differentiation, |
---|
93 | Chain Rule, Overloading, Taylor Coefficients, |
---|
94 | Gradients, Hessians, Forward Mode, Reverse Mode, |
---|
95 | Implicit Function Differentiation, Inverse Function Differentiation |
---|
96 | \medskip |
---|
97 | |
---|
98 | \noindent |
---|
99 | {\bf Abbreviated title}: Automatic differentiation by overloading in C++ |
---|
100 | % |
---|
101 | \end{titlepage} |
---|
102 | % |
---|
103 | \tableofcontents |
---|
104 | % |
---|
105 | |
---|
106 | \newpage |
---|
107 | % |
---|
108 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
109 | % |
---|
110 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
111 | \section{Preparing a Section of C or C++ Code for Differentiation} |
---|
112 | \label{prepar} |
---|
113 | % |
---|
114 | \subsection{Introduction} |
---|
115 | % |
---|
116 | \setcounter{equation}{0} |
---|
117 | The package \mbox{ADOL-C} |
---|
118 | utilizes overloading in C++, but the |
---|
119 | user has to know only C. The acronym stands for {\bf A}utomatic |
---|
120 | {\bf D}ifferentiation by {\bf O}ver{\bf L}oading in {\bf C}++. |
---|
121 | In contrast to source transformation approaches, overloading does not generate intermediate |
---|
122 | source code. |
---|
123 | As starting points to retrieve further information on techniques and |
---|
124 | application of automatic differentiation, as well as on other AD |
---|
125 | tools, we refer to the book \cite{GrWa08}. Furthermore, the web page |
---|
126 | \verb=http://www.autodiff.org= of the AD community forms a rich source |
---|
127 | of further information and pointers. |
---|
128 | |
---|
129 | |
---|
130 | ADOL-C facilitates the simultaneous |
---|
131 | evaluation of arbitrarily high directional derivatives and the |
---|
132 | gradients of these Taylor coefficients with respect to all independent |
---|
133 | variables. Relative to the cost of evaluating the underlying function, |
---|
134 | the cost for evaluating any such scalar-vector pair grows as the |
---|
135 | square of the degree of the derivative but is still completely |
---|
136 | independent of the numbers $m$ and $n$. |
---|
137 | |
---|
138 | This manual is organized as follows. This section explains the |
---|
139 | modifications required to convert undifferentiated code to code that |
---|
140 | compiles with ADOL-C. |
---|
141 | \autoref{tape} covers aspects of the tape of recorded data that ADOL-C uses to |
---|
142 | evaluate arbitrarily high order derivatives. The discussion includes storage |
---|
143 | requirements and the tailoring of certain tape characteristics to fit specific |
---|
144 | user needs. Descriptions of easy-to-use drivers for a convenient derivative |
---|
145 | evaluation are contained in \autoref{drivers}. |
---|
146 | \autoref{forw_rev_ad} offers a more mathematical characterization of |
---|
147 | the different modes of AD to compute derivatives. At the same time, the |
---|
148 | corresponding drivers of ADOL-C are explained. |
---|
149 | The overloaded derivative evaluation routines using the forward and the reverse |
---|
150 | mode of AD are explained in \autoref{forw_rev}. |
---|
151 | Advanced differentiation techniques as the optimal checkpointing for |
---|
152 | time integrations, the exploitation of fixed point iterations, the usages |
---|
153 | of external differentiated functions and the differentiation of OpenMP |
---|
154 | parallel programs are described in \autoref{adv_ad}. |
---|
155 | The tapeless forward mode is presented in \autoref{tapeless}. |
---|
156 | \autoref{install} details the installation and |
---|
157 | use of the ADOL-C package. Finally, \autoref{example} |
---|
158 | furnishes some example programs that incorporate the ADOL-C package to |
---|
159 | evaluate first and higher-order |
---|
160 | derivatives. These and other examples are distributed with the ADOL-C |
---|
161 | source code. |
---|
162 | The user should simply refer to them if the more abstract and general |
---|
163 | descriptions of ADOL-C provided in this document do not suffice. |
---|
164 | % |
---|
165 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
166 | \subsection{Declaring Active Variables} |
---|
167 | % |
---|
168 | \label{DecActVar} |
---|
169 | % |
---|
170 | The key ingredient of automatic differentiation by overloading is the |
---|
171 | concept of an {\em active variable}. All variables that may be |
---|
172 | considered as differentiable quantities at some time |
---|
173 | during the program execution must be of an active |
---|
174 | type. ADOL-C uses one |
---|
175 | active scalar type, called {\sf adouble}, whose real part is of the |
---|
176 | standard type {\sf double}. |
---|
177 | Typically, one will declare the independent variables |
---|
178 | and all quantities that directly or indirectly depend on them as |
---|
179 | {\em active}. Other variables that do not depend on the independent |
---|
180 | variables but enter, for example, as parameters, may remain one of the |
---|
181 | {\em passive} types {\sf double, float}, or {\sf int}. There is no |
---|
182 | implicit type conversion from {\sf adouble} to any of these passive |
---|
183 | types; thus, {\bf failure to declare variables as active when they |
---|
184 | depend on other active variables will result in a compile-time error |
---|
185 | message}. In data flow terminology, the set of active variable names |
---|
186 | must contain all its successors in the dependency graph. All components |
---|
187 | of indexed arrays must have the same activity status. |
---|
188 | |
---|
189 | The real component of an {\sf adouble x} can be extracted as |
---|
190 | {\sf x.value()}. In particular, |
---|
191 | such explicit conversions are needed for the standard output procedure |
---|
192 | {\sf printf}. The output stream operator \boldmath $\ll$ \unboldmath is overloaded such |
---|
193 | that first the real part of an {\sf adouble} and then the string |
---|
194 | ``{\sf (a)}" is added to the stream. The input stream operator \boldmath $\gg$ \unboldmath can |
---|
195 | be used to assign a constant value to an {\sf adouble}. |
---|
196 | Naturally, {\sf adouble}s may be |
---|
197 | components of vectors, matrices, and other arrays, as well as |
---|
198 | members of structures or classes. |
---|
199 | |
---|
200 | The C++ class {\sf adouble}, its member functions, and the overloaded |
---|
201 | versions of all arithmetic operations, comparison operators, and |
---|
202 | most ANSI C functions are contained in the file \verb=adouble.cpp= and its |
---|
203 | header \verb=<adolc/adouble.h>=. The latter must be included for compilation |
---|
204 | of all program files containing {\sf adouble}s and corresponding |
---|
205 | operations. |
---|
206 | % |
---|
207 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
208 | \subsection{Marking Active Sections} |
---|
209 | \label{markingActive} |
---|
210 | % |
---|
211 | All calculations involving active variables that occur between |
---|
212 | the void function calls |
---|
213 | \begin{center} |
---|
214 | {\sf trace\_on(tag,keep)} \hspace{0.3in} and \hspace{0.3in} |
---|
215 | {\sf trace\_off(file)} |
---|
216 | \end{center} |
---|
217 | are recorded on a sequential data set called {\em tape}. Pairs of |
---|
218 | these function calls can appear anywhere in a C++ program, but |
---|
219 | they must not overlap. The nonnegative integer argument {\sf tag} identifies the |
---|
220 | particular tape for subsequent function or derivative evaluations. |
---|
221 | Unless several tapes need to be kept, ${\sf tag} =0$ may be used throughout. |
---|
222 | The optional integer arguments {\sf keep} and |
---|
223 | {\sf file} will be discussed in \autoref{tape}. We will refer to the |
---|
224 | sequence of statements executed between a particular call to |
---|
225 | {\sf trace\_on} and the following call to {\sf trace\_off} as an |
---|
226 | {\em active section} of the code. The same active section may be |
---|
227 | entered repeatedly, and one can successively generate several traces |
---|
228 | on distinct tapes by changing the value of {\sf tag}. |
---|
229 | Both functions {\sf trace\_on} and {\sf trace\_off} are prototyped in |
---|
230 | the header file \verb=<adolc/taputil.h>=, which is included by the header |
---|
231 | \verb=<adolc/adouble.h>= automatically. |
---|
232 | |
---|
233 | Active sections may contain nested or even recursive calls to functions |
---|
234 | provided by the user. Naturally, their formal and actual parameters |
---|
235 | must have matching types. In particular, the functions must be |
---|
236 | compiled with their active variables declared as |
---|
237 | {\sf adouble}s and with the header file \verb=<adolc/adouble.h>= included. |
---|
238 | Variables of type {\sf adouble} may be declared outside an active section and need not |
---|
239 | go out of scope before the end of an active section. |
---|
240 | It is not necessary -- though desirable -- that free-store {\sf adouble}s |
---|
241 | allocated within |
---|
242 | an active section be deleted before its completion. The values of all |
---|
243 | {\sf adouble}s that exist at the beginning and end of an active section |
---|
244 | are automatically |
---|
245 | recorded by {\sf trace\_on} and {\sf trace\_off}, respectively. |
---|
246 | % |
---|
247 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
248 | \subsection{Selecting Independent and Dependent Variables} |
---|
249 | % |
---|
250 | One or more active variables that are read in or initialized to |
---|
251 | the values of constants or passive variables must be distinguished as |
---|
252 | independent variables. Other active variables that are similarly |
---|
253 | initialized may be considered as temporaries (e.g., a variable that |
---|
254 | accumulates the partial sums of a scalar product after being |
---|
255 | initialized to zero). In order to distinguish an active variable {\sf x} as |
---|
256 | independent, ADOL-C requires an assignment of the form |
---|
257 | \begin{center} |
---|
258 | {\sf x} \boldmath $\ll=$ \unboldmath {\sf px}\hspace{0.2in}// {\sf px} of any passive numeric type $\enspace .$ |
---|
259 | \end{center} |
---|
260 | This special initialization ensures that {\sf x.value()} = {\sf px}, and it should |
---|
261 | precede any other assignment to {\sf x}. However, {\sf x} may be reassigned |
---|
262 | other values subsequently. Similarly, one or more active variables {\sf y} |
---|
263 | must be distinguished as dependent by an assignment of the form |
---|
264 | \begin{center} |
---|
265 | {\sf y \boldmath $\gg=$ \unboldmath py}\hspace{0.2in}// {\sf py} of any passive type $\enspace ,$ |
---|
266 | \end{center} |
---|
267 | which ensures that {\sf py} = {\sf y.value()} and should not be succeeded |
---|
268 | by any other assignment to {\sf y}. However, a dependent variable {\sf y} |
---|
269 | may have been assigned other real values previously, and it could even be an |
---|
270 | independent variable as well. The derivative values calculated after |
---|
271 | the |
---|
272 | completion of an active section always represent {\bf derivatives of the final |
---|
273 | values of the dependent variables with respect to the initial values of the |
---|
274 | independent variables}. |
---|
275 | |
---|
276 | The order in which the independent and dependent variables are marked |
---|
277 | by the \boldmath $\ll=$ \unboldmath and \boldmath $\gg=$ \unboldmath statements matters crucially for the subsequent |
---|
278 | derivative evaluations. However, these variables do not have to be |
---|
279 | combined into contiguous vectors. ADOL-C counts the number of |
---|
280 | independent and dependent variable specifications within each active |
---|
281 | section and records them in the header of the tape. |
---|
282 | % |
---|
283 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
284 | \subsection{A Subprogram as an Active Section} |
---|
285 | % |
---|
286 | As a generic example let us consider a C(++) function of the form |
---|
287 | shown in \autoref{code1}. |
---|
288 | % |
---|
289 | \begin{figure}[hbt] |
---|
290 | \framebox[\textwidth]{\parbox{\textwidth}{ |
---|
291 | \begin{center} |
---|
292 | \begin{tabbing} |
---|
293 | {\sf void eval(}\= {\sf int n, int m,} \hspace{0.5 in} \= // number of independents and dependents\\ |
---|
294 | \>{\sf double *x,} \> // independent variable vector \\ |
---|
295 | \>{\sf double *y,} \> // dependent variable vector \\ |
---|
296 | \> {\sf int *k, } \> // integer parameters \\ |
---|
297 | \>{\sf double *z)} \> // real parameters \\ |
---|
298 | {\sf \{ }\hspace{0.1 in } \= \> // beginning of function body \\ |
---|
299 | \>{\sf double t = 0;} \> // local variable declaration \\ |
---|
300 | \>{\sf for (int i=0; i \boldmath $<$ \unboldmath n; i++)} \> // begin of computation \\ |
---|
301 | \>\hspace{0.2in}{\sf t += z[i]*x[i];} \> // continue \\ |
---|
302 | \>{\sf $\cdots \cdots \cdots \cdots $} \> // continue \\ |
---|
303 | \>{\sf y[m-1] = t/m; } \> // end of computation \\ |
---|
304 | {\sf \} } \> \> // end of function |
---|
305 | \end{tabbing} |
---|
306 | \end{center} |
---|
307 | }} |
---|
308 | \caption{Generic example of a subprogram to be activated} |
---|
309 | \label{code1} |
---|
310 | \end{figure} |
---|
311 | % |
---|
312 | |
---|
313 | If {\sf eval} is to be called from within an active C(++) |
---|
314 | section with {\sf x} |
---|
315 | and {\sf y} as vectors of {\sf adouble}s and the other parameters |
---|
316 | passive, then one merely has to change the type declarations of all |
---|
317 | variables that depend on {\sf x} from {\sf double} or {\sf float} to |
---|
318 | {\sf adouble}. Subsequently, the subprogram must be compiled with the |
---|
319 | header file \verb=<adolc/adouble.h>= included as described |
---|
320 | in \autoref{DecActVar}. Now let us consider the situation when {\sf eval} is |
---|
321 | still to be called with integer and real arguments, possibly from |
---|
322 | a program written in Fortran77, which does not allow overloading. |
---|
323 | |
---|
324 | To automatically compute derivatives of the dependent |
---|
325 | variables {\sf y} with respect to the independent variables {\sf x}, we |
---|
326 | can make the body of the function into an active section. For |
---|
327 | example, we may modify the previous program segment |
---|
328 | as in \autoref{adolcexam}. |
---|
329 | The renaming and doubling up of the original independent and dependent |
---|
330 | variable vectors by active counterparts may seem at first a bit clumsy. |
---|
331 | However, this transformation has the advantage that the calling |
---|
332 | sequence and the computational part, i.e., where the function is |
---|
333 | really evaluated, of {\sf eval} remain completely |
---|
334 | unaltered. If the temporary variable {\sf t} had remained a {\sf double}, |
---|
335 | the code would not compile, because of a type conflict in the assignment |
---|
336 | following the declaration. More detailed example codes are listed in |
---|
337 | \autoref{example}. |
---|
338 | |
---|
339 | \begin{figure}[htb] |
---|
340 | \framebox[\textwidth]{\parbox{\textwidth}{ |
---|
341 | \begin{center} |
---|
342 | \begin{tabbing} |
---|
343 | {\sf void eval(} \= {\sf int n,m,} \hspace{1.0 in}\= // number of independents and dependents\\ |
---|
344 | \> {\sf double *px,} \> // independent passive variable vector \\ |
---|
345 | \> {\sf double *py,} \> // dependent passive variable vector \\ |
---|
346 | \> {\sf int *k,} \> // integer parameters \\ |
---|
347 | \> {\sf double *z)} \> // parameter vector \\ |
---|
348 | {\sf \{}\hspace{0.1 in}\= \> // beginning of function body \\ |
---|
349 | \>{\sf short int tag = 0;} \> // tape array and/or tape file specifier\\ |
---|
350 | \>{\sf trace\_on(tag);} \> // start tracing \\ |
---|
351 | \>{\sf adouble *x, *y;} \> // declare active variable pointers \\ |
---|
352 | \>{\sf x = new adouble[n];}\>// declare active independent variables \\ |
---|
353 | \>{\sf y = new adouble[m];} \> // declare active dependent variables \\ |
---|
354 | \>{\sf for (int i=0; i \boldmath $<$ \unboldmath n; i++)} \\ |
---|
355 | \>\hspace{0.2in} {\sf x[i] \boldmath $\ll=$ \unboldmath px[i];} \> // select independent variables \\ |
---|
356 | \>{\sf adouble t = 0;} \> // local variable declaration \\ |
---|
357 | \>{\sf for (int i=0; i \boldmath $<$ \unboldmath n; i++)} \> // begin crunch \\ |
---|
358 | \>\hspace{0.2in}{\sf t += z[i]*x[i];} \> // continue crunch \\ |
---|
359 | \>{\sf $\cdots \cdots \cdots \cdots $} \> // continue crunch \\ |
---|
360 | \>{\sf $\cdots \cdots \cdots \cdots $} \> // continue crunch \\ |
---|
361 | \>{\sf y[m-1] = t/m; } \> // end crunch as before\\ |
---|
362 | \>{\sf for (int j=0; j \boldmath $<$ \unboldmath m; j++)} \\ |
---|
363 | \>\hspace{0.2in}{\sf y[j] \boldmath $\gg=$ \unboldmath py[j];} \> // select dependent variables \\ |
---|
364 | \>{\sf delete[] y;} \>// delete dependent active variables \\ |
---|
365 | \>{\sf delete[] x;} \>// delete independent active variables \\ |
---|
366 | \>{\sf trace\_off();} \> // complete tape \\ |
---|
367 | {\sf \}} \>\> // end of function |
---|
368 | \end{tabbing} |
---|
369 | \end{center}}} |
---|
370 | \caption{Activated version of the code listed in \autoref{code1}} |
---|
371 | \label{adolcexam} |
---|
372 | \end{figure} |
---|
373 | % |
---|
374 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
375 | \subsection{Overloaded Operators and Functions} |
---|
376 | \label{OverOper} |
---|
377 | % |
---|
378 | As in the subprogram discussed above, the actual computational |
---|
379 | statements of a C(++) code need not be altered for the purposes of |
---|
380 | automatic differentiation. All arithmetic operations, as well as the |
---|
381 | comparison and assignment operators, are overloaded, so any or all of |
---|
382 | their operands can be an active variable. An {\sf adouble x} occurring |
---|
383 | in a comparison operator is effectively replaced by its real value |
---|
384 | {\sf x.value()}. Most functions contained in the ANSI C standard for |
---|
385 | the math library are overloaded for active arguments. The only |
---|
386 | exceptions are the non-differentiable functions {\sf fmod} and |
---|
387 | {\sf modf}. Otherwise, legitimate C code in active sections can remain |
---|
388 | completely unchanged, provided the direct output of active variables |
---|
389 | is avoided. The rest of this subsection may be skipped by first time |
---|
390 | users who are not worried about marginal issues of differentiability |
---|
391 | and efficiency. |
---|
392 | |
---|
393 | The modulus {\sf fabs(x)} is everywhere Lipschitz continuous but not |
---|
394 | properly differentiable at the origin, which raises the question of |
---|
395 | how this exception ought to be handled. Fortunately, one can easily |
---|
396 | see that {\sf fabs(x)} and all its compositions with smooth |
---|
397 | functions are still directionally differentiable. These |
---|
398 | directional derivatives of arbitrary order can be propagated in the |
---|
399 | forward mode without any ambiguity. In other words, the forward mode as |
---|
400 | implemented in ADOL-C computes Gateaux derivatives |
---|
401 | in certain directions, which reduce to Fr\'echet derivatives only |
---|
402 | if the dependence on the direction is linear. Otherwise, |
---|
403 | the directional derivatives are merely positively homogeneous with |
---|
404 | respect to the scaling of the directions. |
---|
405 | For the reverse mode, ADOL-C sets the derivative of {\sf fabs(x)} at |
---|
406 | the origin somewhat arbitrarily to zero. |
---|
407 | |
---|
408 | We have defined binary functions {\sf fmin} and {\sf fmax} for {\sf adouble} |
---|
409 | arguments, so that function and derivative values are obtained consistent |
---|
410 | with those of {\sf fabs} according to the identities |
---|
411 | \[ |
---|
412 | \min(a,b) = [a+b-|a-b|]/2 \quad {\rm and} \quad |
---|
413 | \max(a,b) = [a+b+|a-b|]/2 \quad . |
---|
414 | \] |
---|
415 | These relations cannot hold if either $a$ or $b$ is infinite, in which |
---|
416 | case {\sf fmin} or {\sf fmax} and their derivatives may still be well |
---|
417 | defined. It should be noted that the directional differentiation of |
---|
418 | {\sf fmin} and {\sf fmax} yields at ties $a=b$ different results from |
---|
419 | the corresponding assignment based on the sign of $a-b$. For example, |
---|
420 | the statement |
---|
421 | \begin{center} |
---|
422 | {\sf if (a $<$ b) c = a; else c = b;} |
---|
423 | \end{center} |
---|
424 | yields for {\sf a}~=~{\sf b} and {\sf a}$^\prime < $~{\sf b}$^\prime$ |
---|
425 | the incorrect directional derivative value |
---|
426 | {\sf c}$^\prime = $~{\sf b}$^\prime$ rather than the correct |
---|
427 | {\sf c}$^\prime = $~{\sf a}$^\prime$. Therefore this form of conditional assignment |
---|
428 | should be avoided by use of the function $\sf fmin(a,b)$. There |
---|
429 | are also versions of {\sf fmin} and {\sf fmax} for two passive |
---|
430 | arguments and mixed passive/active arguments are handled by |
---|
431 | implicit conversion. |
---|
432 | On the function class obtained by composing the modulus with real |
---|
433 | analytic functions, the concept of directional differentiation can be |
---|
434 | extended to the propagation of unique one-sided Taylor expansions. |
---|
435 | The branches taken by {\sf fabs, fmin}, and {\sf fmax}, are recorded |
---|
436 | on the tape. |
---|
437 | |
---|
438 | The functions {\sf sqrt}, {\sf pow}, and some inverse trigonometric |
---|
439 | functions have infinite slopes at the boundary points of their domains. |
---|
440 | At these marginal points the derivatives are set by ADOL-C to |
---|
441 | either {\sf $\pm$InfVal}, 0 |
---|
442 | or {\sf NoNum}, where {\sf InfVal} and {\sf NoNum} are user-defined |
---|
443 | parameters, see \autoref{Customizing}. |
---|
444 | On IEEE machines {\sf InfVal} can be set to the special value |
---|
445 | {\sf Inf}~=~$1.0/0.0$ and {\sf NoNum} to {\sf NaN}~=~$0.0/0.0$. |
---|
446 | For example, at {\sf a}~=~0 the first derivative {\sf b}$^\prime$ |
---|
447 | of {\sf b}~=~{\sf sqrt(a)} is set to |
---|
448 | \[ |
---|
449 | {\sf b}^\prime = \left\{ |
---|
450 | \begin{array}{ll} |
---|
451 | \sf InfVal&\mbox{if}\;\; {\sf a}^\prime>0 \\ |
---|
452 | 0&\mbox{if}\;\;{\sf a}^\prime =0 \\ |
---|
453 | \sf NoNum&\mbox{if}\;\;{\sf a}^\prime <0\\ |
---|
454 | \end{array} \right. \enspace . |
---|
455 | \] |
---|
456 | In other words, we consider {\sf a} and |
---|
457 | consequently {\sf b} as a constant when {\sf a}$^\prime$ or more generally |
---|
458 | all computed Taylor coefficients are zero. |
---|
459 | |
---|
460 | The general power function ${\sf pow(x,y)=x^y}$ is computed whenever |
---|
461 | it is defined for the corresponding {\sf double} arguments. If {\sf x} is |
---|
462 | negative, however, the partial derivative with respect to an integral exponent |
---|
463 | is set to zero. |
---|
464 | %Similarly, the partial of {\bf pow} with respect to both arguments |
---|
465 | %is set to zero at the origin, where both arguments vanish. |
---|
466 | The derivatives of the step functions |
---|
467 | {\sf floor}, {\sf ceil}, {\sf frexp}, and {\sf ldexp} are set to zero at all |
---|
468 | arguments {\sf x}. The result values of the step functions |
---|
469 | are recorded on the tape and can later be checked to recognize |
---|
470 | whether a step to another level was taken during a forward sweep |
---|
471 | at different arguments than at taping time. |
---|
472 | |
---|
473 | Some C implementations supply other special |
---|
474 | functions, in particular the error function {\sf erf(x)}. For the |
---|
475 | latter, we have included an {\sf adouble} version in \verb=<adouble.cpp>=, which |
---|
476 | has been commented out for systems on which the {\sf double} valued version |
---|
477 | is not available. The increment and decrement operators {\sf ++}, \boldmath $--$ \unboldmath (prefix and |
---|
478 | postfix) are available for {\sf adouble}s. |
---|
479 | % |
---|
480 | % XXX: Vector and matrix class have to be reimplemented !!! |
---|
481 | % |
---|
482 | % and also the |
---|
483 | %active subscripts described in the \autoref{act_subscr}. |
---|
484 | Ambiguous statements like {\sf a += a++;} must be |
---|
485 | avoided because the compiler may sequence the evaluation of the |
---|
486 | overloaded |
---|
487 | expression differently from the original in terms of {\sf double}s. |
---|
488 | |
---|
489 | As we have indicated above, all subroutines called with active arguments |
---|
490 | must be modified or suitably overloaded. The simplest procedure is |
---|
491 | to declare the local variables of the function as active so that |
---|
492 | their internal calculations are also recorded on the tape. |
---|
493 | Unfortunately, this approach is likely to be unnecessarily inefficient |
---|
494 | and inaccurate if the original subroutine evaluates a special function |
---|
495 | that is defined as the solution of a particular mathematical problem. |
---|
496 | The most important examples are implicit functions, quadratures, |
---|
497 | and solutions of ordinary differential equations. Often |
---|
498 | the numerical methods for evaluating such special functions are |
---|
499 | elaborate, and their internal workings are not at all differentiable in |
---|
500 | the data. Rather than differentiating through such an adaptive |
---|
501 | procedure, one can obtain first and higher derivatives directly from |
---|
502 | the mathematical definition of the special function. Currently this |
---|
503 | direct approach has been implemented only for user-supplied quadratures |
---|
504 | as described in \autoref{quadrat}. |
---|
505 | % |
---|
506 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
507 | \subsection{Reusing the Tape for Arbitrary Input Values} |
---|
508 | \label{reuse_tape} |
---|
509 | % |
---|
510 | In some situations it may be desirable to calculate the value and |
---|
511 | derivatives of a function at arbitrary arguments by using a tape of |
---|
512 | the function evaluation at one argument and reevaluating the |
---|
513 | function and its derivatives using the given ADOL-C |
---|
514 | routines. This approach can |
---|
515 | significantly reduce run times, and it |
---|
516 | also allows to port problem functions, in the form of the |
---|
517 | corresponding tape files, into a computing environment that |
---|
518 | does not support C++ but does support C or Fortran. |
---|
519 | Therefore, the routines provided by ADOL-C for the evaluation of derivatives |
---|
520 | can be used to at arguments $x$ other than the |
---|
521 | point at which the tape was generated, provided there are |
---|
522 | no user defined quadratures and all comparisons involving |
---|
523 | {\sf adouble}s yield the same result. The last condition |
---|
524 | implies that the control flow is unaltered by the change |
---|
525 | of the independent variable values. Therefore, this sufficient |
---|
526 | condition is tested by ADOL-C and if it is not met |
---|
527 | the ADOL-C routine called for derivative calculations indicates this |
---|
528 | contingency through its return value. Currently, there are six return values, |
---|
529 | see \autoref{retvalues}. |
---|
530 | \begin{table}[h] |
---|
531 | \center\small |
---|
532 | \begin{tabular}{|r|l|}\hline |
---|
533 | +3 & |
---|
534 | \begin{minipage}{12.5cm} |
---|
535 | \vspace*{1ex} |
---|
536 | The function is locally analytic. |
---|
537 | \vspace*{1ex} |
---|
538 | \end{minipage} \\ \hline |
---|
539 | +2 & |
---|
540 | \begin{minipage}{12.5cm} |
---|
541 | \vspace*{1ex} |
---|
542 | The function is locally analytic but the sparsity |
---|
543 | structure (compared to the situation at the taping point) |
---|
544 | may have changed, e.g. while at taping arguments |
---|
545 | {\sf fmax(a,b)} returned {\sf a} we get {\sf b} at |
---|
546 | the argument currently used. |
---|
547 | \vspace*{1ex} |
---|
548 | \end{minipage} \\ \hline |
---|
549 | +1 & |
---|
550 | \begin{minipage}{12.5cm} |
---|
551 | \vspace*{1ex} |
---|
552 | At least one of the functions {\sf fmin}, {\sf fmax} or {\sf fabs} |
---|
553 | is evaluated at a tie or zero, respectively. Hence, the function to be differentiated is |
---|
554 | Lipschitz-continuous but possibly non-differentiable. |
---|
555 | \vspace*{1ex} |
---|
556 | \end{minipage} \\ \hline |
---|
557 | 0 & |
---|
558 | \begin{minipage}{12.5cm} |
---|
559 | \vspace*{1ex} |
---|
560 | Some arithmetic comparison involving {\sf adouble}s yields a tie. |
---|
561 | Hence, the function to be differentiated may be discontinuous. |
---|
562 | \vspace*{1ex} |
---|
563 | \end{minipage} \\ \hline |
---|
564 | -1 & |
---|
565 | \begin{minipage}{12.5cm} |
---|
566 | \vspace*{1ex} |
---|
567 | An {\sf adouble} comparison yields different results |
---|
568 | from the evaluation point at which the tape was generated. |
---|
569 | \vspace*{1ex} |
---|
570 | \end{minipage} \\ \hline |
---|
571 | -2 & |
---|
572 | \begin{minipage}{12.5cm} |
---|
573 | \vspace*{1ex} |
---|
574 | The argument of a user-defined quadrature has changed |
---|
575 | from the evaluation point at which the tape was generated. |
---|
576 | \vspace*{1ex} |
---|
577 | \end{minipage} \\ \hline |
---|
578 | \end{tabular} |
---|
579 | \caption{Description of return values} |
---|
580 | \label{retvalues} |
---|
581 | \end{table} |
---|
582 | |
---|
583 | \begin{figure}[h] |
---|
584 | \centering\includegraphics[width=10.0cm]{tap_point} |
---|
585 | \caption{Return values around the taping point} |
---|
586 | \label{fi:tap_point} |
---|
587 | \end{figure} |
---|
588 | |
---|
589 | In \autoref{fi:tap_point} these return values are illustrated. |
---|
590 | If the user finds the return value of an ADOL-C routine to be negative the |
---|
591 | taping process simply has to be repeated by executing the active section again. |
---|
592 | The crux of the problem lies in the fact that the tape records only |
---|
593 | the operations that are executed during one particular evaluation of the |
---|
594 | function. |
---|
595 | It also has no way to evaluate integrals since the corresponding |
---|
596 | quadratures are never recorded on the tape. |
---|
597 | Therefore, when there are user-defined quadratures the retaping is necessary at each |
---|
598 | new point. If there are only branches conditioned on {\sf adouble} |
---|
599 | comparisons one may hope that re-taping becomes unnecessary when |
---|
600 | the points settle down in some small neighborhood, as one would |
---|
601 | expect for example in an iterative equation solver. |
---|
602 | % |
---|
603 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
604 | \subsection{Conditional Assignments} |
---|
605 | \label{condassign} |
---|
606 | % |
---|
607 | It appears unsatisfactory that, for example, a simple table lookup |
---|
608 | of some physical property forces the re-recording of a possibly |
---|
609 | much larger calculation. However, the basic philosophy of ADOL-C |
---|
610 | is to overload arithmetic, rather than to generate a new program |
---|
611 | with jumps between ``instructions'', which would destroy the |
---|
612 | strictly sequential tape access and |
---|
613 | require the infusion of substantial compiler technology. |
---|
614 | Therefore, we introduce the two constructs of conditional |
---|
615 | assignments and active integers as partial remedies to the |
---|
616 | branching problem. |
---|
617 | |
---|
618 | In many cases, the functionality of branches |
---|
619 | can be replaced by conditional assignments. |
---|
620 | For this purpose, we provide a special function called |
---|
621 | {\sf condassign(a,b,c,d)}. Its calling sequence corresponds to the |
---|
622 | syntax of the conditional assignment |
---|
623 | \begin{center} |
---|
624 | {\sf a = (b \boldmath $>$ \unboldmath 0) ? c : d;} |
---|
625 | \end{center} |
---|
626 | which C++ inherited from C. However, here the arguments are restricted to be |
---|
627 | active or passive scalar arguments, and all expression arguments |
---|
628 | are evaluated before the test on {\sf b}, which is different from |
---|
629 | the usual conditional assignment or the code segment. |
---|
630 | |
---|
631 | Suppose the original program contains the code segment |
---|
632 | \begin{center} |
---|
633 | {\sf if (b \boldmath $>$ \unboldmath 0) a = c; else a = d;}\\ |
---|
634 | \end{center} |
---|
635 | Here, only one of the expressions (or, more generally, program blocks) |
---|
636 | {\sf c} and {\sf d} is evaluated, which exactly constitutes the problem |
---|
637 | for ADOL-C. To obtain the correct value {\sf a} with ADOL-C, one |
---|
638 | may first execute both branches and then pick either {\sf c} |
---|
639 | or {\sf d} using |
---|
640 | {\sf condassign(a,b,c,d)}. To maintain |
---|
641 | consistency with the original code, one has to make sure |
---|
642 | that the two branches do not have any side effects that can |
---|
643 | interfere with each other or may be important for subsequent |
---|
644 | calculations. Furthermore the test parameter {\sf b} has to be an |
---|
645 | {\sf adouble} or an {\sf adouble} expression. Otherwise the |
---|
646 | test condition {\sf b} is recorded on the tape as a {\em constant} with its |
---|
647 | run time value. Thus the original dependency of {\sf b} on |
---|
648 | active variables gets lost, for instance if {\sf b} is a comparison |
---|
649 | expression, see \autoref{OverOper}. |
---|
650 | If there is no {\sf else} part in a conditional assignment, one may call |
---|
651 | the three argument version |
---|
652 | {\sf condassign(a,b,c)}, which |
---|
653 | is logically equivalent to {\sf condassign(a,b,c,a)} in that |
---|
654 | nothing happens if {\sf b} is non-positive. |
---|
655 | The header file \verb=<adolc/adouble.h>= |
---|
656 | contains also corresponding definitions of |
---|
657 | {\sf condassign(a,b,c,d)} |
---|
658 | and {\sf condassign(a,b,c)} for |
---|
659 | passive {\sf double} arguments so that the modified code |
---|
660 | without any differentiation can be tested |
---|
661 | for correctness. |
---|
662 | % |
---|
663 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
664 | \subsection{Step-by-Step Modification Procedure} |
---|
665 | % |
---|
666 | To prepare a section of given C or C++ code for automatic |
---|
667 | differentiation as described above, one applies the following step-by-step procedure. |
---|
668 | \begin{enumerate} |
---|
669 | \item |
---|
670 | Use the statements {\sf trace\_on(tag)} or {\sf trace\_on(tag,keep)} |
---|
671 | and {\sf trace\_off()} or {\sf trace\_off(file)} to mark the |
---|
672 | beginning and end of the active section. |
---|
673 | \item |
---|
674 | Select the set of active variables, and change their type from |
---|
675 | {\sf double} or {\sf float} to {\sf adouble}. |
---|
676 | \item |
---|
677 | Select a sequence of independent variables, and initialize them with |
---|
678 | \boldmath $\ll=$ \unboldmath assignments from passive variables or vectors. |
---|
679 | \item |
---|
680 | Select a sequence of dependent variables among the active variables, |
---|
681 | and pass their final values to passive variable or vectors thereof |
---|
682 | by \boldmath $\gg=$ \unboldmath assignments. |
---|
683 | \item |
---|
684 | Compile the codes after including the header file \verb=<adolc/adouble.h>=. |
---|
685 | \end{enumerate} |
---|
686 | Typically, the first compilation will detect several type conflicts |
---|
687 | -- usually attempts to convert from active to passive |
---|
688 | variables or to perform standard I/O of active variables. |
---|
689 | Since all standard |
---|
690 | C programs can be activated by a mechanical application of the |
---|
691 | procedure above, the following section is of importance |
---|
692 | only to advanced users. |
---|
693 | % |
---|
694 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
695 | \section{Numbering the Tapes and Controlling the Buffer} |
---|
696 | \label{tape} |
---|
697 | % |
---|
698 | The trace generated by the execution of an active section may stay |
---|
699 | within a triplet of internal arrays or it may be written out |
---|
700 | to three corresponding files. We will refer to these triplets as the |
---|
701 | tape array or tape file, in general tape, which may subsequently be |
---|
702 | used to evaluate the |
---|
703 | underlying function and its derivatives at the original point or at |
---|
704 | alternative arguments. If the active section involves user-defined |
---|
705 | quadratures it must be executed and |
---|
706 | re-taped at each new argument. Similarly, if conditions on |
---|
707 | {\sf adouble} values lead to a different program branch being taken at |
---|
708 | a new argument the evaluation process also needs to be re-taped at the |
---|
709 | new point. Otherwise, direct evaluation from |
---|
710 | the tape by the routine {\sf function} (\autoref{optdrivers}) is |
---|
711 | likely to be |
---|
712 | faster. The use of quadratures and the results of all comparisons on |
---|
713 | {\sf adouble}s are recorded on the tape so that {\sf function} and other |
---|
714 | forward routines stop and return appropriate flags if their use without |
---|
715 | prior re-taping is unsafe. To avoid any re-taping certain types of |
---|
716 | branches can be recorded on the tape through |
---|
717 | the use of conditional assignments |
---|
718 | described before in \autoref{condassign}. |
---|
719 | |
---|
720 | Several tapes may be generated and kept simultaneously. |
---|
721 | A tape array is used as a triplet of buffers or a tape file is generated if |
---|
722 | the length of any of the buffers exceeds the maximal array lengths of |
---|
723 | {\sf OBUFSIZE}, {\sf VBUFSIZE} or {\sf LBUFSIZE}. These parameters are |
---|
724 | defined in the header file \verb=<adolc/usrparms.h>= |
---|
725 | and may be adjusted by the user in the header file before compiling |
---|
726 | the ADOL-C library, or on runtime using a file named \verb=.adolcrc=. |
---|
727 | The filesystem folder, where the tapes files may be written to disk, |
---|
728 | can be changed by changing the definition of {\sf TAPE\_DIR} in |
---|
729 | the header file \verb=<adolc/dvlparms.h>= before |
---|
730 | compiling the ADOL-C library, or on runtime by defining {\sf |
---|
731 | TAPE\_DIR} in the \verb=.adolcrc= file. By default this is defined |
---|
732 | to be the present working directory (\verb=.=). |
---|
733 | |
---|
734 | For simple usage, {\sf trace\_on} may be called with only the tape |
---|
735 | {\sf tag} as argument, and {\sf trace\_off} may be called |
---|
736 | without argument. The optional integer argument {\sf keep} of |
---|
737 | {\sf trace\_on} determines whether the numerical values of all |
---|
738 | active variables are recorded in a buffered temporary array or file |
---|
739 | called the taylor stack. |
---|
740 | This option takes effect if |
---|
741 | {\sf keep} = 1 and prepares the scene for an immediately following |
---|
742 | gradient evaluation by a call to a routine implementing the reverse mode |
---|
743 | as described in the \autoref{forw_rev_ad} and \autoref{forw_rev}. A |
---|
744 | file is used instead of an array if the size exceeds the maximal array |
---|
745 | length of {\sf TBUFSIZE} defined in \verb=<adolc/usrparms.h>= and may |
---|
746 | be adjusted in the same way like the other buffer sizes mentioned above. |
---|
747 | Alternatively, gradients may be evaluated by a call |
---|
748 | to {\sf gradient}, which includes a preparatory forward sweep |
---|
749 | for the creation of the temporary file. If omitted, the argument |
---|
750 | {\sf keep} defaults to 0, so that no temporary |
---|
751 | taylor stack file is generated. |
---|
752 | |
---|
753 | By setting the optional integer argument {\sf file} of |
---|
754 | {\sf trace\_off} to 1, the user may force a numbered tape |
---|
755 | file to be written even if the tape array (buffer) does not overflow. |
---|
756 | If the argument {\sf file} is omitted, it |
---|
757 | defaults to 0, so that the tape array is written onto a tape file only |
---|
758 | if the length of any of the buffers exceeds {\sf [OLVT]BUFSIZE} elements. |
---|
759 | |
---|
760 | After the execution of an active section, if a tape file was generated, i.e., |
---|
761 | if the length of some buffer exceeded {\sf [OLVT]BUFSIZE} elements or if the |
---|
762 | argument {\sf file} of {\sf trace\_off} was set to 1, the files will be |
---|
763 | saved in the directory defined as {\sf ADOLC\_TAPE\_DIR} (by default |
---|
764 | the current working directory) under filenames formed by |
---|
765 | the strings {\sf ADOLC\_OPERATIONS\_NAME}, {\sf |
---|
766 | ADOLC\_LOCATIONS\_NAME}, {\sf ADOLC\_VALUES\_NAME} and {\sf |
---|
767 | ADOLC\_TAYLORS\_NAME} defined in |
---|
768 | the header file \verb=<adolc/dvlparms.h>= appended with the number |
---|
769 | given as the {\sf tag} argument to {\sf trace\_on} and have the |
---|
770 | extension {\sf .tap}. |
---|
771 | |
---|
772 | Later, all problem-independent routines |
---|
773 | like {\sf gradient}, {\sf jacobian}, {\sf forward}, {\sf reverse}, and others |
---|
774 | expect as first argument a {\sf tag} to determine |
---|
775 | the tape on which their respective computational task is to be performed. |
---|
776 | By calling {\sf trace\_on} with different tape {\sf tag}s, one can create |
---|
777 | several tapes for various function evaluations and subsequently perform |
---|
778 | function and derivative evaluations on one or more of them. |
---|
779 | |
---|
780 | For example, suppose one wishes to calculate for two smooth functions |
---|
781 | $f_1(x)$ and $f_2(x)$ |
---|
782 | \[ |
---|
783 | f(x) = \max \{f_1(x) ,f_2(x)\},\qquad \nabla f(x), |
---|
784 | \] |
---|
785 | and possibly higher derivatives where the two functions do not tie. |
---|
786 | Provided $f_1$ and $f_2$ are evaluated in two separate active sections, |
---|
787 | one can generate two different tapes by calling {\sf trace\_on} with |
---|
788 | {\sf tag} = 1 and {\sf tag} = 2 at the beginning of the respective active |
---|
789 | sections. |
---|
790 | Subsequently, one can decide whether $f(x)=f_1(x)$ or $f(x)=f_2(x)$ at the |
---|
791 | current argument and then evaluate the gradient $\nabla f(x)$ by calling |
---|
792 | {\sf gradient} with the appropriate argument value {\sf tag} = 1 or |
---|
793 | {\sf tag} = 2. |
---|
794 | % |
---|
795 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
796 | \subsection{Examining the Tape and Predicting Storage Requirements } |
---|
797 | \label{examiningTape} |
---|
798 | % |
---|
799 | At any point in the program, one may call the routine |
---|
800 | \begin{center} |
---|
801 | {\sf void tapestats(unsigned short tag, int* counts)} |
---|
802 | \end{center} |
---|
803 | with {\sf counts} beeing an array of at least eleven integers. |
---|
804 | The first argument {\sf tag} specifies the particular tape of |
---|
805 | interest. The components of {\sf counts} represent |
---|
806 | \[ |
---|
807 | \begin{tabular}{ll} |
---|
808 | {\sf counts[0]}: & the number of independents, i.e.~calls to \boldmath $\ll=$ \unboldmath, \\ |
---|
809 | {\sf counts[1]}: & the number of dependents, i.e.~calls to \boldmath $\gg=$ \unboldmath,\\ |
---|
810 | {\sf counts[2]}: & the maximal number of live active variables,\\ |
---|
811 | {\sf counts[3]}: & the size of taylor stack (number of overwrites),\\ |
---|
812 | {\sf counts[4]}: & the buffer size (a multiple of eight), |
---|
813 | \end{tabular} |
---|
814 | \] |
---|
815 | \[ |
---|
816 | \begin{tabular}{ll} |
---|
817 | {\sf counts[5]}: & the total number of operations recorded,\\ |
---|
818 | {\sf counts[6-13]}: & other internal information about the tape. |
---|
819 | \end{tabular} |
---|
820 | \] |
---|
821 | The values {\sf maxlive} = {\sf counts[2]} and {\sf tssize} = {\sf counts[3]} |
---|
822 | determine the temporary |
---|
823 | storage requirements during calls to the routines |
---|
824 | implementing the forward and the reverse mode. |
---|
825 | For a certain degree {\sf deg} $\geq$ 0, the scalar version of the |
---|
826 | forward mode involves apart from the tape buffers an array of |
---|
827 | $(${\sf deg}$+1)*${\sf maxlive} {\sf double}s in |
---|
828 | core and, in addition, a sequential data set called the value stack |
---|
829 | of {\sf tssize}$*${\sf keep} {\sf revreal}s if called with the |
---|
830 | option {\sf keep} $>$ 0. Here |
---|
831 | the type {\sf revreal} is defined as {\sf double} or {\sf float} in |
---|
832 | the header file \verb=<adolc/usrparms.h>=. The latter choice halves the storage |
---|
833 | requirement for the sequential data set, which stays in core if |
---|
834 | its length is less than {\sf TBUFSIZE} bytes and is otherwise written |
---|
835 | out to a temporary file. The parameter {\sf TBUFSIZE} is defined in the header file \verb=<adolc/usrparms.h>=. |
---|
836 | The drawback of the economical |
---|
837 | {\sf revreal} = {\sf float} choice is that subsequent calls to reverse mode implementations |
---|
838 | yield gradients and other adjoint vectors only in single-precision |
---|
839 | accuracy. This may be acceptable if the adjoint vectors |
---|
840 | represent rows of a Jacobian that is used for the calculation of |
---|
841 | Newton steps. In its scalar version, the reverse mode implementation involves |
---|
842 | the same number of {\sf double}s and twice as many {\sf revreal}s as the |
---|
843 | forward mode implementation. |
---|
844 | The storage requirements of the vector versions of the forward mode and |
---|
845 | reverse mode implementation are equal to that of the scalar versions multiplied by |
---|
846 | the vector length. |
---|
847 | % |
---|
848 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
849 | \subsection{Customizing ADOL-C} |
---|
850 | \label{Customizing} |
---|
851 | % |
---|
852 | Based on the information provided by the routine {\sf tapestats}, the user may alter the |
---|
853 | following types and constant dimensions in the header file \verb=<adolc/usrparms.h>= |
---|
854 | to suit his problem and environment. |
---|
855 | |
---|
856 | \begin{description} |
---|
857 | \item[{\sf OBUFSIZE}, {\sf LBUFSIZE}, {\sf VBUFSIZE}{\rm :}] These integer determines the length of |
---|
858 | in\-ter\-nal buf\-fers (default: 65$\,$536). If the buffers are large enough to accommodate all |
---|
859 | required data, any file access is avoided unless {\sf trace\_off} |
---|
860 | is called with a positive argument. This desirable situation can |
---|
861 | be achieved for many problem functions with an execution trace of moderate |
---|
862 | size. Primarily these values occur as an argument |
---|
863 | to {\sf malloc}, so that setting it unnecessarily large may have no |
---|
864 | ill effects, unless the operating system prohibits or penalizes large |
---|
865 | array allocations. |
---|
866 | |
---|
867 | \item[{\sf TBUFSIZE}{\rm :}] This integer determines the length of the |
---|
868 | in\-ter\-nal buf\-fer for a taylor stack (default: 65$\,$536). |
---|
869 | |
---|
870 | \item[{\sf TBUFNUM}{\rm :}] This integer determines the maximal number of taylor stacks (default: 32). |
---|
871 | |
---|
872 | \item[{\sf locint}{\rm :}] The range of the integer type |
---|
873 | {\sf locint} determines how many {\sf adouble}s can be simultaneously |
---|
874 | alive (default: {\sf unsigned int}). In extreme cases when there are more than $2^{32}$ {\sf adouble}s |
---|
875 | alive at any one time, the type {\sf locint} must be changed to |
---|
876 | {\sf unsigned long}. |
---|
877 | |
---|
878 | \item[{\sf revreal}{\rm :}] The choice of this floating-point type |
---|
879 | trades accuracy with storage for reverse sweeps (default: {\sf double}). While functions |
---|
880 | and their derivatives are always evaluated in double precision |
---|
881 | during forward sweeps, gradients and other adjoint vectors are obtained |
---|
882 | with the precision determined by the type {\sf revreal}. The less |
---|
883 | accurate choice {\sf revreal} = {\sf float} nearly halves the |
---|
884 | storage requirement during reverse sweeps. |
---|
885 | |
---|
886 | \item[{\sf fint}{\rm :}] The integer data type used by Fortran callable versions of functions. |
---|
887 | |
---|
888 | \item[{\sf fdouble}{\rm :}] The floating point data type used by Fortran callable versions of functions. |
---|
889 | |
---|
890 | \item[{\sf inf\_num}{\rm :}] This together with {\sf inf\_den} |
---|
891 | sets the ``vertical'' slope {\sf InfVal} = {\sf inf\_num/inf\_den} |
---|
892 | of special functions at the boundaries of their domains (default: {\sf inf\_num} = 1.0). On IEEE machines |
---|
893 | the default setting produces the standard {\sf Inf}. On non-IEEE machines |
---|
894 | change these values to produce a small {\sf InfVal} value and compare |
---|
895 | the results of two forward sweeps with different {\sf InfVal} settings |
---|
896 | to detect a ``vertical'' slope. |
---|
897 | |
---|
898 | \item[{\sf inf\_den}{\rm :}] See {\sf inf\_num} (default: 0.0). |
---|
899 | |
---|
900 | \item[{\sf non\_num}{\rm :}] This together with {\sf non\_den} |
---|
901 | sets the mathematically |
---|
902 | undefined derivative value {\sf NoNum} = {\sf non\_num/non\_den} |
---|
903 | of special functions at the boundaries of their domains (default: {\sf non\_num} = 0.0). On IEEE machines |
---|
904 | the default setting produces the standard {\sf NaN}. On non-IEEE machines |
---|
905 | change these values to produce a small {\sf NoNum} value and compare |
---|
906 | the results of two forward sweeps with different {\sf NoNum} settings |
---|
907 | to detect the occurrence of undefined derivative values. |
---|
908 | |
---|
909 | \item[{\sf non\_den}{\rm :}] See {\sf non\_num} (default: 0.0). |
---|
910 | |
---|
911 | \item[{\sf ADOLC\_EPS}{\rm :}] For testing on small numbers to avoid overflows (default: 10E-20). |
---|
912 | |
---|
913 | \item[{\sf ATRIG\_ERF}{\rm :}] By removing the comment signs |
---|
914 | the overloaded versions of the inverse hyperbolic functions and |
---|
915 | the error function are enabled (default: undefined). |
---|
916 | |
---|
917 | \item[{\sf DIAG\_OUT}{\rm :}] File identifier used as standard output for ADOL-C diagnostics (default: stdout). |
---|
918 | |
---|
919 | \item[{\sf ADOLC\_USE\_CALLOC}{\rm :}] Selects the memory allocation routine |
---|
920 | used by ADOL-C. {\sf Malloc} will be used if this variable is |
---|
921 | undefined. {\sf ADOLC\_USE\_CALLOC} is defined by default to avoid incorrect |
---|
922 | result caused by uninitialized memory. |
---|
923 | \end{description} |
---|
924 | % |
---|
925 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
926 | \subsection{Warnings and Suggestions for Improved Efficiency} |
---|
927 | \label{WarSug} |
---|
928 | % |
---|
929 | Since the type {\sf adouble} has a nontrivial constructor, |
---|
930 | the mere declaration of large {\sf adouble} arrays may take up |
---|
931 | considerable run time. The user should be warned against |
---|
932 | the usual Fortran practice of declaring fixed-size arrays |
---|
933 | that can accommodate the largest possible case of an evaluation program |
---|
934 | with variable dimensions. If such programs are converted to or written |
---|
935 | in C, the overloading in combination with ADOL-C will lead to very |
---|
936 | large run time increases for comparatively small values of the |
---|
937 | problem dimension, because the actual computation is completely |
---|
938 | dominated by the construction of the large {\sf adouble} arrays. |
---|
939 | The user is advised to |
---|
940 | create dynamic arrays of |
---|
941 | {\sf adouble}s by using the C++ operator {\sf new} and to destroy them |
---|
942 | using {\sf delete}. For storage efficiency it is desirable that |
---|
943 | dynamic objects are created and destroyed in a last-in-first-out |
---|
944 | fashion. |
---|
945 | |
---|
946 | Whenever an {\sf adouble} is declared, the constructor for the type |
---|
947 | {\sf adouble} assigns it a nominal address, which we will refer to as |
---|
948 | its {\em location}. The location is of the type {\sf locint} defined |
---|
949 | in the header file \verb=<adolc/usrparms.h>=. Active vectors occupy |
---|
950 | a range of contiguous locations. As long as the program execution |
---|
951 | never involves more than 65$\,$536 active variables, the type {\sf locint} |
---|
952 | may be defined as {\sf unsigned short}. Otherwise, the range may be |
---|
953 | extended by defining {\sf locint} as {\sf (unsigned) int} or |
---|
954 | {\sf (unsigned) long}, which may nearly double |
---|
955 | the overall mass storage requirement. Sometimes one can avoid exceeding |
---|
956 | the accessible range of {\sf unsigned short}s by using more local variables and deleting |
---|
957 | {\sf adouble}s created by the new operator in a |
---|
958 | last-in-first-out |
---|
959 | fashion. When memory for {\sf adouble}s is requested through a call to |
---|
960 | {\sf malloc()} or other related C memory-allocating |
---|
961 | functions, the storage for these {\sf adouble}s is allocated; however, the |
---|
962 | C++ {\sf adouble} constructor is never called. The newly defined |
---|
963 | {\sf adouble}s are never assigned a location and are not counted in |
---|
964 | the stack of live variables. Thus, any results depending upon these |
---|
965 | pseudo-{\sf adouble}s will be incorrect. For these reasons {\bf DO NOT use |
---|
966 | malloc() and related C memory-allocating |
---|
967 | functions when declaring adoubles (see the following paragraph).} |
---|
968 | % |
---|
969 | % XXX: Vector and matrix class have to be reimplemented !!! |
---|
970 | % |
---|
971 | %The same point applies, of course, |
---|
972 | % for active vectors. |
---|
973 | |
---|
974 | When an {\sf adouble} |
---|
975 | % |
---|
976 | % XXX: Vector and matrix class have to be reimplemented !!! |
---|
977 | % |
---|
978 | % or {\bf adoublev} |
---|
979 | goes out of |
---|
980 | scope or is explicitly deleted, the destructor notices that its |
---|
981 | location(s) may be |
---|
982 | freed for subsequent (nominal) reallocation. In general, this is not done |
---|
983 | immediately but is delayed until the locations to be deallocated form a |
---|
984 | contiguous tail of all locations currently being used. |
---|
985 | |
---|
986 | As a consequence of this allocation scheme, the currently |
---|
987 | alive {\sf adouble} locations always form a contiguous range of integers |
---|
988 | that grows and shrinks like a stack. Newly declared {\sf adouble}s are |
---|
989 | placed on the top so that vectors of {\sf adouble}s obtain a contiguous |
---|
990 | range of locations. While the C++ compiler can be expected to construct |
---|
991 | and destruct automatic variables in a last-in-first-out fashion, the |
---|
992 | user may upset this desirable pattern by deleting free-store {\sf adouble}s |
---|
993 | too early or too late. Then the {\sf adouble} stack may grow |
---|
994 | unnecessarily, but the numerical results will still be |
---|
995 | correct, unless an exception occurs because the range of {\sf locint} |
---|
996 | is exceeded. In general, free-store {\sf adouble}s |
---|
997 | % |
---|
998 | % XXX: Vector and matrix class have to be reimplemented !!! |
---|
999 | % |
---|
1000 | %and {\bf adoublev}s |
---|
1001 | should be deleted in a last-in-first-out fashion toward the end of |
---|
1002 | the program block in which they were created. |
---|
1003 | When this pattern is maintained, the maximum number of |
---|
1004 | {\sf adouble}s alive and, as a consequence, the |
---|
1005 | randomly accessed storage space |
---|
1006 | of the derivative evaluation routines is bounded by a |
---|
1007 | small multiple of the memory used in the relevant section of the |
---|
1008 | original program. Failure to delete dynamically allocated {\sf adouble}s |
---|
1009 | may cause that the maximal number of {\sf adouble}s alive at one time will be exceeded |
---|
1010 | if the same active section is called repeatedly. The same effect |
---|
1011 | occurs if static {\sf adouble}s are used. |
---|
1012 | |
---|
1013 | To avoid the storage and manipulation of structurally |
---|
1014 | trivial derivative values, one should pay careful attention to |
---|
1015 | the naming of variables. Ideally, the intermediate |
---|
1016 | values generated during the evaluation of a vector function |
---|
1017 | should be assigned to program variables that are |
---|
1018 | consistently either active or passive, in that all their values |
---|
1019 | either are or are not dependent on the independent variables |
---|
1020 | in a nontrivial way. For example, this rule is violated if a temporary |
---|
1021 | variable is successively used to accumulate inner products involving |
---|
1022 | first only passive and later active arrays. Then the first inner |
---|
1023 | product and all its successors in the data dependency graph become |
---|
1024 | artificially active and the derivative evaluation routines |
---|
1025 | described later will waste |
---|
1026 | time allocating and propagating |
---|
1027 | trivial or useless derivatives. Sometimes even values that do |
---|
1028 | depend on the independent variables may be of only transitory |
---|
1029 | importance and may not affect the dependent variables. For example, |
---|
1030 | this is true for multipliers that are used to scale linear |
---|
1031 | equations, but whose values do not influence the dependent |
---|
1032 | variables in a mathematical sense. Such dead-end variables |
---|
1033 | can be deactivated by the use of the {\sf value} function, which |
---|
1034 | converts {\sf adouble}s to {\sf double}s. The deleterious effects |
---|
1035 | of unnecessary activity are partly alleviated by run time |
---|
1036 | activity flags in the derivative routine |
---|
1037 | {\sf hov\_reverse} presented in \autoref{forw_rev_ad}. |
---|
1038 | % |
---|
1039 | % |
---|
1040 | % |
---|
1041 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1042 | \section{Easy-To-Use Drivers} |
---|
1043 | \label{drivers} |
---|
1044 | % |
---|
1045 | For the convenience of the user, ADOL-C provides several |
---|
1046 | easy-to-use drivers that compute the most frequently required |
---|
1047 | derivative objects. Throughout, we assume that after the execution of an |
---|
1048 | active section, the corresponding tape with the identifier {\sf tag} |
---|
1049 | contains a detailed record of the computational process by which the |
---|
1050 | final values $y$ of the dependent variables were obtained from the |
---|
1051 | values $x$ of the independent variables. We will denote this functional |
---|
1052 | relation between the input variables $x$ and the output variables $y$ by |
---|
1053 | \[ |
---|
1054 | F : \R^n \mapsto \R^m, \qquad x \rightarrow F(x) \equiv y. |
---|
1055 | \] |
---|
1056 | The return value of all drivers presented in this section |
---|
1057 | indicate the validity of the tape as explained in \autoref{reuse_tape}. |
---|
1058 | The presented drivers are all C functions and therefore can be used within |
---|
1059 | C and C++ programs. Some Fortran-callable companions can be found |
---|
1060 | in the appropriate header files. |
---|
1061 | % |
---|
1062 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1063 | \subsection{Drivers for Optimization and Nonlinear Equations} |
---|
1064 | % |
---|
1065 | \label{optdrivers} |
---|
1066 | % |
---|
1067 | The drivers provided for solving optimization problems and nonlinear |
---|
1068 | equations are prototyped in the header file \verb=<adolc/drivers/drivers.h>=, |
---|
1069 | which is included automatically by the global header file \verb=<adolc/adolc.h>= |
---|
1070 | (see \autoref{ssec:DesIH}). |
---|
1071 | |
---|
1072 | The routine {\sf function} allows to evaluate the desired function from |
---|
1073 | the tape instead of executing the corresponding source code: |
---|
1074 | % |
---|
1075 | \begin{tabbing} |
---|
1076 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1077 | \>{\sf int function(tag,m,n,x,y)}\\ |
---|
1078 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1079 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1080 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1081 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1082 | \>{\sf double y[m];} \> // dependent vector $y=F(x)$ |
---|
1083 | \end{tabbing} |
---|
1084 | % |
---|
1085 | If the original evaluation program is available this double version |
---|
1086 | should be used to compute the function value in order to avoid the |
---|
1087 | interpretative overhead. |
---|
1088 | |
---|
1089 | For the calculation of whole derivative vectors and matrices up to order |
---|
1090 | 2 there are the following procedures: |
---|
1091 | % |
---|
1092 | \begin{tabbing} |
---|
1093 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1094 | \>{\sf int gradient(tag,n,x,g)}\\ |
---|
1095 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1096 | \>{\sf int n;} \> // number of independent variables $n$ and $m=1$\\ |
---|
1097 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1098 | \>{\sf double g[n];} \> // resulting gradient $\nabla F(x)$ |
---|
1099 | \end{tabbing} |
---|
1100 | % |
---|
1101 | \begin{tabbing} |
---|
1102 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1103 | \>{\sf int jacobian(tag,m,n,x,J)}\\ |
---|
1104 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1105 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1106 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1107 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1108 | \>{\sf double J[m][n];} \> // resulting Jacobian $F^\prime (x)$ |
---|
1109 | \end{tabbing} |
---|
1110 | % |
---|
1111 | \begin{tabbing} |
---|
1112 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1113 | \>{\sf int hessian(tag,n,x,H)}\\ |
---|
1114 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1115 | \>{\sf int n;} \> // number of independent variables $n$ and $m=1$\\ |
---|
1116 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1117 | \>{\sf double H[n][n];} \> // resulting Hessian matrix $\nabla^2F(x)$ |
---|
1118 | \end{tabbing} |
---|
1119 | % |
---|
1120 | The driver routine {\sf hessian} computes only the lower half of |
---|
1121 | $\nabla^2f(x_0)$ so that all values {\sf H[i][j]} with $j>i$ |
---|
1122 | of {\sf H} allocated as a square array remain untouched during the call |
---|
1123 | of {\sf hessian}. Hence only $i+1$ {\sf double}s need to be |
---|
1124 | allocated starting at the position {\sf H[i]}. |
---|
1125 | |
---|
1126 | To use the full capability of automatic differentiation when the |
---|
1127 | product of derivatives with certain weight vectors or directions are needed, ADOL-C offers |
---|
1128 | the following four drivers: |
---|
1129 | % |
---|
1130 | \begin{tabbing} |
---|
1131 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1132 | \>{\sf int vec\_jac(tag,m,n,repeat,x,u,z)}\\ |
---|
1133 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1134 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1135 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1136 | \>{\sf int repeat;} \> // indicate repeated call at same argument\\ |
---|
1137 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1138 | \>{\sf double u[m];} \> // range weight vector $u$ \\ |
---|
1139 | \>{\sf double z[n];} \> // result $z = u^TF^\prime (x)$ |
---|
1140 | \end{tabbing} |
---|
1141 | If a nonzero value of the parameter {\sf repeat} indicates that the |
---|
1142 | routine {\sf vec\_jac} has been called at the same argument immediately |
---|
1143 | before, the internal forward mode evaluation will be skipped and only |
---|
1144 | reverse mode evaluation with the corresponding arguments is executed |
---|
1145 | resulting in a reduced computational complexity of the function {\sf vec\_jac}. |
---|
1146 | % |
---|
1147 | \begin{tabbing} |
---|
1148 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1149 | \>{\sf int jac\_vec(tag,m,n,x,v,z)}\\ |
---|
1150 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1151 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1152 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1153 | \>{\sf double x[n];} \> // independent vector $x$\\ |
---|
1154 | \>{\sf double v[n];} \> // tangent vector $v$\\ |
---|
1155 | \>{\sf double z[m];} \> // result $z = F^\prime (x)v$ |
---|
1156 | \end{tabbing} |
---|
1157 | % |
---|
1158 | \begin{tabbing} |
---|
1159 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1160 | \>{\sf int hess\_vec(tag,n,x,v,z)}\\ |
---|
1161 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1162 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1163 | \>{\sf double x[n];} \> // independent vector $x$\\ |
---|
1164 | \>{\sf double v[n];} \> // tangent vector $v$\\ |
---|
1165 | \>{\sf double z[n];} \> // result $z = \nabla^2F(x) v$ |
---|
1166 | \end{tabbing} |
---|
1167 | % |
---|
1168 | \begin{tabbing} |
---|
1169 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1170 | \>{\sf int hess\_mat(tag,n,p,x,V,Z)}\\ |
---|
1171 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1172 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1173 | \>{\sf int p;} \> // number of columns in $V$\\ |
---|
1174 | \>{\sf double x[n];} \> // independent vector $x$\\ |
---|
1175 | \>{\sf double V[n][p];} \> // tangent matrix $V$\\ |
---|
1176 | \>{\sf double Z[n][p];} \> // result $Z = \nabla^2F(x) V$ |
---|
1177 | \end{tabbing} |
---|
1178 | % |
---|
1179 | \begin{tabbing} |
---|
1180 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1181 | \>{\sf int lagra\_hess\_vec(tag,m,n,x,v,u,h)}\\ |
---|
1182 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1183 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1184 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1185 | \>{\sf double x[n];} \> // independent vector $x$\\ |
---|
1186 | \>{\sf double v[n];} \> // tangent vector $v$\\ |
---|
1187 | \>{\sf double u[m];} \> // range weight vector $u$ \\ |
---|
1188 | \>{\sf double h[n];} \> // result $h = u^T\nabla^2F(x) v $ |
---|
1189 | \end{tabbing} |
---|
1190 | % |
---|
1191 | The next procedure allows the user to perform Newton steps only |
---|
1192 | having the corresponding tape at hand: |
---|
1193 | % |
---|
1194 | \begin{tabbing} |
---|
1195 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1196 | \>{\sf int jac\_solv(tag,n,x,b,mode)} \\ |
---|
1197 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1198 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1199 | \>{\sf double x[n];} \> // independent vector $x$ as\\ |
---|
1200 | \>{\sf double b[n];} \> // in: right-hand side b, out: result $w$ of |
---|
1201 | $F(x)w = b$\\ |
---|
1202 | \>{\sf int mode;} \> // option to choose different solvers |
---|
1203 | \end{tabbing} |
---|
1204 | % |
---|
1205 | On entry, parameter {\sf b} of the routine {\sf jac\_solv} |
---|
1206 | contains the right-hand side of the equation $F(x)w = b$ to be solved. On exit, |
---|
1207 | {\sf b} equals the solution $w$ of this equation. If {\sf mode} = 0 only |
---|
1208 | the Jacobian of the function |
---|
1209 | given by the tape labeled with {\sf tag} is provided internally. |
---|
1210 | The LU-factorization of this Jacobian is computed for {\sf mode} = 1. The |
---|
1211 | solution of the equation is calculated if {\sf mode} = 2. |
---|
1212 | Hence, it is possible to compute the |
---|
1213 | LU-factorization only once. Then the equation can be solved for several |
---|
1214 | right-hand sides $b$ without calculating the Jacobian and |
---|
1215 | its factorization again. |
---|
1216 | |
---|
1217 | If the original evaluation code of a function contains neither |
---|
1218 | quadratures nor branches, all drivers described above can be used to |
---|
1219 | evaluate derivatives at any argument in its domain. The same still |
---|
1220 | applies if there are no user defined quadratures and |
---|
1221 | all comparisons involving {\sf adouble}s have the same result as |
---|
1222 | during taping. If this assumption is falsely made all drivers |
---|
1223 | while internally calling the forward mode evaluation will return the value -1 or -2 |
---|
1224 | as already specified in \autoref{reuse_tape}. |
---|
1225 | % |
---|
1226 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1227 | \subsection{Drivers for Ordinary Differential Equations} |
---|
1228 | \label{odedrivers} |
---|
1229 | % |
---|
1230 | When $F$ is the right-hand side of an (autonomous) ordinary |
---|
1231 | differential equation |
---|
1232 | \[ |
---|
1233 | x^\prime(t) \; = \; F(x(t)) , |
---|
1234 | \] |
---|
1235 | we must have $m=n$. Along any solution path $x(t)$ its Taylor |
---|
1236 | coefficients $x_j$ at some time, e.g., $t=0$, must satisfy |
---|
1237 | the relation |
---|
1238 | \[ |
---|
1239 | x_{i+1} = \frac{1}{1+i} y_i. |
---|
1240 | \] |
---|
1241 | with the $y_j$ the Taylor coefficients of its derivative $y(t)=x^\prime(t)$, namely, |
---|
1242 | \[ |
---|
1243 | y(t) \; \equiv \; F(x(t)) \; : \; I\!\!R \;\mapsto \;I\!\!R^m |
---|
1244 | \] |
---|
1245 | defined by an autonomous right-hand side $F$ recorded on the tape. |
---|
1246 | Using this relation, one can generate the Taylor coefficients $x_i$, |
---|
1247 | $i \le deg$, |
---|
1248 | recursively from the current point $x_0$. This task is achieved by the |
---|
1249 | driver routine {\sf forode} defined as follows: |
---|
1250 | % |
---|
1251 | \begin{tabbing} |
---|
1252 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1253 | \>{\sf int forode(tag,n,tau,dol,deg,X)}\\ |
---|
1254 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1255 | \>{\sf int n;} \> // number of state variables $n$\\ |
---|
1256 | \>{\sf double tau;} \> // scaling parameter\\ |
---|
1257 | \>{\sf int dol;} \> // degree on previous call\\ |
---|
1258 | \>{\sf int deg;} \> // degree on current call\\ |
---|
1259 | \>{\sf double X[n][deg+1];} \> // Taylor coefficient vector $X$ |
---|
1260 | \end{tabbing} |
---|
1261 | % |
---|
1262 | If {\sf dol} is positive, it is assumed that {\sf forode} |
---|
1263 | has been called before at the same point so that all Taylor coefficient |
---|
1264 | vectors up to the {\sf dol}-th are already correct. |
---|
1265 | |
---|
1266 | Subsequently one may call the driver routine {\sf reverse} or corresponding |
---|
1267 | low level routines as explained in the \autoref{forw_rev} and |
---|
1268 | \autoref{forw_rev_ad}, respectively, to compute |
---|
1269 | the family of square matrices {\sf Z[n][n][deg]} defined by |
---|
1270 | \[ |
---|
1271 | Z_j \equiv U\/\frac{\partial y_j}{\partial x_0} \in{I\!\!R}^{q \times n} , |
---|
1272 | \] |
---|
1273 | with {\sf double** U}$=I_n$ the identity matrix of order {\sf n}. |
---|
1274 | |
---|
1275 | For the numerical solutions of ordinary differential equations, |
---|
1276 | one may also wish to calculate the Jacobians |
---|
1277 | \begin{equation} |
---|
1278 | \label{eq:bees} |
---|
1279 | B_j \; \equiv \; \frac{\mbox{d}x_{j+1}}{\mbox{d} x_0}\;\in\;{I\!\!R}^{n \times n}\, , |
---|
1280 | \end{equation} |
---|
1281 | which exist provided $F$ is sufficiently smooth. These matrices can |
---|
1282 | be obtained from the partial derivatives $\partial y_i/\partial x_0$ |
---|
1283 | by an appropriate version of the chain rule. |
---|
1284 | To compute the total derivatives $B = (B_j)_{0\leq j <d}$ |
---|
1285 | defined in \eqref{eq:bees}, one has to evaluate $\frac{1}{2}d(d-1)$ |
---|
1286 | matrix-matrix products. This can be done by a call of the routine {\sf accode} after the |
---|
1287 | corresponding evaluation of the {\sf hov\_reverse} function. The interface of |
---|
1288 | {\sf accode} is defined as follows: |
---|
1289 | % |
---|
1290 | \begin{tabbing} |
---|
1291 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1292 | \>{\sf int accode(n,tau,deg,Z,B,nz)}\\ |
---|
1293 | \>{\sf int n;} \> // number of state variables $n$ \\ |
---|
1294 | \>{\sf double tau;} \> // scaling parameter\\ |
---|
1295 | \>{\sf int deg;} \> // degree on current call\\ |
---|
1296 | \>{\sf double Z[n][n][deg];} \> // partials of coefficient vectors\\ |
---|
1297 | \>{\sf double B[n][n][deg];} \> // result $B$ as defined in \eqref{eq:bees}\\ |
---|
1298 | \>{\sf short nz[n][n];} \> // optional nonzero pattern |
---|
1299 | \end{tabbing} |
---|
1300 | % |
---|
1301 | Sparsity information can be exploited by {\sf accode} using the array {\sf |
---|
1302 | nz}. For this purpose, {\sf nz} has to be set by a call of the routine {\sf |
---|
1303 | reverse} or the corresponding basic routines as explained below in |
---|
1304 | \autoref{forw_rev_ad} and \autoref{forw_rev}, respectively. The |
---|
1305 | non-positive entries of {\sf nz} are then changed by {\sf accode} so that upon |
---|
1306 | return |
---|
1307 | \[ |
---|
1308 | \mbox{{\sf B[i][j][k]}} \; \equiv \; 0 \quad {\rm if} \quad \mbox{\sf k} \leq \mbox{\sf $-$nz[i][j]}\; . |
---|
1309 | \] |
---|
1310 | In other words, the matrices $B_k$ = {\sf B[ ][ ][k]} have a |
---|
1311 | sparsity pattern that fills in as $k$ grows. Note, that there need to be no |
---|
1312 | loss in computational efficiency if a time-dependent ordinary differential equation |
---|
1313 | is rewritten in autonomous form. |
---|
1314 | |
---|
1315 | The prototype of the ODE-drivers {\sf forode} and {\sf accode} is contained in the header file |
---|
1316 | \verb=<adolc/drivers/odedrivers.h>=. The global header file |
---|
1317 | \verb=<adolc/adolc.h>= |
---|
1318 | includes this file automatically, see \autoref{ssec:DesIH}. |
---|
1319 | |
---|
1320 | An example program using the procedures {\sf forode} and {\sf accode} together |
---|
1321 | with more detailed information about the coding can be found in |
---|
1322 | \autoref{exam:ode}. The corresponding source code |
---|
1323 | \verb=odexam.cpp= is contained in the subdirectory |
---|
1324 | \verb=examples=. |
---|
1325 | % |
---|
1326 | % |
---|
1327 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1328 | \subsection{Drivers for Sparse Jacobians and Sparse Hessians} |
---|
1329 | \label{sparse} |
---|
1330 | % |
---|
1331 | Quite often, the Jacobians and Hessians that have to be computed are sparse |
---|
1332 | matrices. Therefore, ADOL-C provides additionally drivers that |
---|
1333 | allow the exploitation of sparsity. The exploitation of sparsity is |
---|
1334 | frequently based on {\em graph coloring} methods, discussed |
---|
1335 | for example in \cite{GeMaPo05} and \cite{GeTaMaPo07}. The sparse drivers of ADOL-C presented in this section |
---|
1336 | rely on the the coloring package ColPack developed by the authors of \cite{GeMaPo05} and \cite{GeTaMaPo07}. |
---|
1337 | ColPack is not directly incorporated in ADOL-C, and therefore needs to be installed |
---|
1338 | separately to use the sparse drivers described here. ColPack is available for download at |
---|
1339 | \verb=http://www.cscapes.org/coloringpage/software.htm=. More information about the required |
---|
1340 | installation of ColPack is given in \autoref{install}. |
---|
1341 | % |
---|
1342 | \subsubsection*{Sparse Jacobians and Sparse Hessians} |
---|
1343 | % |
---|
1344 | To compute the entries of sparse Jacobians and sparse Hessians, |
---|
1345 | respectively, in coordinate format one may use the drivers: |
---|
1346 | \begin{tabbing} |
---|
1347 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1348 | \>{\sf int sparse\_jac(tag,m,n,repeat,x,\&nnz,\&rind,\&cind,\&values,\&options)}\\ |
---|
1349 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1350 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1351 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1352 | \>{\sf int repeat;} \> // indicate repeated call at same argument\\ |
---|
1353 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1354 | \>{\sf int nnz;} \> // number of nonzeros \\ |
---|
1355 | \>{\sf unsigned int rind[nnz];}\> // row index\\ |
---|
1356 | \>{\sf unsigned int cind[nnz];}\> // column index\\ |
---|
1357 | \>{\sf double values[nnz];} \> // non-zero values\\ |
---|
1358 | \>{\sf int options[4];} \> // array of control parameters\\ |
---|
1359 | \end{tabbing} |
---|
1360 | % |
---|
1361 | \begin{tabbing} |
---|
1362 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1363 | \>{\sf int sparse\_hess(tag,n,repeat,x,\&nnz,\&rind,\&cind,\&values,\&options)}\\ |
---|
1364 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1365 | \>{\sf int n;} \> // number of independent variables $n$ and $m=1$\\ |
---|
1366 | \>{\sf int repeat;} \> // indicate repeated call at same argument\\ |
---|
1367 | \>{\sf double x[n];} \> // independent vector $x$ \\ |
---|
1368 | \>{\sf int nnz;} \> // number of nonzeros \\ |
---|
1369 | \>{\sf unsigned int rind[nnz];}\> // row indices\\ |
---|
1370 | \>{\sf unsigned int cind[nnz];}\> // column indices\\ |
---|
1371 | \>{\sf double values[nnz];} \> // non-zero values \\ |
---|
1372 | \>{\sf int options[2];} \> // array of control parameters\\ |
---|
1373 | \end{tabbing} |
---|
1374 | % |
---|
1375 | Once more, the input variables are the identifier for the internal |
---|
1376 | representation {\sf tag}, if required the number of dependents {\sf m}, |
---|
1377 | and the number of independents {\sf n} for a consistency check. |
---|
1378 | Furthermore, the flag {\sf repeat=0} indicates that the functions are called |
---|
1379 | at a point with a new sparsity structure, whereas {\sf repeat=1} results in |
---|
1380 | the re-usage of the sparsity pattern from the previous call. |
---|
1381 | The current values of the independents are given by the array {\sf x}. |
---|
1382 | The input/output |
---|
1383 | variable {\sf nnz} stores the number of the nonzero entries. |
---|
1384 | Therefore, {\sf nnz} denotes also the length of the arrays {\sf r\_ind} storing |
---|
1385 | the row indices, {\sf c\_ind} storing the column indices, and |
---|
1386 | {\sf values} storing the values of the nonzero entries. |
---|
1387 | If {\sf sparse\_jac} and {\sf sparse\_hess} are called with {\sf repeat=0}, |
---|
1388 | the functions determine the number of nonzeros for the sparsity pattern |
---|
1389 | defined by the value of {\sf x}, allocate appropriate arrays {\sf r\_ind}, |
---|
1390 | {\sf c\_ind}, and {\sf values} and store the desired information in these |
---|
1391 | arrays. |
---|
1392 | During the next function call with {\sf repeat=1} the allocated memory |
---|
1393 | is reused such that only the values of the arrays are changed. |
---|
1394 | Before calling {\sf sparse\_jac} or {\sf sparse\_hess} once more with {\sf |
---|
1395 | repeat=0} the user is responsible for the deallocation of the array |
---|
1396 | {\sf r\_ind}, {\sf c\_ind}, and {\sf values} using the function {\sf |
---|
1397 | delete[]}! |
---|
1398 | |
---|
1399 | For each driver the array {\sf options} can be used to adapted the |
---|
1400 | computation of the sparse derivative matrices to the special |
---|
1401 | needs of application under consideration. Most frequently, the default options |
---|
1402 | will give a reasonable performance. The elements of the array {\sf options} control the action of |
---|
1403 | {\sf sparse\_jac} according to \autoref{options_sparse_jac}. |
---|
1404 | \begin{table}[h] |
---|
1405 | \center |
---|
1406 | \begin{tabular}{|c|c|l|} \hline |
---|
1407 | component & value & \\ \hline |
---|
1408 | {\sf options[0]} & & way of sparsity pattern computation \\ |
---|
1409 | & 0 & propagation of index domains (default) \\ |
---|
1410 | & 1 & propagation of bit pattern \\ \hline |
---|
1411 | {\sf options[1]} & & test the computational graph control flow \\ |
---|
1412 | & 0 & safe mode (default) \\ |
---|
1413 | & 1 & tight mode \\ \hline |
---|
1414 | {\sf options[2]} & & way of bit pattern propagation \\ |
---|
1415 | & 0 & automatic detection (default) \\ |
---|
1416 | & 1 & forward mode \\ |
---|
1417 | & 2 & reverse mode \\ \hline |
---|
1418 | {\sf options[3]} & & way of compression \\ |
---|
1419 | & 0 & column compression (default) \\ |
---|
1420 | & 1 & row compression \\ \hline |
---|
1421 | \end{tabular} |
---|
1422 | \caption{ {\sf sparse\_jac} parameter {\sf options}\label{options_sparse_jac}} |
---|
1423 | \end{table} |
---|
1424 | |
---|
1425 | The component {\sf options[1]} determines |
---|
1426 | the usage of the safe or tight mode of sparsity computation. |
---|
1427 | The first, more conservative option is the default. It accounts for all |
---|
1428 | dependences that might occur for any value of the |
---|
1429 | independent variables. For example, the intermediate |
---|
1430 | {\sf c}~$=$~{\sf max}$(${\sf a}$,${\sf b}$)$ is |
---|
1431 | always assumed to depend on all independent variables that {\sf a} or {\sf b} |
---|
1432 | dependent on, i.e.\ the bit pattern associated with {\sf c} is set to the |
---|
1433 | logical {\sf OR} of those associated with {\sf a} and {\sf b}. |
---|
1434 | In contrast |
---|
1435 | the tight option gives this result only in the unlikely event of an exact |
---|
1436 | tie {\sf a}~$=$~{\sf b}. Otherwise it sets the bit pattern |
---|
1437 | associated with {\sf c} either to that of {\sf a} or to that of {\sf b}, |
---|
1438 | depending on whether {\sf c}~$=$~{\sf a} or {\sf c}~$=$~{\sf b} locally. |
---|
1439 | Obviously, the sparsity pattern obtained with the tight option may contain |
---|
1440 | more zeros than that obtained with the safe option. On the other hand, it |
---|
1441 | will only be valid at points belonging to an area where the function $F$ is locally |
---|
1442 | analytic and that contains the point at which the internal representation was |
---|
1443 | generated. Since generating the sparsity structure using the safe version does not |
---|
1444 | require any reevaluation, it may thus reduce the overall computational cost |
---|
1445 | despite the fact that it produces more nonzero entries. |
---|
1446 | The value of {\sf options[2]} selects the direction of bit pattern propagation. |
---|
1447 | Depending on the number of independent $n$ and of dependent variables $m$ |
---|
1448 | one would prefer the forward mode if $n$ is significant smaller than $m$ and |
---|
1449 | would otherwise use the reverse mode. |
---|
1450 | |
---|
1451 | The elements of the array {\sf options} control the action of |
---|
1452 | {\sf sparse\_hess} according to \autoref{options_sparse_hess}. |
---|
1453 | \begin{table}[h] |
---|
1454 | \center |
---|
1455 | \begin{tabular}{|c|c|l|} \hline |
---|
1456 | component & value & \\ \hline |
---|
1457 | {\sf options[0]} & & test the computational graph control flow \\ |
---|
1458 | & 0 & safe mode (default) \\ |
---|
1459 | & 1 & tight mode \\ \hline |
---|
1460 | {\sf options[1]} & & way of recovery \\ |
---|
1461 | & 0 & indirect recovery (default) \\ |
---|
1462 | & 1 & direct recovery \\ \hline |
---|
1463 | \end{tabular} |
---|
1464 | \caption{ {\sf sparse\_hess} parameter {\sf options}\label{options_sparse_hess}} |
---|
1465 | \end{table} |
---|
1466 | |
---|
1467 | The described driver routines for the computation of sparse derivative |
---|
1468 | matrices are prototyped in the header file |
---|
1469 | \verb=<adolc/sparse/sparsedrivers.h>=, which is included automatically by the |
---|
1470 | global header file \verb=<adolc/adolc.h>= (see \autoref{ssec:DesIH}). |
---|
1471 | Example codes illustrating the usage of {\sf |
---|
1472 | sparse\_jac} and {\sf sparse\_hess} can be found in the file |
---|
1473 | \verb=sparse_jacobian.cpp= and \verb=sparse_hessian.cpp= contained in %the subdirectory |
---|
1474 | \verb=examples/additional_examples/sparse=. |
---|
1475 | % |
---|
1476 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1477 | \subsubsection*{Computation of Sparsity Pattern} |
---|
1478 | % |
---|
1479 | ADOL-C offers a convenient way of determining the |
---|
1480 | sparsity structure of a Jacobian matrix using the function: |
---|
1481 | % |
---|
1482 | \begin{tabbing} |
---|
1483 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.3in}\= \kill % define tab position |
---|
1484 | \>{\sf int jac\_pat(tag, m, n, x, JP, options)}\\ |
---|
1485 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1486 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1487 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1488 | \>{\sf double x[n];} \> // independent variables $x_0$\\ |
---|
1489 | \>{\sf unsigned int JP[][];} \> // row compressed sparsity structure\\ |
---|
1490 | \>{\sf int options[2];} \> // array of control parameters |
---|
1491 | \end{tabbing} |
---|
1492 | % |
---|
1493 | The sparsity pattern of the |
---|
1494 | Jacobian is computed in a compressed row format. For this purpose, |
---|
1495 | {\sf JP} has to be an $m$ dimensional array of pointers to {\sf |
---|
1496 | unsigned int}s, i.e., one has {\sf unsigned int* JP[m]}. |
---|
1497 | During the call of {\sf jac\_pat}, the number $\hat{n}_i$ of nonzero |
---|
1498 | entries in row $i$ of the Jacobian is determined for all $1\le i\le |
---|
1499 | m$. Then, a memory allocation is performed such that {\sf JP[i-1]} |
---|
1500 | points to a block of $\hat{n}_i+1$ {\sf unsigned int} for all $1\le |
---|
1501 | i\le m$ and {\sf JP[i-1][0]} is set to $\hat{n}_i$. Subsequently, the |
---|
1502 | column indices of the $j$ nonzero entries in the $i$th row are stored |
---|
1503 | in the components {\sf JP[i-1][1]}, \ldots, {\sf JP[i-1][j]}. |
---|
1504 | |
---|
1505 | The elements of the array {\sf options} control the action of |
---|
1506 | {\sf jac\_pat} according to \autoref{options}. |
---|
1507 | \begin{table}[h] |
---|
1508 | \center |
---|
1509 | \begin{tabular}{|c|c|l|} \hline |
---|
1510 | component & value & \\ \hline |
---|
1511 | {\sf options[0]} & & way of sparsity pattern computation \\ |
---|
1512 | & 0 & propagation of index domains (default) \\ |
---|
1513 | & 1 & propagation of bit pattern \\ \hline |
---|
1514 | {\sf options[1]} & & test the computational graph control flow \\ |
---|
1515 | & 0 & safe mode (default) \\ |
---|
1516 | & 1 & tight mode \\ \hline |
---|
1517 | {\sf options[2]} & & way of bit pattern propagation \\ |
---|
1518 | & 0 & automatic detection (default) \\ |
---|
1519 | & 1 & forward mode \\ |
---|
1520 | & 2 & reverse mode \\ \hline |
---|
1521 | \end{tabular} |
---|
1522 | \caption{ {\sf jac\_pat} parameter {\sf options}\label{options}} |
---|
1523 | \end{table} |
---|
1524 | The value of {\sf options[0]} selects the way to compute the sparsity |
---|
1525 | pattern. The component {\sf options[1]} determines |
---|
1526 | the usage of the safe or tight mode of bit pattern propagation. |
---|
1527 | The first, more conservative option is the default. It accounts for all |
---|
1528 | dependences that might occur for any value of the |
---|
1529 | independent variables. For example, the intermediate |
---|
1530 | {\sf c}~$=$~{\sf max}$(${\sf a}$,${\sf b}$)$ is |
---|
1531 | always assumed to depend on all independent variables that {\sf a} or {\sf b} |
---|
1532 | dependent on, i.e.\ the bit pattern associated with {\sf c} is set to the |
---|
1533 | logical {\sf OR} of those associated with {\sf a} and {\sf b}. |
---|
1534 | In contrast |
---|
1535 | the tight option gives this result only in the unlikely event of an exact |
---|
1536 | tie {\sf a}~$=$~{\sf b}. Otherwise it sets the bit pattern |
---|
1537 | associated with {\sf c} either to that of {\sf a} or to that of {\sf b}, |
---|
1538 | depending on whether {\sf c}~$=$~{\sf a} or {\sf c}~$=$~{\sf b} locally. |
---|
1539 | Obviously, the sparsity pattern obtained with the tight option may contain |
---|
1540 | more zeros than that obtained with the safe option. On the other hand, it |
---|
1541 | will only be valid at points belonging to an area where the function $F$ is locally |
---|
1542 | analytic and that contains the point at which the internal representation was |
---|
1543 | generated. Since generating the sparsity structure using the safe version does not |
---|
1544 | require any reevaluation, it may thus reduce the overall computational cost |
---|
1545 | despite the fact that it produces more nonzero entries. The value of |
---|
1546 | {\sf options[2]} selects the direction of bit pattern propagation. |
---|
1547 | Depending on the number of independent $n$ and of dependent variables $m$ |
---|
1548 | one would prefer the forward mode if $n$ is significant smaller than $m$ and |
---|
1549 | would otherwise use the reverse mode. |
---|
1550 | |
---|
1551 | The routine {\sf jac\_pat} may use the propagation of bitpattern to |
---|
1552 | determine the sparsity pattern. Therefore, a kind of ``strip-mining'' |
---|
1553 | is used to cope with large matrix dimensions. If the system happens to run out of memory, one may reduce |
---|
1554 | the value of the constant {\sf PQ\_STRIPMINE\_MAX} |
---|
1555 | following the instructions in \verb=<adolc/sparse/sparse_fo_rev.h>=. |
---|
1556 | |
---|
1557 | The driver routine is prototyped in the header file |
---|
1558 | \verb=<adolc/sparse/sparsedrivers.h>=, which is included automatically by the |
---|
1559 | global header file \verb=<adolc/adolc.h>= (see |
---|
1560 | \autoref{ssec:DesIH}). The determination of sparsity patterns is |
---|
1561 | illustrated by the examples \verb=sparse_jacobian.cpp= |
---|
1562 | and \verb=jacpatexam.cpp= |
---|
1563 | contained in |
---|
1564 | \verb=examples/additional_examples/sparse=. |
---|
1565 | |
---|
1566 | To compute the sparsity pattern of a Hessian in a row compressed form, ADOL-C provides the |
---|
1567 | driver |
---|
1568 | \begin{tabbing} |
---|
1569 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.3in}\= \kill % define tab position |
---|
1570 | \>{\sf int hess\_pat(tag, n, x, HP, options)}\\ |
---|
1571 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1572 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1573 | \>{\sf double x[n];} \> // independent variables $x_0$\\ |
---|
1574 | \>{\sf unsigned int HP[][];} \> // row compressed sparsity structure\\ |
---|
1575 | \>{\sf int option;} \> // control parameter |
---|
1576 | \end{tabbing} |
---|
1577 | where the user has to provide {\sf HP} as an $n$ dimensional array of pointers to {\sf |
---|
1578 | unsigned int}s. |
---|
1579 | After the function call {\sf HP} contains the sparsity pattern, |
---|
1580 | where {\sf HP[j][0]} contains the number of nonzero elements in the |
---|
1581 | $j$th row for $1 \le j\le n$. |
---|
1582 | The components {\sf P[j][i]}, $0<${\sf i}~$\le$~{\sf P[j][0]} store the |
---|
1583 | indices of these entries. For determining the sparsity pattern, ADOL-C uses |
---|
1584 | the algorithm described in \cite{Wa05a}. The parameter{\sf option} determines |
---|
1585 | the usage of the safe ({\sf option = 0}, default) or tight mode ({\sf |
---|
1586 | option = 1}) of the computation of the sparsity pattern as described |
---|
1587 | above. |
---|
1588 | |
---|
1589 | This driver routine is prototyped in the header file |
---|
1590 | \verb=<adolc/sparse/sparsedrivers.h>=, which is included automatically by the |
---|
1591 | global header file \verb=<adolc/adolc.h>= (see \autoref{ssec:DesIH}). |
---|
1592 | An example employing the procedure {\sf hess\_pat} can be found in the file |
---|
1593 | \verb=sparse_hessian.cpp= contained in |
---|
1594 | \verb=examples/additional_examples/sparse=. |
---|
1595 | % |
---|
1596 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1597 | \subsubsection*{Calculation of Seed Matrices} |
---|
1598 | % |
---|
1599 | To compute a compressed derivative matrix from a given sparsity |
---|
1600 | pattern, one has to calculate an appropriate seed matrix that can be |
---|
1601 | used as input for the derivative calculation. To facilitate the |
---|
1602 | generation of seed matrices for a sparsity pattern given in |
---|
1603 | row compressed form, ADOL-C provides the following two drivers, |
---|
1604 | which are based on the ColPack library: |
---|
1605 | \begin{tabbing} |
---|
1606 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.3in}\= \kill % define tab position |
---|
1607 | \>{\sf int generate\_seed\_jac(m, n, JP, S, p)}\\ |
---|
1608 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
1609 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1610 | \>{\sf unsigned int JP[][];} \> // row compressed sparsity structure |
---|
1611 | of Jacobian\\ |
---|
1612 | \>{\sf double S[n][p];} \> // seed matrix\\ |
---|
1613 | \>{\sf int p;} \> // number of columns in $S$ |
---|
1614 | \end{tabbing} |
---|
1615 | The input variables to {\sf generate\_seed\_jac} are the number of dependent variables $m$, the |
---|
1616 | number of independent variables {\sf n} and the sparsity pattern {\sf |
---|
1617 | JP} of the Jacobian computed for example by {\sf jac\_pat}. First, |
---|
1618 | {\sf generate\_seed\_jac} performs a distance-2 coloring of the bipartite graph defined by the sparsity |
---|
1619 | pattern {\sf JP} as described in \cite{GeMaPo05}. The number of colors needed for the coloring |
---|
1620 | determines the number of columns {\sf p} in the seed |
---|
1621 | matrix. Subsequently, {\sf generate\_seed\_jac} allocates the memory needed by {\sf |
---|
1622 | S} and initializes {\sf S} according to the graph coloring. |
---|
1623 | The coloring algorithm that is applied in {\sf |
---|
1624 | generate\_seed\_jac} is used also by the driver {\sf sparse\_jac} |
---|
1625 | described earlier. |
---|
1626 | |
---|
1627 | \begin{tabbing} |
---|
1628 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.3in}\= \kill % define tab position |
---|
1629 | \>{\sf int generate\_seed\_hess(n, HP, S, p)}\\ |
---|
1630 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1631 | \>{\sf unsigned int HP[][];} \> // row compressed sparsity structure |
---|
1632 | of Jacobian\\ |
---|
1633 | \>{\sf double S[n][p];} \> // seed matrix\\ |
---|
1634 | \>{\sf int p;} \> // number of columns in $S$ |
---|
1635 | \end{tabbing} |
---|
1636 | The input variables to {\sf generate\_seed\_hess} are the number of independents $n$ |
---|
1637 | and the sparsity pattern {\sf HP} of the Hessian computed for example |
---|
1638 | by {\sf hess\_pat}. First, {\sf generate\_seed\_hess} performs an |
---|
1639 | appropriate coloring of the adjacency graph defined by the sparsity |
---|
1640 | pattern {\sf HP}: An acyclic coloring in the case of an indirect recovery of the Hessian from its |
---|
1641 | compressed representation and a star coloring in the case of a direct recovery. |
---|
1642 | Subsequently, {\sf generate\_seed\_hess} allocates the memory needed by {\sf |
---|
1643 | S} and initializes {\sf S} according to the graph coloring. |
---|
1644 | The coloring algorithm applied in {\sf |
---|
1645 | generate\_seed\_hess} is used also by the driver {\sf sparse\_hess} |
---|
1646 | described earlier. |
---|
1647 | |
---|
1648 | The specific set of criteria used to define a seed matrix $S$ depends |
---|
1649 | on whether the sparse derivative matrix |
---|
1650 | to be computed is a Jacobian (nonsymmetric) or a Hessian (symmetric). |
---|
1651 | It also depends on whether the entries of the derivative matrix are to be |
---|
1652 | recovered from the compressed representation \emph{directly} |
---|
1653 | (without requiring any further arithmetic) or \emph{indirectly} (for |
---|
1654 | example, by solving for unknowns via successive substitutions). |
---|
1655 | Appropriate recovery routines are provided by ColPack and used |
---|
1656 | in the drivers {\sf sparse\_jac} and {\sf sparse\_hess} described in |
---|
1657 | the previous subsection. Examples with a detailed analysis of the |
---|
1658 | employed drivers for the exploitation of sparsity can be found in the |
---|
1659 | papers \cite{GePoTaWa06} and \cite{GePoWa08}. |
---|
1660 | |
---|
1661 | |
---|
1662 | These driver routines are prototyped in |
---|
1663 | \verb=<adolc/sparse/sparsedrivers.h>=, which is included automatically by the |
---|
1664 | global header file \verb=<adolc/adolc.h>= (see \autoref{ssec:DesIH}). |
---|
1665 | An example code illustrating the usage of {\sf |
---|
1666 | generate\_seed\_jac} and {\sf generate\_seed\_hess} can be found in the file |
---|
1667 | \verb=sparse_jac_hess_exam.cpp= contained in \verb=examples/additional_examples/sparse=. |
---|
1668 | % |
---|
1669 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1670 | \subsection{Higher Derivative Tensors} |
---|
1671 | \label{higherOrderDeriv} |
---|
1672 | % |
---|
1673 | Many applications in scientific computing need second- and higher-order |
---|
1674 | derivatives. Often, one does not require full derivative tensors but |
---|
1675 | only the derivatives in certain directions $s_i \in \R^{n}$. |
---|
1676 | Suppose a collection of $p$ directions |
---|
1677 | $s_i \in \R^{n}$ is given, which form a matrix |
---|
1678 | \[ |
---|
1679 | S\; =\; \left [ s_1, s_2,\ldots, s_p \right ]\; \in \; |
---|
1680 | \R^{n \times p}. |
---|
1681 | \] |
---|
1682 | One possible choice is $S = I_n$ with $p = n$, which leads to |
---|
1683 | full tensors being evaluated. |
---|
1684 | ADOL-C provides the function {\sf tensor\_eval} |
---|
1685 | to calculate the derivative tensors |
---|
1686 | \begin{eqnarray} |
---|
1687 | \label{eq:tensor} |
---|
1688 | \left. \nabla_{\mbox{$\scriptstyle \!\!S$}}^{k} |
---|
1689 | F(x_0) \; = \; \frac{\partial^k}{\partial z^k} F(x_0+Sz) \right |_{z=0} |
---|
1690 | \in \R^{p^k}\quad \mbox{for} \quad k = 0,\ldots,d |
---|
1691 | \end{eqnarray} |
---|
1692 | simultaneously. The function {\sf tensor\_eval} has the following calling sequence and |
---|
1693 | parameters: |
---|
1694 | % |
---|
1695 | \begin{tabbing} |
---|
1696 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1697 | \>{\sf void tensor\_eval(tag,m,n,d,p,x,tensor,S)}\\ |
---|
1698 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1699 | \>{\sf int m;} \> // number of dependent variables $m$ \\ |
---|
1700 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
1701 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
1702 | \>{\sf int p;} \> // number of directions $p$\\ |
---|
1703 | \>{\sf double x[n];} \> // values of independent variables $x_0$\\ |
---|
1704 | \>{\sf double tensor[m][size];}\> // result as defined in \eqref{eq:tensor} in compressed form\\ |
---|
1705 | \>{\sf double S[n][p];} \> // seed matrix $S$ |
---|
1706 | \end{tabbing} |
---|
1707 | % |
---|
1708 | Using the symmetry of the tensors defined by \eqref{eq:tensor}, the memory |
---|
1709 | requirement can be reduced enormously. The collection of tensors up to order $d$ comprises |
---|
1710 | $\binom{p+d}{d}$ distinct elements. Hence, the second dimension of {\sf tensor} must be |
---|
1711 | greater or equal to $\binom{p+d}{d}$. |
---|
1712 | To compute the derivatives, {\sf tensor\_eval} propagates internally univariate Taylor |
---|
1713 | series along $\binom{n+d-1}{d}$ directions. Then the desired values are interpolated. This |
---|
1714 | approach is described in \cite{Griewank97}. |
---|
1715 | |
---|
1716 | The access of individual entries in symmetric tensors of |
---|
1717 | higher order is a little tricky. We always store the derivative |
---|
1718 | values in the two dimensional array {\sf tensor} and provide two |
---|
1719 | different ways of accessing them. |
---|
1720 | The leading dimension of the tensor array ranges over |
---|
1721 | the component index $i$ of the function $F$, i.e., $F_{i+1}$ for $i = |
---|
1722 | 0,\ldots,m-1$. The sub-arrays pointed to by {\sf tensor[i]} have identical |
---|
1723 | structure for all $i$. Each of them represents the symmetric tensors up to |
---|
1724 | order $d$ of the scalar function $F_{i+1}$ in $p$ variables. |
---|
1725 | % |
---|
1726 | The $\binom{p+d}{d}$ mixed partial derivatives in each of the $m$ |
---|
1727 | tensors are linearly ordered according to the tetrahedral |
---|
1728 | scheme described by Knuth \cite{Knuth73}. In the familiar quadratic |
---|
1729 | case $d=2$ the derivative with respect to $z_j$ and $z_k$ with $z$ |
---|
1730 | as in \eqref{eq:tensor} and $j \leq k$ is stored at {\sf tensor[i][l]} with |
---|
1731 | $l = k*(k+1)/2+j$. At $j = 0 = k$ and hence $l = 0$ we find the |
---|
1732 | function value $F_{i+1}$ itself and the gradient |
---|
1733 | $\nabla F_{i+1}= \partial F_{i+1}/\partial x_k $ is stored at $l=k(k+1)/2$ |
---|
1734 | with $j=0$ for $k=1,\ldots,p$. |
---|
1735 | |
---|
1736 | For general $d$ we combine the variable |
---|
1737 | indices to a multi-index $j = (j_1,j_2,\ldots,j_d)$, |
---|
1738 | where $j_k$ indicates differentiation with respect to variable |
---|
1739 | $x_{j_k}$ with $j_k \in \{0,1,\ldots,p\}$. The value $j_k=0$ indicates |
---|
1740 | no differentiation so that all lower derivatives are also |
---|
1741 | contained in the same data structure as described above for |
---|
1742 | the quadratic case. The location of the partial derivative specified |
---|
1743 | by $j$ is computed by the function |
---|
1744 | % |
---|
1745 | \begin{tabbing} |
---|
1746 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1747 | \>{\sf int address(d,$\,$j)} \\ |
---|
1748 | \>{\sf int d;} \> // highest derivative degree $d$ \\ |
---|
1749 | \>{\sf int j[d];} \> // multi-index $j$ |
---|
1750 | \end{tabbing} |
---|
1751 | % |
---|
1752 | and it may thus be referenced as {\sf tensor[i][address(d,$\,$j)]}. |
---|
1753 | Notice that the address computation does depend on the degree $d$ |
---|
1754 | but not on the number of directions $p$, which could theoretically be |
---|
1755 | enlarged without the need to reallocate the original tensor. |
---|
1756 | Also, the components of $j$ need to be non-increasing. |
---|
1757 | % |
---|
1758 | To some C programmers it may appear more natural to access tensor |
---|
1759 | entries by successive dereferencing in the form |
---|
1760 | {\sf tensorentry[i][$\,$j1$\,$][$\,$j2$\,$]$\ldots$[$\,$jd$\,$]}. |
---|
1761 | We have also provided this mode, albeit with the restriction |
---|
1762 | that the indices $j_1,j_2,\ldots,j_d$ are non-increasing. |
---|
1763 | In the second order case this means that the Hessian entries must be |
---|
1764 | specified in or below the diagonal. If this restriction is |
---|
1765 | violated the values are almost certain to be wrong and array bounds |
---|
1766 | may be violated. We emphasize that subscripting is not overloaded |
---|
1767 | but that {\sf tensorentry} is a conventional and |
---|
1768 | thus moderately efficient C pointer structure. |
---|
1769 | Such a pointer structure can be allocated and set up completely by the |
---|
1770 | function |
---|
1771 | % |
---|
1772 | \begin{tabbing} |
---|
1773 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1774 | \>{\sf void** tensorsetup(m,p,d,tensor)} \\ |
---|
1775 | \>{\sf int m;} \> // number of dependent variables $n$ \\ |
---|
1776 | \>{\sf int p;} \> // number of directions $p$\\ |
---|
1777 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
1778 | \>{\sf double tensor[m][size];}\> // pointer to two dimensional array |
---|
1779 | \end{tabbing} |
---|
1780 | % |
---|
1781 | Here, {\sf tensor} is the array of $m$ pointers pointing to arrays of {\sf size} |
---|
1782 | $\geq \binom{p+d}{d}$ allocated by the user before. During the execution of {\sf tensorsetup}, |
---|
1783 | $d-1$ layers of pointers are set up so that the return value |
---|
1784 | allows the direct dereferencing of individual tensor elements. |
---|
1785 | |
---|
1786 | For example, suppose some active section involving $m \geq 5$ dependents and |
---|
1787 | $n \geq 2$ independents has been executed and taped. We may |
---|
1788 | select $p=2$, $d=3$ and initialize the $n\times 2$ seed matrix $S$ with two |
---|
1789 | columns $s_1$ and $s_2$. Then we are able to execute the code segment |
---|
1790 | \begin{tabbing} |
---|
1791 | \hspace{0.5in}\={\sf double**** tensorentry = (double****) tensorsetup(m,p,d,tensor);} \\ |
---|
1792 | \>{\sf tensor\_eval(tag,m,n,d,p,x,tensor,S);} |
---|
1793 | \end{tabbing} |
---|
1794 | This way, we evaluated all tensors defined in \eqref{eq:tensor} up to degree 3 |
---|
1795 | in both directions $s_1$ and |
---|
1796 | $s_2$ at some argument $x$. To allow the access of tensor entries by dereferencing the pointer |
---|
1797 | structure {\sf tensorentry} has been created. Now, |
---|
1798 | the value of the mixed partial |
---|
1799 | \[ |
---|
1800 | \left. \frac{\partial ^ 3 F_5(x+s_1 z_1+s_2 z_2)}{\partial z_1^2 \partial z_2} \right |_{z_1=0=z_2} |
---|
1801 | \] |
---|
1802 | can be recovered as |
---|
1803 | \begin{center} |
---|
1804 | {\sf tensorentry[4][2][1][1]} \hspace{0.2in} or \hspace{0.2in} {\sf tensor[4][address(d,$\,$j)]}, |
---|
1805 | \end{center} |
---|
1806 | where the integer array {\sf j} may equal (1,1,2), (1,2,1) or (2,1,1). |
---|
1807 | Analogously, the entry |
---|
1808 | \begin{center} |
---|
1809 | {\sf tensorentry[2][1][0][0]} \hspace{0.2in} or \hspace{0.2in} {\sf tensor[2][address(d,$\,$j)]} |
---|
1810 | \end{center} |
---|
1811 | with {\sf j} = (1,0,0) contains the first derivative of the third dependent |
---|
1812 | variable $F_3$ with respect to the first differentiation parameter $z_1$. |
---|
1813 | |
---|
1814 | Note, that the pointer structure {\sf tensorentry} has to be set up only once. Changing the values of the |
---|
1815 | array {\sf tensor}, e.g.~by a further call of {\sf tensor\_eval}, directly effects the values accessed |
---|
1816 | by {\sf tensorentry}. |
---|
1817 | % |
---|
1818 | When no more derivative evaluations are desired the pointer structure |
---|
1819 | {\sf tensorentry} can be deallocated by a call to the function |
---|
1820 | % |
---|
1821 | \begin{tabbing} |
---|
1822 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1823 | \>{\sf int freetensor(m,p,d, (double ****) tensorentry)}\\ |
---|
1824 | \>{\sf int m;} \> // number of dependent variables $m$ \\ |
---|
1825 | \>{\sf int p;} \> // number of independent variables $p$\\ |
---|
1826 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
1827 | \>{\sf double*** tensorentry[m];} \> // return value of {\sf tensorsetup} |
---|
1828 | \end{tabbing} |
---|
1829 | % |
---|
1830 | that does not deallocate the array {\sf tensor}. |
---|
1831 | |
---|
1832 | The drivers provided for efficient calculation of higher order |
---|
1833 | derivatives are prototyped in the header file \verb=<adolc/drivers/taylor.h>=, |
---|
1834 | which is included by the global header file \verb=<adolc/adolc.h>= automatically |
---|
1835 | (see \autoref{ssec:DesIH}). |
---|
1836 | Example codes using the above procedures can be found in the files |
---|
1837 | \verb=taylorexam.C= and \verb=accessexam.C= contained in the subdirectory |
---|
1838 | \verb=examples/additional_examples/taylor=. |
---|
1839 | % |
---|
1840 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1841 | \subsection{Derivatives of Implicit and Inverse Functions} |
---|
1842 | \label{implicitInverse} |
---|
1843 | % |
---|
1844 | Frequently, one needs derivatives of variables |
---|
1845 | $y \in \R^{m}$ that are implicitly defined as |
---|
1846 | functions of some variables $x \in \R^{n-m}$ |
---|
1847 | by an algebraic system of equations |
---|
1848 | \[ |
---|
1849 | G(z) \; = \; 0 \in \R^m \quad |
---|
1850 | {\rm with} \quad z = (y, x) \in \R^n . |
---|
1851 | \] |
---|
1852 | Naturally, the $n$ arguments of $G$ need not be partitioned in |
---|
1853 | this regular fashion and we wish to provide flexibility for a |
---|
1854 | convenient selection of the $n-m$ {\em truly} independent |
---|
1855 | variables. Let $P \in \R^{(n-m)\times n}$ be a $0-1$ matrix |
---|
1856 | that picks out these variables so that it is a column |
---|
1857 | permutation of the matrix $[0,I_{n-m}] \in \R^{(n-m)\times n}$. |
---|
1858 | Then the nonlinear system |
---|
1859 | \[ |
---|
1860 | G(z) \; = \; 0, \quad P z = x, |
---|
1861 | \] |
---|
1862 | has a regular Jacobian, wherever the implicit function theorem |
---|
1863 | yields $y$ as a function of $x$. Hence, we may also write |
---|
1864 | \begin{equation} |
---|
1865 | \label{eq:inv_tensor} |
---|
1866 | F(z) = \left(\begin{array}{c} |
---|
1867 | G(z) \\ |
---|
1868 | P z |
---|
1869 | \end{array} \right)\; \equiv \; |
---|
1870 | \left(\begin{array}{c} |
---|
1871 | 0 \\ |
---|
1872 | P z |
---|
1873 | \end{array} \right)\; \equiv \; S\, x, |
---|
1874 | \end{equation} |
---|
1875 | where $S = [0,I_p]^{T} \in \R^{n \times p}$ with $p=n-m$. Now, we have rewritten |
---|
1876 | the original implicit functional relation between $x$ and $y$ as an inverse |
---|
1877 | relation $F(z) = Sx$. In practice, we may implement the projection $P$ simply |
---|
1878 | by marking $n-m$ of the independents also dependent. |
---|
1879 | |
---|
1880 | Given any $ F : \R^n \mapsto \R^n $ that is locally invertible and an arbitrary |
---|
1881 | seed matrix $S \in \R^{n \times p}$ we may evaluate all derivatives of $z \in \R^n$ |
---|
1882 | with respect to $x \in \R^p$ by calling the following routine: |
---|
1883 | % |
---|
1884 | \begin{tabbing} |
---|
1885 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
1886 | \>{\sf void inverse\_tensor\_eval(tag,n,d,p,z,tensor,S)}\\ |
---|
1887 | \>{\sf short int tag;} \> // tape identification \\ |
---|
1888 | \>{\sf int n;} \> // number of variables $n$\\ |
---|
1889 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
1890 | \>{\sf int p;} \> // number of directions $p$\\ |
---|
1891 | \>{\sf double z[n];} \> // values of independent variables $z$\\ |
---|
1892 | \>{\sf double tensor[n][size];}\> // partials of $z$ with respect to $x$\\ |
---|
1893 | \>{\sf double S[n][p];} \> // seed matrix $S$ |
---|
1894 | \end{tabbing} |
---|
1895 | % |
---|
1896 | The results obtained in {\sf tensor} are exactly the same as if we had called {\sf tensor\_eval} with |
---|
1897 | {\sf tag} pointing to a tape for the evaluation of the inverse function |
---|
1898 | $z=F^{-1}(y)$ for which naturally $n=m$. Note that the columns of $S$ belong |
---|
1899 | to the domain of that function. Individual derivative components can be |
---|
1900 | accessed in tensor exactly as in the explicit case described above. |
---|
1901 | |
---|
1902 | It must be understood that {\sf inverse\_tensor\_eval} actually computes the |
---|
1903 | derivatives of $z$ with respect to $x$ that is defined by the equation |
---|
1904 | $F(z)=F(z_0)+S \, x$. In other words the base point at |
---|
1905 | which the inverse function is differentiated is given by $F(z_0)$. |
---|
1906 | The routine has no capability for inverting $F$ itself as |
---|
1907 | solving systems of nonlinear |
---|
1908 | equations $F(z)=0$ in the first place is not just a differentiation task. |
---|
1909 | However, the routine {\sf jac\_solv} described in \autoref{optdrivers} may certainly be very |
---|
1910 | useful for that purpose. |
---|
1911 | |
---|
1912 | As an example consider the following two nonlinear expressions |
---|
1913 | \begin{eqnarray*} |
---|
1914 | G_1(z_1,z_2,z_3,z_4) & = & z_1^2+z_2^2-z_3^2 \\ |
---|
1915 | G_2(z_1,z_2,z_3,z_4) & = & \cos(z_4) - z_1/z_3 \enspace . |
---|
1916 | \end{eqnarray*} |
---|
1917 | The equations $G(z)=0$ describe the relation between the Cartesian |
---|
1918 | coordinates $(z_1,z_2)$ and the polar coordinates $(z_3,z_4)$ in the plane. |
---|
1919 | Now, suppose we are interested in the derivatives of the second Cartesian |
---|
1920 | $y_1=z_2$ and the second (angular) polar coordinate $y_2=z_4$ with respect |
---|
1921 | to the other two variables $x_1=z_1$ and $x_2=z_3$. Then the active section |
---|
1922 | could look simply like |
---|
1923 | % |
---|
1924 | \begin{tabbing} |
---|
1925 | \hspace{1.5in}\={\sf for (j=1; j $<$ 5;$\,$j++)}\hspace{0.15in} \= {\sf z[j] \boldmath $\ll=$ \unboldmath zp[j];}\\ |
---|
1926 | \>{\sf g[1] = z[1]*z[1]+z[2]*z[2]-z[3]*z[3]; }\\ |
---|
1927 | \>{\sf g[2] = cos(z[4]) - z[1]/z[3]; }\\ |
---|
1928 | \>{\sf g[1] \boldmath $\gg=$ \unboldmath gp[1];} \> {\sf g[2] \boldmath $\gg=$ \unboldmath gp[2];}\\ |
---|
1929 | \>{\sf z[1] \boldmath $\gg=$ \unboldmath zd[1];} \> {\sf z[3] \boldmath $\gg=$ \unboldmath zd[2];} |
---|
1930 | \end{tabbing} |
---|
1931 | % |
---|
1932 | where {\sf zd[1]} and {\sf zd[2]} are dummy arguments. |
---|
1933 | In the last line the two independent variables {\sf z[1]} and |
---|
1934 | {\sf z[3]} are made |
---|
1935 | simultaneously dependent thus generating a square system that can be |
---|
1936 | inverted (at most arguments). The corresponding projection and seed |
---|
1937 | matrix are |
---|
1938 | \begin{eqnarray*} |
---|
1939 | P \;=\; \left( \begin{array}{cccc} |
---|
1940 | 1 & 0 & 0 & 0 \\ |
---|
1941 | 0 & 0 & 1 & 0 |
---|
1942 | \end{array}\right) \quad \mbox{and} \quad |
---|
1943 | S^T \; = \; \left( \begin{array}{cccc} |
---|
1944 | 0 & 0 & 1 & 0 \\ |
---|
1945 | 0 & 0 & 0 & 1 |
---|
1946 | \end{array}\right) \enspace . |
---|
1947 | \end{eqnarray*} |
---|
1948 | Provided the vector {\sf zp} is consistent in that its Cartesian and polar |
---|
1949 | components describe the same point in the plane the resulting tuple |
---|
1950 | {\sf gp} must vanish. The call to {\sf inverse\_tensor\_eval} with |
---|
1951 | $n=4$, $p=2$ and $d$ |
---|
1952 | as desired will yield the implicit derivatives, provided |
---|
1953 | {\sf tensor} has been allocated appropriately of course and $S$ has the value |
---|
1954 | given above. |
---|
1955 | % |
---|
1956 | The example is untypical in that the implicit function could also be |
---|
1957 | obtained explicitly by symbolic mani\-pu\-lations. It is typical in that |
---|
1958 | the subset of $z$ components that are to be considered as truly |
---|
1959 | independent can be selected and altered with next to no effort at all. |
---|
1960 | |
---|
1961 | The presented drivers are prototyped in the header file |
---|
1962 | \verb=<adolc/drivers/taylor.h>=. As indicated before this header |
---|
1963 | is included by the global header file \verb=<adolc/adolc.h>= automatically |
---|
1964 | (see \autoref{ssec:DesIH}). |
---|
1965 | The example programs \verb=inversexam.cpp=, \verb=coordinates.cpp= and |
---|
1966 | \verb=trigger.cpp= in the directory \verb=examples/additional_examples/taylor= |
---|
1967 | show the application of the procedures described here. |
---|
1968 | % |
---|
1969 | % |
---|
1970 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
1971 | \section{Basic Drivers for the Forward and Reverse Mode} |
---|
1972 | \label{forw_rev_ad} |
---|
1973 | % |
---|
1974 | In this section, we present tailored drivers for different |
---|
1975 | variants of the forward mode and the reverse mode, respectively. |
---|
1976 | For a better understanding, we start with a short |
---|
1977 | description of the mathematical background. |
---|
1978 | |
---|
1979 | Provided no arithmetic exception occurs, |
---|
1980 | no comparison including {\sf fmax} or {\sf fmin} yields a tie, |
---|
1981 | {\sf fabs} does not yield zero, |
---|
1982 | and all special functions were evaluated in the |
---|
1983 | interior of their domains, the functional relation between the input |
---|
1984 | variables $x$ |
---|
1985 | and the output variables $y$ denoted by $y=F(x)$ is in |
---|
1986 | fact analytic. In other words, we can compute arbitrarily high |
---|
1987 | derivatives of the vector function $F : I\!\!R^n \mapsto I\!\!R^m$ defined |
---|
1988 | by the active section. |
---|
1989 | We find it most convenient to describe and |
---|
1990 | compute derivatives in terms of univariate Taylor expansions, which |
---|
1991 | are truncated after the highest derivative degree $d$ that is desired |
---|
1992 | by the user. Let |
---|
1993 | \begin{equation} |
---|
1994 | \label{eq:x_of_t} |
---|
1995 | x(t) \; \equiv \; \sum_{j=0}^dx_jt^j \; : \; I\!\!R \; \mapsto \; |
---|
1996 | I\!\!R^n |
---|
1997 | \end{equation} |
---|
1998 | denote any vector polynomial in the scalar variable $t \in I\!\!R$. |
---|
1999 | In other words, $x(t)$ describes a path in $I\!\!R^n$ parameterized by $t$. |
---|
2000 | The Taylor coefficient vectors |
---|
2001 | \[ x_j \; = \; |
---|
2002 | \frac{1}{j!} \left . \frac{\partial ^j}{\partial t^j} x(t) |
---|
2003 | \right |_{t=0} |
---|
2004 | \] |
---|
2005 | are simply the scaled derivatives of $x(t)$ at the parameter |
---|
2006 | origin $t=0$. The first two vectors $x_1,x_2 \in I\!\!R^n$ can be |
---|
2007 | visualized as tangent and curvature at the base point $x_0$, |
---|
2008 | respectively. |
---|
2009 | Provided that $F$ is $d$ times continuously differentiable, it |
---|
2010 | follows from the chain rule that the image path |
---|
2011 | \begin{equation} |
---|
2012 | \label{eq:rela} |
---|
2013 | y(t) \; \equiv \; F(x(t)) \; : \; I\!\!R \;\mapsto \;I\!\!R^m |
---|
2014 | \end{equation} |
---|
2015 | is also smooth and has $(d+1)$ Taylor coefficient vectors |
---|
2016 | $y_j \in I\!\!R^m$ at $t=0$, so that |
---|
2017 | \begin{equation} |
---|
2018 | \label{eq:series} |
---|
2019 | y(t) \; = \; \sum_{j=0}^d y_jt^j + O(t^{d+1}). |
---|
2020 | \end{equation} |
---|
2021 | Also as a consequence of the chain rule, one can observe that |
---|
2022 | each $y_j$ is uniquely and smoothly determined by the coefficient |
---|
2023 | vectors $x_i$ with $i \leq j$. In particular we have |
---|
2024 | \begin{align} |
---|
2025 | \label{eq:y_0y_1} |
---|
2026 | y_0 & = F(x_0) \nonumber \\ |
---|
2027 | y_1 & = F'(x_0) x_1 \nonumber\\ |
---|
2028 | y_2 & = F'(x_0) x_2 + \frac{1}{2}F''(x_0)x_1 x_1 \\ |
---|
2029 | y_3 & = F'(x_0) x_3 + F''(x_0)x_1 x_2 |
---|
2030 | + \frac{1}{6}F'''(x_0)x_1 x_1 x_1\nonumber\\ |
---|
2031 | & \ldots\nonumber |
---|
2032 | \end{align} |
---|
2033 | In writing down the last equations we have already departed from the |
---|
2034 | usual matrix-vector notation. It is well known that the number of |
---|
2035 | terms that occur in these ``symbolic'' expressions for |
---|
2036 | the $y_j$ in terms of the first $j$ derivative tensors of $F$ and |
---|
2037 | the ``input'' coefficients $x_i$ with $i\leq j$ grows very rapidly |
---|
2038 | with $j$. Fortunately, this exponential growth does not occur |
---|
2039 | in automatic differentiation, where the many terms are somehow |
---|
2040 | implicitly combined so that storage and operations count grow only |
---|
2041 | quadratically in the bound $d$ on $j$. |
---|
2042 | |
---|
2043 | Provided $F$ is analytic, this property is inherited by the functions |
---|
2044 | \[ |
---|
2045 | y_j = y_j (x_0,x_1, \ldots ,x_j) \in {I\!\!R}^m , |
---|
2046 | \] |
---|
2047 | and their derivatives satisfy the identities |
---|
2048 | \begin{equation} |
---|
2049 | \label{eq:identity} |
---|
2050 | \frac{\partial y_j}{\partial x_i} = \frac{\partial y_{j-i}} |
---|
2051 | {\partial x_0} = A_{j-i}(x_0,x_1, \ldots ,x_{j-i}) |
---|
2052 | \end{equation} |
---|
2053 | as established in \cite{Chri91a}. This yields in particular |
---|
2054 | \begin{align*} |
---|
2055 | \frac{\partial y_0}{\partial x_0} = |
---|
2056 | \frac{\partial y_1}{\partial x_1} = |
---|
2057 | \frac{\partial y_2}{\partial x_2} = |
---|
2058 | \frac{\partial y_3}{\partial x_3} = |
---|
2059 | A_0 & = F'(x_0) \\ |
---|
2060 | \frac{\partial y_1}{\partial x_0} = |
---|
2061 | \frac{\partial y_2}{\partial x_1} = |
---|
2062 | \frac{\partial y_3}{\partial x_2} = |
---|
2063 | A_1 & = F''(x_0) x_1 \\ |
---|
2064 | \frac{\partial y_2}{\partial x_0} = |
---|
2065 | \frac{\partial y_3}{\partial x_1} = |
---|
2066 | A_2 & = F''(x_0) x_2 + \frac{1}{2}F'''(x_0)x_1 x_1 \\ |
---|
2067 | \frac{\partial y_3}{\partial x_0} = |
---|
2068 | A_3 & = F''(x_0) x_3 + F'''(x_0)x_1 x_2 |
---|
2069 | + \frac{1}{6}F^{(4)}(x_0)x_1 x_1 x_1 \\ |
---|
2070 | & \ldots |
---|
2071 | \end{align*} |
---|
2072 | The $m \times n$ matrices $A_k, k=0,\ldots,d$, are actually the Taylor |
---|
2073 | coefficients of the Jacobian path $F^\prime(x(t))$, a fact that is of |
---|
2074 | interest primarily in the context of ordinary differential |
---|
2075 | equations and differential algebraic equations. |
---|
2076 | |
---|
2077 | Given the tape of an active section and the coefficients $x_j$, |
---|
2078 | the resulting $y_j$ and their derivatives $A_j$ can be evaluated |
---|
2079 | by appropriate calls to the ADOL-C forward mode implementations and |
---|
2080 | the ADOL-C reverse mode implementations. The scalar versions of the forward |
---|
2081 | mode propagate just one truncated Taylor series from the $(x_j)_{j\leq d}$ |
---|
2082 | to the $(y_j)_{j\leq d}$. The vector versions of the forward |
---|
2083 | mode propagate families of $p\geq 1$ such truncated Taylor series |
---|
2084 | in order to reduce the relative cost of the overhead incurred |
---|
2085 | in the tape interpretation. In detail, ADOL-C provides |
---|
2086 | \begin{tabbing} |
---|
2087 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2088 | \>{\sf int zos\_forward(tag,m,n,keep,x,y)}\\ |
---|
2089 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2090 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2091 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2092 | \>{\sf int keep;} \> // flag for reverse mode preparation\\ |
---|
2093 | \>{\sf double x[n];} \> // independent vector $x=x_0$\\ |
---|
2094 | \>{\sf double y[m];} \> // dependent vector $y=F(x_0)$ |
---|
2095 | \end{tabbing} |
---|
2096 | for the {\bf z}ero-{\bf o}rder {\bf s}calar forward mode. This driver computes |
---|
2097 | $y=F(x)$ with $0\leq\text{\sf keep}\leq 1$. The integer |
---|
2098 | flag {\sf keep} plays a similar role as in the call to |
---|
2099 | {\sf trace\_on}: It determines if {\sf zos\_forward} writes |
---|
2100 | the first Taylor coefficients of all intermediate quantities into a buffered |
---|
2101 | temporary file, i.e., the value stack, in preparation for a subsequent |
---|
2102 | reverse mode evaluation. The value {\sf keep} $=1$ |
---|
2103 | prepares for {\sf fos\_reverse} or {\sf fov\_reverse} as exlained below. |
---|
2104 | |
---|
2105 | To compute first-order derivatives, one has |
---|
2106 | \begin{tabbing} |
---|
2107 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2108 | \>{\sf int fos\_forward(tag,m,n,keep,x0,x1,y0,y1)}\\ |
---|
2109 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2110 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2111 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2112 | \>{\sf int keep;} \> // flag for reverse mode preparation\\ |
---|
2113 | \>{\sf double x0[n];} \> // independent vector $x_0$\\ |
---|
2114 | \>{\sf double x1[n];} \> // tangent vector $x_1$\\ |
---|
2115 | \>{\sf double y0[m];} \> // dependent vector $y_0=F(x_0)$\\ |
---|
2116 | \>{\sf double y1[m];} \> // first derivative $y_1=F'(x_0)x_1$ |
---|
2117 | \end{tabbing} |
---|
2118 | for the {\bf f}irst-{\bf o}rder {\bf s}calar forward mode. Here, one has |
---|
2119 | $0\leq\text{\sf keep}\leq 2$, where |
---|
2120 | \begin{align*} |
---|
2121 | \text{\sf keep} = \left\{\begin{array}{cl} |
---|
2122 | 1 & \text{prepares for {\sf fos\_reverse} or {\sf fov\_reverse}} \\ |
---|
2123 | 2 & \text{prepares for {\sf hos\_reverse} or {\sf hov\_reverse}} |
---|
2124 | \end{array}\right. |
---|
2125 | \end{align*} |
---|
2126 | as exlained below. For the {\bf f}irst-{\bf o}rder {\bf v}ector forward mode, |
---|
2127 | ADOL-C provides |
---|
2128 | \begin{tabbing} |
---|
2129 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2130 | \>{\sf int fov\_forward(tag,m,n,p,x0,X,y0,Y)}\\ |
---|
2131 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2132 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2133 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2134 | \>{\sf int p;} \> // number of directions\\ |
---|
2135 | \>{\sf double x0[n];} \> // independent vector $x_0$\\ |
---|
2136 | \>{\sf double X[n][p];} \> // tangent matrix $X$\\ |
---|
2137 | \>{\sf double y0[m];} \> // dependent vector $y_0=F(x_0)$\\ |
---|
2138 | \>{\sf double Y[m][p];} \> // first derivative matrix $Y=F'(x)X$ |
---|
2139 | \end{tabbing} |
---|
2140 | For the computation of higher derivative, the driver |
---|
2141 | \begin{tabbing} |
---|
2142 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2143 | \>{\sf int hos\_forward(tag,m,n,d,keep,x0,X,y0,Y)}\\ |
---|
2144 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2145 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2146 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2147 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2148 | \>{\sf int keep;} \> // flag for reverse mode preparation\\ |
---|
2149 | \>{\sf double x0[n];} \> // independent vector $x_0$\\ |
---|
2150 | \>{\sf double X[n][d];} \> // tangent matrix $X$\\ |
---|
2151 | \>{\sf double y0[m];} \> // dependent vector $y_0=F(x_0)$\\ |
---|
2152 | \>{\sf double Y[m][d];} \> // derivative matrix $Y$ |
---|
2153 | \end{tabbing} |
---|
2154 | implementing the {\bf h}igher-{\bf o}rder {\bf s}calar forward mode. |
---|
2155 | The rows of the matrix $X$ must correspond to the independent variables in the order of their |
---|
2156 | initialization by the \boldmath $\ll=$ \unboldmath operator. The columns of |
---|
2157 | $X = \{x_j\}_{j=1\ldots d}$ represent Taylor coefficient vectors as in |
---|
2158 | \eqref{eq:x_of_t}. The rows of the matrix $Y$ must correspond to the |
---|
2159 | dependent variables in the order of their selection by the \boldmath $\gg=$ \unboldmath operator. |
---|
2160 | The columns of $Y = \{y_j\}_{j=1\ldots d}$ represent |
---|
2161 | Taylor coefficient vectors as in \eqref{eq:series}, i.e., {\sf hos\_forward} |
---|
2162 | computes the values |
---|
2163 | $y_0=F(x_0)$, $y_1=F'(x_0)x_1$, \ldots, where |
---|
2164 | $X=[x_1,x_2,\ldots,x_d]$ and $Y=[y_1,y_2,\ldots,y_d]$. Furthermore, one has |
---|
2165 | $0\leq\text{\sf keep}\leq d+1$, with |
---|
2166 | \begin{align*} |
---|
2167 | \text{\sf keep} \left\{\begin{array}{cl} |
---|
2168 | = 1 & \text{prepares for {\sf fos\_reverse} or {\sf fov\_reverse}} \\ |
---|
2169 | > 1 & \text{prepares for {\sf hos\_reverse} or {\sf hov\_reverse}} |
---|
2170 | \end{array}\right. |
---|
2171 | \end{align*} |
---|
2172 | Once more, there is also a vector version given by |
---|
2173 | \begin{tabbing} |
---|
2174 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2175 | \>{\sf int hov\_forward(tag,m,n,d,p,x0,X,y0,Y)}\\ |
---|
2176 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2177 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2178 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2179 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2180 | \>{\sf int p;} \> // number of directions $p$\\ |
---|
2181 | \>{\sf double x0[n];} \> // independent vector $x_0$\\ |
---|
2182 | \>{\sf double X[n][p][d];} \> // tangent matrix $X$\\ |
---|
2183 | \>{\sf double y0[m];} \> // dependent vector $y_0=F(x_0)$\\ |
---|
2184 | \>{\sf double Y[m][p][d];} \> // derivative matrix $Y$ |
---|
2185 | \end{tabbing} |
---|
2186 | for the {\bf h}igher-{\bf o}rder {\bf v}ector forward mode that computes |
---|
2187 | $y_0=F(x_0)$, $Y_1=F'(x_0)X_1$, \ldots, where $X=[X_1,X_2,\ldots,X_d]$ and |
---|
2188 | $Y=[Y_1,Y_2,\ldots,Y_d]$. |
---|
2189 | |
---|
2190 | There are also overloaded versions providing a general {\sf forward}-call. |
---|
2191 | Details of the appropriate calling sequences are given in \autoref{forw_rev}. |
---|
2192 | |
---|
2193 | Once, the required information is generated due to a forward mode evaluation |
---|
2194 | with an approriate value of the parameter {\sf keep}, one may use the |
---|
2195 | following implementation variants of the reverse mode. To compute first-order derivatives |
---|
2196 | one can use |
---|
2197 | \begin{tabbing} |
---|
2198 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2199 | \>{\sf int fos\_reverse(tag,m,n,u,z)}\\ |
---|
2200 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2201 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2202 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2203 | \>{\sf double u[m];} \> // weight vector $u$\\ |
---|
2204 | \>{\sf double z[n];} \> // resulting adjoint value $z^T=u^T F'(x)$ |
---|
2205 | \end{tabbing} |
---|
2206 | as {\bf f}irst-{\bf o}rder {\bf s}calar reverse mode implementation that computes |
---|
2207 | the product $z^T=u^T F'(x)$ after calling {\sf zos\_forward}, {\sf fos\_forward}, or |
---|
2208 | {\sf hos\_forward} with {\sf keep}=1. The corresponding {\bf f}irst-{\bf |
---|
2209 | o}rder {\bf v}ector reverse mode driver is given by |
---|
2210 | \begin{tabbing} |
---|
2211 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2212 | \>{\sf int fov\_reverse(tag,m,n,q,U,Z)}\\ |
---|
2213 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2214 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2215 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2216 | \>{\sf int q;} \> // number of weight vectors $q$\\ |
---|
2217 | \>{\sf double U[q][m];} \> // weight matrix $U$\\ |
---|
2218 | \>{\sf double Z[q][n];} \> // resulting adjoint $Z=U F'(x)$ |
---|
2219 | \end{tabbing} |
---|
2220 | that can be used after calling {\sf zos\_forward}, {\sf fos\_forward}, or |
---|
2221 | {\sf hos\_forward} with {\sf keep}=1. To compute higher-order derivatives, |
---|
2222 | ADOL-C provides |
---|
2223 | \begin{tabbing} |
---|
2224 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2225 | \>{\sf int hos\_reverse(tag,m,n,d,u,Z)}\\ |
---|
2226 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2227 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2228 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2229 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2230 | \>{\sf double u[m];} \> // weight vector $u$\\ |
---|
2231 | \>{\sf double Z[n][d+1];} \> // resulting adjoints |
---|
2232 | \end{tabbing} |
---|
2233 | as {\bf h}igher-{\bf o}rder {\bf s}calar reverse mode implementation yielding |
---|
2234 | the adjoints $z_0^T=u^T F'(x_0)=u^T A_0$, $z_1^T=u^T F''(x_0)x_1=u^T A_1$, |
---|
2235 | \ldots, where $Z=[z_0,z_1,\ldots,z_d]$ after calling {\sf fos\_forward} or |
---|
2236 | {\sf hos\_forward} with {\sf keep} $=d+1>1$. The vector version is given by |
---|
2237 | \begin{tabbing} |
---|
2238 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2239 | \>{\sf int hov\_reverse(tag,m,n,d,q,U,Z,nz)}\\ |
---|
2240 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2241 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2242 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2243 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2244 | \>{\sf double U[q][m];} \> // weight vector $u$\\ |
---|
2245 | \>{\sf double Z[q][n][d+1];} \> // resulting adjoints\\ |
---|
2246 | \>{\sf short int nz[q][n];} \> // nonzero pattern of {\sf Z} |
---|
2247 | \end{tabbing} |
---|
2248 | as {\bf h}igher-{\bf o}rder {\bf v}ector reverse mode driver to compute |
---|
2249 | the adjoints $Z_0=U F'(x_0)=U A_0$, $Z_1=U F''(x_0)x_1=U A_1$, |
---|
2250 | \ldots, where $Z=[Z_0,Z_1,\ldots,Z_d]$ after calling {\sf fos\_forward} or |
---|
2251 | {\sf hos\_forward} with {\sf keep} $=d+1>1$. |
---|
2252 | After the function call, the last argument of {\sf hov\_reverse} |
---|
2253 | contains information about the sparsity pattern, i.e. each {\sf nz[i][j]} |
---|
2254 | has a value that characterizes the functional relation between the |
---|
2255 | $i$-th component of $UF^\prime(x)$ and the $j$-th independent value |
---|
2256 | $x_j$ as: |
---|
2257 | \begin{center} |
---|
2258 | \begin{tabular}{ll} |
---|
2259 | 0 & trivial \\ |
---|
2260 | 1 & linear |
---|
2261 | \end{tabular} \hspace*{4ex} |
---|
2262 | \begin{tabular}{ll} |
---|
2263 | 2 & polynomial\\ |
---|
2264 | 3 & rational |
---|
2265 | \end{tabular} \hspace*{4ex} |
---|
2266 | \begin{tabular}{ll} |
---|
2267 | 4 & transcendental\\ |
---|
2268 | 5 & non-smooth |
---|
2269 | \end{tabular} |
---|
2270 | \end{center} |
---|
2271 | Here, ``trivial'' means that there is no dependence at all and ``linear'' means |
---|
2272 | that the partial derivative is a constant that |
---|
2273 | does not dependent on other variables either. ``Non-smooth'' means that one of |
---|
2274 | the functions on the path between $x_i$ and $y_j$ was evaluated at a point |
---|
2275 | where it is not differentiable. All positive labels |
---|
2276 | $1, 2, 3, 4, 5$ are pessimistic in that the actual functional relation may |
---|
2277 | in fact be simpler, for example due to exact cancellations. |
---|
2278 | |
---|
2279 | There are also overloaded versions providing a general {\sf reverse}-call. |
---|
2280 | Details of the appropriate calling sequences are given in the following \autoref{forw_rev}. |
---|
2281 | % |
---|
2282 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
2283 | \section{Overloaded Forward and Reverse Calls} |
---|
2284 | \label{forw_rev} |
---|
2285 | % |
---|
2286 | In this section, the several versions of the {\sf forward} and |
---|
2287 | {\sf reverse} routines, which utilize the overloading capabilities |
---|
2288 | of C++, are described in detail. With exception of the bit pattern |
---|
2289 | versions all interfaces are prototyped in the header file |
---|
2290 | \verb=<adolc/interfaces.h>=, where also some more specialized {\sf forward} |
---|
2291 | and {\sf reverse} routines are explained. Furthermore, \mbox{ADOL-C} provides |
---|
2292 | C and Fortran-callable versions prototyped in the same header file. |
---|
2293 | The bit pattern versions of {\sf forward} and {\sf reverse} introduced |
---|
2294 | in the \autoref{ProBit} are prototyped in the header file |
---|
2295 | \verb=<adolc/sparse/sparsedrivers.h>=, which will be included by the header |
---|
2296 | file \verb=<adolc/interfaces.h>= automatically. |
---|
2297 | % |
---|
2298 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
2299 | \subsection{The Scalar Case} |
---|
2300 | % |
---|
2301 | \label{ScaCas} |
---|
2302 | % |
---|
2303 | Given any correct tape, one may call from within |
---|
2304 | the generating program, or subsequently during another run, the following |
---|
2305 | procedure: |
---|
2306 | % |
---|
2307 | \begin{tabbing} |
---|
2308 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2309 | \>{\sf int forward(tag,m,n,d,keep,X,Y)} \\ |
---|
2310 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2311 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2312 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2313 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2314 | \>{\sf int keep;} \> // flag for reverse sweep \\ |
---|
2315 | \>{\sf double X[n][d+1];} \> // Taylor coefficients $X$ of |
---|
2316 | independent variables \\ |
---|
2317 | \>{\sf double Y[m][d+1];} \> // Taylor coefficients $Y$ as |
---|
2318 | in \eqref{eq:series} |
---|
2319 | \end{tabbing} |
---|
2320 | % |
---|
2321 | The rows of the matrix $X$ must correspond to the independent variables in the order of their |
---|
2322 | initialization by the \boldmath $\ll=$ \unboldmath operator. The columns of |
---|
2323 | $X = \{x_j\}_{j=0\ldots d}$ represent Taylor coefficient vectors as in |
---|
2324 | \eqref{eq:x_of_t}. The rows of the matrix $Y$ must |
---|
2325 | correspond to the |
---|
2326 | dependent variables in the order of their selection by the \boldmath $\gg=$ \unboldmath operator. |
---|
2327 | The columns of $Y = \{y_j\}_{j=0\ldots d}$ represent |
---|
2328 | Taylor coefficient vectors as in \eqref{eq:series}. |
---|
2329 | Thus the first column of $Y$ contains the |
---|
2330 | function value $F(x)$ itself, the next column represents the first |
---|
2331 | Taylor coefficient vector of $F$, and the last column the |
---|
2332 | $d$-th Taylor coefficient vector. The integer flag {\sf keep} determines |
---|
2333 | how many Taylor coefficients of all intermediate quantities are |
---|
2334 | written into the value stack as explained in \autoref{forw_rev_ad}. |
---|
2335 | If {\sf keep} is omitted, it defaults to 0. |
---|
2336 | |
---|
2337 | The given {\sf tag} value is used by {\sf forward} to determine the |
---|
2338 | name of the file on which the tape was written. If the tape file does |
---|
2339 | not exist, {\sf forward} assumes that the relevant |
---|
2340 | tape is still in core and reads from the buffers. |
---|
2341 | After the execution of an active section with \mbox{{\sf keep} = 1} or a call to |
---|
2342 | {\sf forward} with any {\sf keep} $\leq$ $d+1$, one may call |
---|
2343 | the function {\sf reverse} with \mbox{{\sf d} = {\sf keep} $-$ 1} and the same tape |
---|
2344 | identifier {\sf tag}. When $u$ is a vector |
---|
2345 | and $Z$ an $n\times (d+1)$ matrix |
---|
2346 | {\sf reverse} is executed in the scalar mode by the calling |
---|
2347 | sequence |
---|
2348 | % |
---|
2349 | \begin{tabbing} |
---|
2350 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2351 | \>{\sf int reverse(tag,m,n,d,u,Z)}\\ |
---|
2352 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2353 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2354 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2355 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2356 | \>{\sf double u[m];} \> // weighting vector $u$\\ |
---|
2357 | \>{\sf double Z[n][d+1];} \> // resulting adjoints $Z$ |
---|
2358 | \end{tabbing} |
---|
2359 | to compute |
---|
2360 | the adjoints $z_0^T=u^T F'(x_0)=u^T A_0$, $z_1^T=u^T F''(x_0)x_1=u^T A_1$, |
---|
2361 | \ldots, where $Z=[z_0,z_1,\ldots,z_d]$. |
---|
2362 | % |
---|
2363 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
2364 | \subsection{The Vector Case} |
---|
2365 | % |
---|
2366 | \label{vecCas} |
---|
2367 | % |
---|
2368 | When $U$ is a matrix {\sf reverse} is executed in the vector mode by the following calling sequence |
---|
2369 | % |
---|
2370 | \begin{tabbing} |
---|
2371 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2372 | \>{\sf int reverse(tag,m,n,d,q,U,Z,nz)}\\ |
---|
2373 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2374 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2375 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2376 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2377 | \>{\sf int q;} \> // number of weight vectors $q$\\ |
---|
2378 | \>{\sf double U[q][m];} \> // weight matrix $U$\\ |
---|
2379 | \>{\sf double Z[q][n][d+1];} \> // resulting adjoints \\ |
---|
2380 | \>{\sf short nz[q][n];} \> // nonzero pattern of {\sf Z} |
---|
2381 | \end{tabbing} |
---|
2382 | % |
---|
2383 | to compute the adjoints $Z_0=U F'(x_0)=U A_0$, $Z_1=U F''(x_0)x_1=U A_1$, |
---|
2384 | \ldots, where $Z=[Z_0,Z_1,\ldots,Z_d]$. |
---|
2385 | When the arguments {\sf p} and {\sf U} are omitted, they default to |
---|
2386 | $m$ and the identity matrix of order $m$, respectively. |
---|
2387 | |
---|
2388 | Through the optional argument {\sf nz} of {\sf reverse} one can compute |
---|
2389 | information about the sparsity pattern of $Z$ as described in detail |
---|
2390 | in the previous \autoref{forw_rev_ad}. |
---|
2391 | |
---|
2392 | The return values of {\sf reverse} calls can be interpreted according |
---|
2393 | to \autoref{retvalues}, but negative return values are not |
---|
2394 | valid, since the corresponding forward sweep would have |
---|
2395 | stopped without completing the necessary taylor file. |
---|
2396 | The return value of {\sf reverse} may be higher |
---|
2397 | than that of the preceding {\sf forward} call because some operations |
---|
2398 | that were evaluated at a critical argument during the forward sweep |
---|
2399 | were found not to impact the dependents during the reverse sweep. |
---|
2400 | |
---|
2401 | In both scalar and vector mode, the degree $d$ must agree with |
---|
2402 | {\sf keep}~$-$~1 for the most recent call to {\sf forward}, or it must be |
---|
2403 | equal to zero if {\sf reverse} directly follows the taping of an active |
---|
2404 | section. Otherwise, {\sf reverse} will return control with a suitable error |
---|
2405 | message. |
---|
2406 | In order to avoid possible confusion, the first four arguments must always be |
---|
2407 | present in the calling sequence. However, if $m$ or $d$ |
---|
2408 | attain their trivial values 1 and 0, respectively, then |
---|
2409 | corresponding dimensions of the arrays {\sf X}, {\sf Y}, {\sf u}, |
---|
2410 | {\sf U}, or {\sf Z} can be omitted, thus eliminating one level of |
---|
2411 | indirection. For example, we may call |
---|
2412 | {\sf reverse(tag,1,n,0,1.0,g)} after declaring |
---|
2413 | {\sf double g[n]} |
---|
2414 | to calculate a gradient of a scalar-valued function. |
---|
2415 | |
---|
2416 | Sometimes it may be useful to perform a forward sweep for families of |
---|
2417 | Taylor series with the same leading term. |
---|
2418 | This vector version of {\sf forward} can be called in the form |
---|
2419 | % |
---|
2420 | \begin{tabbing} |
---|
2421 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2422 | \>{\sf int forward(tag,m,n,d,p,x0,X,y0,Y)}\\ |
---|
2423 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2424 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2425 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2426 | \>{\sf int d;} \> // highest derivative degree $d$\\ |
---|
2427 | \>{\sf int p;} \> // number of Taylor series $p$\\ |
---|
2428 | \>{\sf double x0[n];} \> // values of independent variables $x_0$\\ |
---|
2429 | \>{\sf double X[n][p][d];} \> // Taylor coefficients $X$ of independent variables\\ |
---|
2430 | \>{\sf double y0[m];} \> // values of dependent variables $y_0$\\ |
---|
2431 | \>{\sf double Y[m][p][d];} \> // Taylor coefficients $Y$ of dependent variables |
---|
2432 | \end{tabbing} |
---|
2433 | % |
---|
2434 | where {\sf X} and {\sf Y} hold the Taylor coefficients of first |
---|
2435 | and higher degree and {\sf x0}, {\sf y0} the common Taylor coefficients of |
---|
2436 | degree 0. There is no option to keep the values of active variables |
---|
2437 | that are going out of scope or that are overwritten. Therefore this |
---|
2438 | function cannot prepare a subsequent reverse sweep. |
---|
2439 | The return integer serves as a flag to indicate quadratures or altered |
---|
2440 | comparisons as described above in \autoref{reuse_tape}. |
---|
2441 | |
---|
2442 | Since the calculation of Jacobians is probably the most important |
---|
2443 | automatic differentia\-tion task, we have provided a specialization |
---|
2444 | of vector {\sf forward} to the case where $d = 1$. This version can be |
---|
2445 | called in the form |
---|
2446 | % |
---|
2447 | \begin{tabbing} |
---|
2448 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2449 | \>{\sf int forward(tag,m,n,p,x,X,y,Y)}\\ |
---|
2450 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2451 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2452 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2453 | \>{\sf int p;} \> // number of partial derivatives $p$ \\ |
---|
2454 | \>{\sf double x[n];} \> // values of independent variables $x_0$\\ |
---|
2455 | \>{\sf double X[n][p];} \> // seed derivatives of independent variables $X$\\ |
---|
2456 | \>{\sf double y[m];} \> // values of dependent variables $y_0$\\ |
---|
2457 | \>{\sf double Y[m][p];} \> // first derivatives of dependent variables $Y$ |
---|
2458 | \end{tabbing} |
---|
2459 | % |
---|
2460 | When this routine is called with {\sf p} = {\sf n} and {\sf X} the identity matrix, |
---|
2461 | the resulting {\sf Y} is simply the Jacobian $F^\prime(x_0)$. In general, |
---|
2462 | one obtains the $m\times p$ matrix $Y=F^\prime(x_0)\,X $ for the |
---|
2463 | chosen initialization of $X$. In a workstation environment a value |
---|
2464 | of $p$ somewhere between $10$ and $50$ |
---|
2465 | appears to be fairly optimal. For smaller $p$ the interpretive |
---|
2466 | overhead is not appropriately amortized, and for larger $p$ the |
---|
2467 | $p$-fold increase in storage causes too many page faults. Therefore, |
---|
2468 | large Jacobians that cannot be compressed via column coloring |
---|
2469 | as could be done for example using the driver {\sf sparse\_jac} |
---|
2470 | should be ``strip-mined'' in the sense that the above |
---|
2471 | first-order-vector version of {\sf forward} is called |
---|
2472 | repeatedly with the successive \mbox{$n \times p$} matrices $X$ forming |
---|
2473 | a partition of the identity matrix of order $n$. |
---|
2474 | % |
---|
2475 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
2476 | \subsection{Dependence Analysis} |
---|
2477 | % |
---|
2478 | \label{ProBit} |
---|
2479 | % |
---|
2480 | The sparsity pattern of Jacobians is often needed to set up data structures |
---|
2481 | for their storage and factorization or to allow their economical evaluation |
---|
2482 | by compression \cite{BeKh96}. Compared to the evaluation of the full |
---|
2483 | Jacobian $F'(x_0)$ in real arithmetic computing the Boolean matrix |
---|
2484 | $\tilde{P}\in\left\{0,1\right\}^{m\times n}$ representing its sparsity |
---|
2485 | pattern in the obvious way requires a little less run-time and |
---|
2486 | certainly a lot less memory. |
---|
2487 | |
---|
2488 | The entry $\tilde{P}_{ji}$ in the $j$-th row and $i$-th column |
---|
2489 | of $\tilde{P}$ should be $1 = true$ exactly when there is a data |
---|
2490 | dependence between the $i$-th independent variable $x_{i}$ and |
---|
2491 | the $j$-th dependent variable $y_{j}$. Just like for real arguments |
---|
2492 | one would wish to compute matrix-vector and vector-matrix products |
---|
2493 | of the form $\tilde{P}\tilde{v}$ or $\tilde{u}^{T}\tilde{P}$ |
---|
2494 | by appropriate {\sf forward} and {\sf reverse} routines where |
---|
2495 | $\tilde{v}\in\{0,1\}^{n}$ and $\tilde{u}\in\{0,1\}^{m}$. |
---|
2496 | Here, multiplication corresponds to logical |
---|
2497 | {\sf AND} and addition to logical {\sf OR}, so that algebra is performed in a |
---|
2498 | semi-ring. |
---|
2499 | |
---|
2500 | For practical reasons it is assumed that |
---|
2501 | $s=8*${\sf sizeof}$(${\sf unsigned long int}$)$ such Boolean vectors |
---|
2502 | $\tilde{v}$ and $\tilde{u}$ are combined to integer vectors |
---|
2503 | $v\in\N^{n}$ and $u\in\N^{m}$ whose components can be interpreted |
---|
2504 | as bit patterns. Moreover $p$ or $q$ such integer vectors may |
---|
2505 | be combined column-wise or row-wise to integer matrices $X\in\N^{n \times p}$ |
---|
2506 | and $U\in\N^{q \times m}$, which naturally correspond |
---|
2507 | to Boolean matrices $\tilde{X}\in\{0,1\}^{n\times\left(sp\right)}$ |
---|
2508 | and $\tilde{U}\in\{0,1\}^{\left(sq\right)\times m}$. The provided |
---|
2509 | bit pattern versions of {\sf forward} and {\sf reverse} allow |
---|
2510 | to compute integer matrices $Y\in\N^{m \times p}$ and |
---|
2511 | $Z\in\N^{q \times m}$ corresponding to |
---|
2512 | \begin{equation} |
---|
2513 | \label{eq:int_forrev} |
---|
2514 | \tilde{Y} = \tilde{P}\tilde{X} \qquad \mbox{and} \qquad |
---|
2515 | \tilde{Z} = \tilde{U}\tilde{P} \, , |
---|
2516 | \end{equation} |
---|
2517 | respectively, with $\tilde{Y}\in\{0,1\}^{m\times\left(sp\right)}$ |
---|
2518 | and $\tilde{U}\in\{0,1\}^{\left(sq\right)\times n}$. |
---|
2519 | In general, the application of the bit pattern versions of |
---|
2520 | {\sf forward} or {\sf reverse} can be interpreted as |
---|
2521 | propagating dependences between variables forward or backward, therefore |
---|
2522 | both the propagated integer matrices and the corresponding |
---|
2523 | Boolean matrices are called {\em dependence structures}. |
---|
2524 | |
---|
2525 | The bit pattern {\sf forward} routine |
---|
2526 | % |
---|
2527 | \begin{tabbing} |
---|
2528 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2529 | \>{\sf int forward(tag,m,n,p,x,X,y,Y,mode)}\\ |
---|
2530 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2531 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2532 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2533 | \>{\sf int p;} \> // number of integers propagated $p$\\ |
---|
2534 | \>{\sf double x[n];} \> // values of independent variables $x_0$\\ |
---|
2535 | \>{\sf unsigned long int X[n][p];} \> // dependence structure $X$ \\ |
---|
2536 | \>{\sf double y[m];} \> // values of dependent variables $y_0$\\ |
---|
2537 | \>{\sf unsigned long int Y[m][p];} \> // dependence structure $Y$ according to |
---|
2538 | \eqref{eq:int_forrev}\\ |
---|
2539 | \>{\sf char mode;} \> // 0 : safe mode (default), 1 : tight mode |
---|
2540 | \end{tabbing} |
---|
2541 | % |
---|
2542 | can be used to obtain the dependence structure $Y$ for a given dependence structure |
---|
2543 | $X$. The dependence structures are |
---|
2544 | represented as arrays of {\sf unsigned long int} the entries of which are |
---|
2545 | interpreted as bit patterns as described above. |
---|
2546 | For example, for $n=3$ the identity matrix $I_3$ should be passed |
---|
2547 | with $p=1$ as the $3 \times 1$ array |
---|
2548 | \begin{eqnarray*} |
---|
2549 | {\sf X} \; = \; |
---|
2550 | \left( \begin{array}{r} |
---|
2551 | {\sf 1}0000000 \: 00000000 \: 00000000 \: 00000000_2 \\ |
---|
2552 | 0{\sf 1}000000 \: 00000000 \: 00000000 \: 00000000_2 \\ |
---|
2553 | 00{\sf 1}00000 \: 00000000 \: 00000000 \: 00000000_2 |
---|
2554 | \end{array} \right) |
---|
2555 | \end{eqnarray*} |
---|
2556 | in the 4-byte long integer format. The parameter {\sf mode} determines |
---|
2557 | the mode of dependence analysis as explained already in \autoref{sparse}. |
---|
2558 | |
---|
2559 | A call to the corresponding bit pattern {\sf reverse} routine |
---|
2560 | % |
---|
2561 | \begin{tabbing} |
---|
2562 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2563 | \>{\sf int reverse(tag,m,n,q,U,Z,mode)}\\ |
---|
2564 | \>{\sf short int tag;} \> // tape identification \\ |
---|
2565 | \>{\sf int m;} \> // number of dependent variables $m$\\ |
---|
2566 | \>{\sf int n;} \> // number of independent variables $n$\\ |
---|
2567 | \>{\sf int q;} \> // number of integers propagated q\\ |
---|
2568 | \>{\sf unsigned long int U[q][m];} \> // dependence structure $U$ \\ |
---|
2569 | \>{\sf unsigned long int Z[q][n];} \> // dependence structure $Z$ according |
---|
2570 | to \eqref{eq:int_forrev}\\ |
---|
2571 | \>{\sf char mode;} \> // 0 : safe mode (default), 1 : tight mode |
---|
2572 | \end{tabbing} |
---|
2573 | % |
---|
2574 | yields the dependence structure $Z$ for a given dependence structure |
---|
2575 | $U$. |
---|
2576 | |
---|
2577 | To determine the whole sparsity pattern $\tilde{P}$ of the Jacobian $F'(x)$ |
---|
2578 | as an integer matrix $P$ one may call {\sf forward} or {\sf reverse} |
---|
2579 | with $p \ge n/s$ or $q \ge m/s$, respectively. For this purpose the |
---|
2580 | corresponding dependence structure $X$ or $U$ must be defined to represent |
---|
2581 | the identity matrix of the respective dimension. |
---|
2582 | Due to the fact that always a multiple of $s$ Boolean vectors are propagated |
---|
2583 | there may be superfluous vectors, which can be set to zero. |
---|
2584 | |
---|
2585 | The return values of the bit pattern {\sf forward} and {\sf reverse} routines |
---|
2586 | correspond to those described in \autoref{retvalues}. |
---|
2587 | |
---|
2588 | One can control the storage growth by the factor $p$ using |
---|
2589 | ``strip-mining'' for the calls of {\sf forward} or {\sf reverse} with successive |
---|
2590 | groups of columns or respectively rows at a time, i.e.~partitioning |
---|
2591 | $X$ or $U$ appropriately as described for the computation of Jacobians |
---|
2592 | in \autoref{vecCas}. |
---|
2593 | % |
---|
2594 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
2595 | % |
---|
2596 | \section{Advance algorithmic differentiation in ADOL-C} |
---|
2597 | \label{adv_ad} |
---|
2598 | % |
---|
2599 | \subsection{External differentiated functions} |
---|
2600 | % |
---|
2601 | Ideally, AD is applied to a given function as a whole. |
---|
2602 | In practice, however, sophisticated projects usually evolve over a long period of time. |
---|
2603 | Within this process, a heterogeneous code base for the project |
---|
2604 | develops, which may include the incorporation of external solutions, |
---|
2605 | changes in programming paradigms or even of programming languages. |
---|
2606 | Equally heterogeneous, the computation of derivative values appears. |
---|
2607 | Hence, different \mbox{AD-tools} may be combined with hand-derived |
---|
2608 | codes based on the same or different programming languages. |
---|
2609 | ADOL-C support such settings by the concept of external |
---|
2610 | differentiated functions. Hence, a external differentiated function |
---|
2611 | itself is not differentiated by ADOL-C. The required derivative |
---|
2612 | information have to be provided by the user. |
---|
2613 | |
---|
2614 | For this purpose, it is assumed that the external differentiated |
---|
2615 | function has the signature |
---|
2616 | \smallskip |
---|
2617 | |
---|
2618 | \noindent |
---|
2619 | \hspace*{2cm}{\sf int ext\_func(int n, double *yin, int m, double *yout);} |
---|
2620 | \medskip |
---|
2621 | |
---|
2622 | \noindent |
---|
2623 | where the function names can be chosen by the user as long as the names are |
---|
2624 | unique. This {\sf double} version of the external differentiated function has to |
---|
2625 | be {\em registered} using the \mbox{ADOL-C} function |
---|
2626 | \smallskip |
---|
2627 | |
---|
2628 | \noindent |
---|
2629 | \hspace*{2cm}{\sf edf = reg\_ext\_fct(ext\_func);}. |
---|
2630 | \smallskip |
---|
2631 | |
---|
2632 | \noindent |
---|
2633 | This function initializes the structure {\sf edf}. Then, |
---|
2634 | the user has to provide the remaining information |
---|
2635 | by the following commands: |
---|
2636 | \begin{tabbing} |
---|
2637 | \hspace*{2cm}\= {\sf edf-$>$zos\_forward = zos\_for\_ext\_func;}\\ |
---|
2638 | \> {\sf // function pointer for computing |
---|
2639 | Zero-Order-Scalar (=zos)}\\ |
---|
2640 | \> {\sf // forward information}\\ |
---|
2641 | \> {\sf edf-$>$dp\_x = xp;}\\ |
---|
2642 | \> {\sf edf-$>$dp\_y = yp;}\\ |
---|
2643 | \> {\sf // double arrays for arguments and results}\\ |
---|
2644 | \> {\sf edf-$>$fos\_reverse = fos\_rev\_ext\_func;} \\ |
---|
2645 | \> {\sf // function pointer for computing |
---|
2646 | First-Order-Scalar (=fos)}\\ |
---|
2647 | \> {\sf reverse information} |
---|
2648 | \end{tabbing} |
---|
2649 | Subsequently, the call to the external differentiated function in the function evaluation can be |
---|
2650 | substituted by the call of |
---|
2651 | \medskip |
---|
2652 | |
---|
2653 | \noindent |
---|
2654 | \hspace*{2cm}{\sf int call\_ext\_fct(edf, n, xp, x, m, yp, y);} |
---|
2655 | \medskip |
---|
2656 | |
---|
2657 | \noindent |
---|
2658 | The usage of the external function facility is illustrated by the |
---|
2659 | example \verb=ext_diff_func= contained in |
---|
2660 | \verb=examples/additional_examples/ext_diff_func=. |
---|
2661 | Here,the external differentiated function is also a C code, but the |
---|
2662 | handling as external differentiated functions also a decrease of the |
---|
2663 | overall required tape size. |
---|
2664 | |
---|
2665 | % |
---|
2666 | \subsection{Advance algorithmic differentiation of time integration processes} |
---|
2667 | % |
---|
2668 | For many time-dependent applications, the corresponding simulations |
---|
2669 | are based on ordinary or partial differential equations. |
---|
2670 | Furthermore, frequently there are quantities that influence the |
---|
2671 | result of the simulation and can be seen as control of the systems. |
---|
2672 | To compute an approximation of the |
---|
2673 | simulated process for a time interval $[0,T]$ and evaluated the |
---|
2674 | desired target function, one applies an |
---|
2675 | appropriate integration scheme given by |
---|
2676 | \begin{tabbing} |
---|
2677 | \hspace{5mm} \= some initializations yielding $x_0$\\ |
---|
2678 | \> for $i=0,\ldots, N-1$\\ |
---|
2679 | \hspace{10mm}\= $x_{i+1} = F(x_i,u_i,t_i)$\\ |
---|
2680 | \hspace{5mm} \= evaluation of the target function |
---|
2681 | \end{tabbing} |
---|
2682 | where $x_i\in {\bf R}^n$ denotes the state and $u_i\in {\bf R}^m$ the control at |
---|
2683 | time $t_i$ for a given time grid $t_0,\ldots,t_N$ with $t_0=0$ and |
---|
2684 | $t_N=T$. The operator $F : {\bf R}^n \times {\bf R}^m \times {\bf R} \mapsto {\bf R}^n$ |
---|
2685 | defines the time step to compute the state at time $t_i$. Note that we |
---|
2686 | do not assume a uniform grid. |
---|
2687 | |
---|
2688 | When computing derivatives of the target function with respect to the |
---|
2689 | control, the consequences for the tape generation using the ``basic'' |
---|
2690 | taping approach as implemented in ADOL-C so far are shown in the left part of |
---|
2691 | \autoref{fig:bas_tap}. |
---|
2692 | \begin{figure}[htbp] |
---|
2693 | \begin{center} |
---|
2694 | \hspace*{0.5cm}\includegraphics[width=5.8cm]{tapebasic}\hfill |
---|
2695 | \includegraphics[width=5.8cm]{tapeadv} \hspace*{0.5cm}\ |
---|
2696 | \end{center} |
---|
2697 | \hspace*{0.8cm} Basic taping process \hspace*{4.3cm} Advanced taping process |
---|
2698 | \caption{Different taping approaches} |
---|
2699 | \label{fig:bas_tap} |
---|
2700 | \end{figure} |
---|
2701 | As can be seen, the iterative process is completely |
---|
2702 | unrolled due to the taping process. That is, the tape contains an internal representation of each |
---|
2703 | time step. Hence, the overall tape comprises a serious amount of redundant |
---|
2704 | information as illustrated by the light grey rectangles in |
---|
2705 | \autoref{fig:bas_tap}. |
---|
2706 | |
---|
2707 | To overcome the repeated storage of essentially the same information, |
---|
2708 | a {\em nested taping} mechanism has been incorporated into ADOL-C as illustrated on |
---|
2709 | the right-hand side of \autoref{fig:bas_tap}. This new |
---|
2710 | capability allows the encapsulation of the time-stepping procedure |
---|
2711 | such that only the last time step $x_{N} = F(x_{N-1},u_{N-1})$ is taped as one |
---|
2712 | representative of the time steps in addition to a function pointer to the |
---|
2713 | evaluation procedure $F$ of the time steps. The function pointer has |
---|
2714 | to be stored for a possibly necessary retaping during the derivative calculation |
---|
2715 | as explained below. |
---|
2716 | |
---|
2717 | Instead of storing the complete tape, only a very limited number of intermediate |
---|
2718 | states are kept in memory. They serve as checkpoints, such that |
---|
2719 | the required information for the backward integration is generated |
---|
2720 | piecewise during the adjoint calculation. |
---|
2721 | For this modified adjoint computation the optimal checkpointing schedules |
---|
2722 | provided by {\bf revolve} are employed. An adapted version of the |
---|
2723 | software package {\sf revolve} is part of ADOL-C and automatically |
---|
2724 | integrated in the ADOL-C library. Based on {\sf revolve}, $c$ checkpoints are |
---|
2725 | distributed such that computational effort is minimized for the given |
---|
2726 | number of checkpoints and time steps $N$. It is important to note that the overall tape |
---|
2727 | size is drastically reduced due to the advanced taping strategy. For the |
---|
2728 | implementation of this nested taping we introduced |
---|
2729 | a so-called ``differentiating context'' that enables \mbox{ADOL-C} to |
---|
2730 | handle different internal function representations during the taping |
---|
2731 | procedure and the derivative calculation. This approach allows the generation of a new |
---|
2732 | tape inside the overall tape, where the coupling of the different tapes is based on |
---|
2733 | the {\em external differentiated function} described above. |
---|
2734 | |
---|
2735 | Written under the objective of minimal user effort, the checkpointing routines |
---|
2736 | of \mbox{ADOL-C} need only very limited information. The user must |
---|
2737 | provide two routines as implementation of the time-stepping function $F$ |
---|
2738 | with the signatures |
---|
2739 | \medskip |
---|
2740 | |
---|
2741 | \noindent |
---|
2742 | \hspace*{2cm}{\sf int time\_step\_function(int n, adouble *u);}\\ |
---|
2743 | \hspace*{2cm}{\sf int time\_step\_function(int n, double *u);} |
---|
2744 | \medskip |
---|
2745 | |
---|
2746 | \noindent |
---|
2747 | where the function names can be chosen by the user as long as the names are |
---|
2748 | unique.It is possible that the result vector of one time step |
---|
2749 | iteration overwrites the argument vector of the same time step. Then, no |
---|
2750 | copy operations are required to prepare the next time step. |
---|
2751 | |
---|
2752 | At first, the {\sf adouble} version of the time step function has to |
---|
2753 | be {\em registered} using the \mbox{ADOL-C} function |
---|
2754 | \medskip |
---|
2755 | |
---|
2756 | \noindent |
---|
2757 | \hspace*{2cm}{\sf CP\_Context cpc(time\_step\_function);}. |
---|
2758 | \medskip |
---|
2759 | |
---|
2760 | \noindent |
---|
2761 | This function initializes the structure {\sf cpc}. Then, |
---|
2762 | the user has to provide the remaining checkpointing information |
---|
2763 | by the following commands: |
---|
2764 | \begin{tabbing} |
---|
2765 | \hspace*{2cm}\= {\sf cpc.setDoubleFct(time\_step\_function);}\\ |
---|
2766 | \> {\sf // double variante of the time step function}\\ |
---|
2767 | \> {\sf cpc.setNumberOfSteps(N);}\\ |
---|
2768 | \> {\sf // number of time steps to perform}\\ |
---|
2769 | \> {\sf cpc.setNumberOfCheckpoints(10);}\\ |
---|
2770 | \> {\sf // number of checkpoint} \\ |
---|
2771 | \> {\sf cpc.setDimensionXY(n);}\\ |
---|
2772 | \> {\sf // dimension of input/output}\\ |
---|
2773 | \> {\sf cpc.setInput(y);}\\ |
---|
2774 | \> {\sf // input vector} \\ |
---|
2775 | \> {\sf cpc.setOutput(y);}\\ |
---|
2776 | \> {\sf // output vector }\\ |
---|
2777 | \> {\sf cpc.setTapeNumber(tag\_check);}\\ |
---|
2778 | \> {\sf // subtape number for checkpointing} \\ |
---|
2779 | \> {\sf cpc.setAlwaysRetaping(false);}\\ |
---|
2780 | \> {\sf // always retape or not ?} |
---|
2781 | \end{tabbing} |
---|
2782 | Subsequently, the time loop in the function evaluation can be |
---|
2783 | substituted by a call of the function |
---|
2784 | \medskip |
---|
2785 | |
---|
2786 | \noindent |
---|
2787 | \hspace*{2cm}{\sf int cpc.checkpointing();} |
---|
2788 | \medskip |
---|
2789 | |
---|
2790 | \noindent |
---|
2791 | Then, ADOL-C computes derivative information using the optimal checkpointing |
---|
2792 | strategy provided by {\sf revolve} internally, i.e., completely hidden from the user. |
---|
2793 | |
---|
2794 | The presented driver is prototyped in the header file |
---|
2795 | \verb=<adolc/checkpointing.h>=. This header |
---|
2796 | is included by the global header file \verb=<adolc/adolc.h>= automatically. |
---|
2797 | An example program \verb=checkpointing.cpp= illustrates the |
---|
2798 | checkpointing facilities. It can be found in the directory \verb=examples/additional_examples/checkpointing=. |
---|
2799 | % |
---|
2800 | % |
---|
2801 | % |
---|
2802 | \subsection{Advance algorithmic differentiation of fixed point iterations} |
---|
2803 | % |
---|
2804 | Quite often, the state of the considered system denoted by $x\in\R^n$ |
---|
2805 | depends on some design parameters denoted by $u\in\R^m$. One example for this setting |
---|
2806 | forms the flow over an aircraft wing. Here, the shape of the wing that |
---|
2807 | is defined by the design vector $u$ |
---|
2808 | determines the flow field $x$. The desired quasi-steady state $x_*$ |
---|
2809 | fulfills the fixed point equation |
---|
2810 | \begin{align} |
---|
2811 | \label{eq:fixedpoint} |
---|
2812 | x_* = F(x_*,u) |
---|
2813 | \end{align} |
---|
2814 | for a given continuously differentiable function |
---|
2815 | $F:\R^n\times\R^m\rightarrow\R^n$. A fixed point property of this kind is |
---|
2816 | also exploited by many other applications. |
---|
2817 | |
---|
2818 | Assume that one can apply the iteration |
---|
2819 | \begin{align} |
---|
2820 | \label{eq:iteration} |
---|
2821 | x_{k+1} = F(x_k,u) |
---|
2822 | \end{align} |
---|
2823 | to obtain a linear converging sequence $\{x_k\}$ generated |
---|
2824 | for any given control $u\in\R^n$. Then the limit point $x_*\in\R^n$ fulfils the fixed |
---|
2825 | point equation~\eqref{eq:fixedpoint}. Moreover, |
---|
2826 | suppose that $\|\frac{dF}{dx}(x_*,u)\|<1$ holds for any pair |
---|
2827 | $(x_*,u)$ satisfying equation \eqref{eq:fixedpoint}. |
---|
2828 | Hence, there exists a |
---|
2829 | differentiable function $\phi:\R^m \rightarrow \R^n$, |
---|
2830 | such that $\phi(u) = F(\phi(u),u)$, where the state |
---|
2831 | $\phi(u)$ is a fixed point of $F$ according to a control |
---|
2832 | $u$. To optimize the system described by the state vector $x=\phi(u)$ with respect to |
---|
2833 | the design vector $u$, derivatives of $\phi$ with respect |
---|
2834 | to $u$ are of particular interest. |
---|
2835 | |
---|
2836 | To exploit the advanced algorithmic differentiation of such fixed point iterations |
---|
2837 | ADOL-C provides the special functions {\tt fp\_iteration(...)}. |
---|
2838 | It has the following interface: |
---|
2839 | \begin{tabbing} |
---|
2840 | \hspace{0.5in}\={\sf short int tag;} \hspace{1.1in}\= \kill % define tab position |
---|
2841 | \>{\sf int |
---|
2842 | fp\_iteration(}\={\sf sub\_tape\_num,double\_F,adouble\_F,norm,norm\_deriv,eps,eps\_deriv,}\\ |
---|
2843 | \> \>{\sf N\_max,N\_max\_deriv,x\_0,u,x\_fix,dim\_x,dim\_u)}\\ |
---|
2844 | \hspace{0.5in}\={\sf short int tag;} \hspace{0.9in}\= \kill % define tab position |
---|
2845 | \>{\sf short int sub\_tape\_num;} \> // tape identification for sub\_tape \\ |
---|
2846 | \>{\sf int *double\_F;} \> // pointer to a function that compute for $x$ and $u$ \\ |
---|
2847 | \> \> // the value $y=F(x,u)$ for {\sf double} arguments\\ |
---|
2848 | \>{\sf int *adouble\_F;} \> // pointer to a function that compute for $x$ and $u$ \\ |
---|
2849 | \> \> // the value $y=F(x,u)$ for {\sf double} arguments\\ |
---|
2850 | \>{\sf int *norm;} \> // pointer to a function that computes\\ |
---|
2851 | \> \> // the norm of a vector\\ |
---|
2852 | \>{\sf int *norm\_deriv;} \> // pointer to a function that computes\\ |
---|
2853 | \> \> // the norm of a vector\\ |
---|
2854 | \>{\sf double eps;} \> // termination criterion for fixed point iteration\\ |
---|
2855 | \>{\sf double eps\_deriv;} \> // termination criterion for adjoint fixed point iteration\\ |
---|
2856 | \>{\sf N\_max;} \> // maximal number of itertions for state computation\\ |
---|
2857 | \>{\sf N\_max\_deriv;} \> // maximal number of itertions for adjoint computation\\ |
---|
2858 | \>{\sf adouble *x\_0;} \> // inital state of fixed point iteration\\ |
---|
2859 | \>{\sf adouble *u;} \> // value of $u$\\ |
---|
2860 | \>{\sf adouble *x\_fic;} \> // final state of fixed point iteration\\ |
---|
2861 | \>{\sf int dim\_x;} \> // dimension of $x$\\ |
---|
2862 | \>{\sf int dim\_u;} \> // dimension of $u$\\ |
---|
2863 | \end{tabbing} |
---|
2864 | % |
---|
2865 | Here {\tt sub\_tape\_num} is an ADOL-C identifier for the subtape that |
---|
2866 | should be used for the fixed point iteration. |
---|
2867 | {\tt double\_F} and {\tt adouble\_F} are pointers to functions, that |
---|
2868 | compute for $x$ and $u$ a single iteration step $y=F(x,u)$. Thereby |
---|
2869 | {\tt double\_F} uses {\tt double} arguments and {\tt adouble\_F} |
---|
2870 | uses ADOL-C {\tt adouble} arguments. The parameters {\tt norm} and |
---|
2871 | {\tt norm\_deriv} are pointers to functions computing the norm |
---|
2872 | of a vector. The latter functions together with {\tt eps}, |
---|
2873 | {\tt eps\_deriv}, {\tt N\_max}, and {\tt N\_max\_deriv} control |
---|
2874 | the iterations. Thus the following loops are performed: |
---|
2875 | \begin{center} |
---|
2876 | \begin{tabular}{ll} |
---|
2877 | do & do \\ |
---|
2878 | ~~~~$k = k+1$ & ~~~~$k = k+1$ \\ |
---|
2879 | ~~~~$x = y$ & ~~~~$\zeta = \xi$ \\ |
---|
2880 | ~~~~$y = F(x,u)$ & ~~~ |
---|
2881 | $(\xi^T,\bar u^T) = \zeta^TF'(x_*,u) + (\bar x^T, 0^T)$ \\ |
---|
2882 | while $\|y-x\|\geq\varepsilon$ and $k\leq N_{max}$ \hspace*{0.5cm} & |
---|
2883 | while $\|\xi -\zeta\|_{deriv}\geq\varepsilon_{deriv}$ \\ |
---|
2884 | & and $k\leq N_{max,deriv}$ |
---|
2885 | \end{tabular} |
---|
2886 | \end{center} |
---|
2887 | The vector for the initial iterate and the control is stored |
---|
2888 | in {\tt x\_0} and {\tt u} respectively. The vector in which the |
---|
2889 | fixed point is stored is {\tt x\_fix}. Finally {\tt dim\_x} |
---|
2890 | and {\tt dim\_u} represent the dimensions $n$ and $m$ of the |
---|
2891 | corresponding vectors. |
---|
2892 | |
---|
2893 | The presented driver is prototyped in the header file |
---|
2894 | \verb=<adolc/fixpoint.h>=. This header |
---|
2895 | is included by the global header file \verb=<adolc/adolc.h>= automatically. |
---|
2896 | An example code that shows also the |
---|
2897 | expected signature of the function pointers is contained in the directory \verb=examples/additional_examples/fixpoint_exam=. |
---|
2898 | % |
---|
2899 | \subsection{Advance algorithmic differentiation of OpenMP parallel programs} |
---|
2900 | % |
---|
2901 | ADOL-C allows to compute derivatives in parallel for functions |
---|
2902 | containing OpenMP parallel loops. |
---|
2903 | This implies that an explicit loop-handling approach is applied. A |
---|
2904 | typical situation is shown in \autoref{fig:basic_layout}, |
---|
2905 | \begin{figure}[hbt] |
---|
2906 | \vspace{3ex} |
---|
2907 | \begin{center} |
---|
2908 | \includegraphics[height=4cm]{multiplexed} \\ |
---|
2909 | \begin{picture}(0,0) |
---|
2910 | \put(-48,40){\vdots} |
---|
2911 | \put(48,40){\vdots} |
---|
2912 | \put(-48,80){\vdots} |
---|
2913 | \put(48,80){\vdots} |
---|
2914 | \put(-83,132){function eval.} |
---|
2915 | \put(5,132){derivative calcul.} |
---|
2916 | \end{picture} |
---|
2917 | \end{center} |
---|
2918 | \vspace{-5ex} |
---|
2919 | \caption{Basic layout of mixed function and the corresponding derivation process} |
---|
2920 | \label{fig:basic_layout} |
---|
2921 | \end{figure} |
---|
2922 | where the OpenMP-parallel loop is preceded by a serial startup |
---|
2923 | calculation and followed by a serial finalization phase. |
---|
2924 | |
---|
2925 | Initialization of the OpenMP-parallel regions for \mbox{ADOL-C} is only a matter of adding a macro to the outermost OpenMP statement. |
---|
2926 | Two macros are available that only differ in the way the global tape information is handled. |
---|
2927 | Using {\tt ADOLC\_OPENMP}, this information, including the values of the augmented variables, is always transferred from the serial to the parallel region using {\it firstprivate} directives for initialization. |
---|
2928 | For the special case of iterative codes where parallel regions, working on the same data structures, are called repeatedly the {\tt ADOLC\_OPENMP\_NC} macro can be used. |
---|
2929 | Then, the information transfer is performed only once within the iterative process upon encounter of the first parallel region through use of the {\it threadprivate} feature of OpenMP that makes use of thread-local storage, i.e., global memory local to a thread. |
---|
2930 | Due to the inserted macro, the OpenMP statement has the following structure: |
---|
2931 | \begin{tabbing} |
---|
2932 | \hspace*{1cm} \= {\sf \#pragma omp ... ADOLC\_OPENMP} \qquad \qquad or \\ |
---|
2933 | \> {\sf \#pragma omp ... ADOLC\_OPENMP\_NC} |
---|
2934 | \end{tabbing} |
---|
2935 | Inside the parallel region, separate tapes may then be created. |
---|
2936 | Each single thread works in its own dedicated AD-environment, and all |
---|
2937 | serial facilities of \mbox{ADOL-C} are applicable as usual. The global |
---|
2938 | derivatives can be computed using the tapes created in the serial and |
---|
2939 | parallel parts of the function evaluation, where user interaction is |
---|
2940 | required for the correct derivative concatenation of the various tapes. |
---|
2941 | |
---|
2942 | For the usage of the parallel facilities, the \verb=configure=-command |
---|
2943 | has to be used with the option \verb?--with-openmp-flag=FLAG?, where |
---|
2944 | \verb=FLAG= stands for the system dependent OpenMP flag. |
---|
2945 | The parallel differentiation of a parallel program is illustrated |
---|
2946 | by the example program \verb=openmp_exam.cpp= contained in \verb=examples/additional_examples/openmp_exam=. |
---|
2947 | % |
---|
2948 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
2949 | % |
---|
2950 | \section{Tapeless forward differentiation in ADOL-C} |
---|
2951 | \label{tapeless} |
---|
2952 | % |
---|
2953 | Up to version 1.9.0, the development of the ADOL-C software package |
---|
2954 | was based on the decision to store all data necessary for derivative |
---|
2955 | computation on tapes, where large applications require the tapes to be |
---|
2956 | written out to corresponding files. In almost all cases this means |
---|
2957 | a considerable drawback in terms of run time due to the excessive |
---|
2958 | memory accesses. Using these tapes enables ADOL-C to offer multiple |
---|
2959 | functions. However, it is not necessary for all tasks of derivative |
---|
2960 | computation to do that. |
---|
2961 | |
---|
2962 | Starting with version 1.10.0, ADOL-C now features a tapeless forward |
---|
2963 | mode for computing first order derivatives in scalar mode, i.e., |
---|
2964 | $\dot{y} = F'(x)\dot{x}$, and in vector mode, i.e., $\dot{Y} = F'(x)\dot{X}$. |
---|
2965 | This tapeless variant coexists with the more universal |
---|
2966 | tape based mode in the package. The following subsections describe |
---|
2967 | the source code modifications required to use the tapeless forward mode of |
---|
2968 | ADOL-C. |
---|
2969 | % |
---|
2970 | \subsection{Modifying the Source Code} |
---|
2971 | % |
---|
2972 | Let us consider the coordinate transformation from Cartesian to spherical |
---|
2973 | polar coordinates given by the function $F: \mathbb{R}^3 \to \mathbb{R}^3$, $y |
---|
2974 | = F(x)$, with |
---|
2975 | \begin{eqnarray*} |
---|
2976 | y_1 = \sqrt{x_1^2 + x_2^2 + x_3^2},\qquad |
---|
2977 | y_2 = \arctan\left(\sqrt{x_1^2 + x_2^2}/x_3\right),\qquad |
---|
2978 | y_3 = \arctan(x_2/x_1), |
---|
2979 | \end{eqnarray*} |
---|
2980 | as an example. The corresponding source code is shown in \autoref{fig:tapeless}. |
---|
2981 | \begin{figure}[htb] |
---|
2982 | \framebox[\textwidth]{\parbox{\textwidth}{ |
---|
2983 | %\begin{center} |
---|
2984 | %\begin{flushleft} |
---|
2985 | \begin{tabbing} |
---|
2986 | \= \kill |
---|
2987 | \> {\sf \#include} {\sf $<$iostream$>$}\\ |
---|
2988 | \> {\sf using namespace std;}\\ |
---|
2989 | \> \\ |
---|
2990 | \> {\sf int main() \{}\\ |
---|
2991 | \> {\sf \rule{0.5cm}{0pt}double x[3], y[3];}\\ |
---|
2992 | \> \\ |
---|
2993 | \> {\sf \rule{0.5cm}{0pt}for (int i=0; i$<$3; ++i)\hspace*{3cm}// Initialize $x_i$}\\ |
---|
2994 | \> {\sf \rule{1cm}{0pt}...}\\ |
---|
2995 | \> \\ |
---|
2996 | \> {\sf \rule{0.5cm}{0pt}y[0] = sqrt(x[0]*x[0]+x[1]*x[1]+x[2]*x[2]);}\\ |
---|
2997 | \> {\sf \rule{0.5cm}{0pt}y[1] = atan(sqrt(x[0]*x[0]+x[1]*x[1])/x[2]);}\\ |
---|
2998 | \> {\sf \rule{0.5cm}{0pt}y[2] = atan(x[1]/x[0]);}\\ |
---|
2999 | \> \\ |
---|
3000 | \> {\sf \rule{0.5cm}{0pt}cout $<<$ "y1=" $<<$ y[0] $<<$ " , y2=" $<<$ y[1] $<<$ " , y3=" $<<$ y[2] $<<$ endl;}\\ |
---|
3001 | \> \\ |
---|
3002 | \> {\sf \rule{0.5cm}{0pt}return 0;}\\ |
---|
3003 | \> \} |
---|
3004 | \end{tabbing} |
---|
3005 | %\end{flushleft} |
---|
3006 | %\end{center} |
---|
3007 | }} |
---|
3008 | \caption{Example for tapeless forward mode} |
---|
3009 | \label{fig:tapeless} |
---|
3010 | \end{figure} |
---|
3011 | % |
---|
3012 | Changes to the source code that are necessary for applying the |
---|
3013 | tapeless forward ADOL-C are described in the following two |
---|
3014 | subsections, where the vector mode version is described |
---|
3015 | as extension of the scalar mode. |
---|
3016 | % |
---|
3017 | \subsubsection*{The scalar mode} |
---|
3018 | % |
---|
3019 | To use the tapeless forward mode, one has to include one |
---|
3020 | of the header files \verb#adolc.h# or \verb#adouble.h# |
---|
3021 | where the latter should be preferred since it does not include the |
---|
3022 | tape based functions defined in other header files. Hence, including |
---|
3023 | \verb#adouble.h# avoids mode mixtures, since |
---|
3024 | \verb#adolc.h# is just a wrapper for including all public |
---|
3025 | headers of the ADOL-C package and does not offer own functions. |
---|
3026 | Since the two ADOL-C forward mode variants tape-based and tapeless, |
---|
3027 | are prototyped in the same header file, the compiler needs to know if a |
---|
3028 | tapeless version is intended. This can be done by defining a |
---|
3029 | preprocessor macro named {\sf ADOLC\_TAPELESS}. Note that it is |
---|
3030 | important to define this macro before the header file is included. |
---|
3031 | Otherwise, the tape-based version of ADOL-C will be used. |
---|
3032 | |
---|
3033 | As in the tape based forward version of ADOL-C all derivative |
---|
3034 | calculations are introduced by calls to overloaded |
---|
3035 | operators. Therefore, similar to the tape-based version all |
---|
3036 | independent, intermediate and dependent variables must be declared |
---|
3037 | with type {\sf adouble}. The whole tapeless functionality provided by |
---|
3038 | \verb#adolc.h# was written as complete inline intended code |
---|
3039 | due to run time aspects, where the real portion of inlined code can |
---|
3040 | be influenced by switches for many compilers. Likely, the whole |
---|
3041 | derivative code is inlined by default. Our experiments |
---|
3042 | with the tapeless mode have produced complete inlined code by using |
---|
3043 | standard switches (optimization) for GNU and Intel C++ |
---|
3044 | compiler. |
---|
3045 | |
---|
3046 | To avoid name conflicts |
---|
3047 | resulting from the inlining the tapeless version has its own namespace |
---|
3048 | \verb#adtl#. As a result four possibilities of using the {\sf adouble} |
---|
3049 | type are available for the tapeless version: |
---|
3050 | \begin{itemize} |
---|
3051 | \item Defining a new type |
---|
3052 | \begin{center} |
---|
3053 | \begin{tabular}{l} |
---|
3054 | {\sf typedef adtl::adouble adouble;}\\ |
---|
3055 | ...\\ |
---|
3056 | {\sf adouble tmp;} |
---|
3057 | \end{tabular} |
---|
3058 | \end{center} |
---|
3059 | This is the preferred way. Remember, you can not write an own |
---|
3060 | {\sf adouble} type/class with different meaning after doing the typedef. |
---|
3061 | \item Declaring with namespace prefix |
---|
3062 | \begin{center} |
---|
3063 | \begin{tabular}{l} |
---|
3064 | {\sf adtl::adouble tmp;} |
---|
3065 | \end{tabular} |
---|
3066 | \end{center} |
---|
3067 | Not the most handsome and efficient way with respect to coding |
---|
3068 | but without any doubt one of the safest ways. The identifier |
---|
3069 | {\sf adouble} is still available for user types/classes. |
---|
3070 | \item Trusting macros |
---|
3071 | \begin{center} |
---|
3072 | \begin{tabular}{l} |
---|
3073 | {\sf \#define adouble adtl::adouble}\\ |
---|
3074 | ...\\ |
---|
3075 | {\sf adouble tmp;} |
---|
3076 | \end{tabular} |
---|
3077 | \end{center} |
---|
3078 | This approach should be used with care, since standard defines are text replacements. |
---|
3079 | \item Using the complete namespace |
---|
3080 | \begin{center} |
---|
3081 | \begin{tabular}{l} |
---|
3082 | {\sf \#using namespace adtl;}\\ |
---|
3083 | ...\\ |
---|
3084 | {\sf adouble tmp;} |
---|
3085 | \end{tabular} |
---|
3086 | \end{center} |
---|
3087 | A very clear approach with the disadvantage of uncovering all the hidden secrets. Name conflicts may arise! |
---|
3088 | \end{itemize} |
---|
3089 | After defining the variables only two things are left to do. First |
---|
3090 | one needs to initialize the values of the independent variables for the |
---|
3091 | function evaluation. This can be done by assigning the variables a {\sf |
---|
3092 | double} value. The {\sf ad}-value is set to zero in this case. |
---|
3093 | Additionally, the tapeless forward mode variant of ADOL-C |
---|
3094 | offers a function named {\sf setValue} for setting the value without |
---|
3095 | changing the {\sf ad}-value. To set the {\sf ad}-values of the independent |
---|
3096 | variables ADOL-C offers two possibilities: |
---|
3097 | \begin{itemize} |
---|
3098 | \item Using the constructor |
---|
3099 | \begin{center} |
---|
3100 | \begin{tabular}{l} |
---|
3101 | {\sf adouble x1(2,1), x2(4,0), y;} |
---|
3102 | \end{tabular} |
---|
3103 | \end{center} |
---|
3104 | This would create three adoubles $x_1$, $x_2$ and $y$. Obviously, the latter |
---|
3105 | remains uninitialized. In terms of function evaluation |
---|
3106 | $x_1$ holds the value 2 and $x_2$ the value 4 whereas the derivative values |
---|
3107 | are initialized to $\dot{x}_1=1$ and $\dot{x}_2=0$. |
---|
3108 | \item Setting point values directly |
---|
3109 | \begin{center} |
---|
3110 | \begin{tabular}{l} |
---|
3111 | {\sf adouble x1=2, x2=4, y;}\\ |
---|
3112 | ...\\ |
---|
3113 | {\sf x1.setADValue(1);}\\ |
---|
3114 | {\sf x2.setADValue(0);} |
---|
3115 | \end{tabular} |
---|
3116 | \end{center} |
---|
3117 | The same example as above but now using {\sf setADValue}-method for initializing the derivative values. |
---|
3118 | \end{itemize} |
---|
3119 | % |
---|
3120 | The derivatives can be obtained at any time during the evaluation |
---|
3121 | process by calling the {\sf getADValue}-method |
---|
3122 | \begin{center} |
---|
3123 | \begin{tabular}{l} |
---|
3124 | {\sf adouble y;}\\ |
---|
3125 | ...\\ |
---|
3126 | {\sf cout $<<$ y.getADValue();} |
---|
3127 | \end{tabular} |
---|
3128 | \end{center} |
---|
3129 | \autoref{fig:modcode} shows the resulting source code incorporating |
---|
3130 | all required changes for the example |
---|
3131 | given above. |
---|
3132 | |
---|
3133 | \begin{figure}[htb] |
---|
3134 | \framebox[\textwidth]{\parbox{\textwidth}{ |
---|
3135 | %\begin{center} |
---|
3136 | \begin{tabbing} |
---|
3137 | \hspace*{-1cm} \= \kill |
---|
3138 | \> {\sf \#include $<$iostream$>$}\\ |
---|
3139 | \> {\sf using namespace std;}\\ |
---|
3140 | \> \\ |
---|
3141 | \> {\sf \#define ADOLC\_TAPELESS}\\ |
---|
3142 | \> {\sf \#include $<$adouble.h$>$}\\ |
---|
3143 | \> {\sf typedef adtl::adouble adouble;}\\ |
---|
3144 | \\ |
---|
3145 | \> {\sf int main() \{}\\ |
---|
3146 | \> {\sf \rule{0.5cm}{0pt}adouble x[3], y[3];}\\ |
---|
3147 | \\ |
---|
3148 | \> {\sf \rule{0.5cm}{0pt}for (int i=0; i$<$3; ++i)\hspace*{3cm}// Initialize $x_i$}\\ |
---|
3149 | \> {\sf \rule{1cm}{0pt}...}\\ |
---|
3150 | \\ |
---|
3151 | \> {\sf \rule{0.5cm}{0pt}x[0].setADValue(1);\hspace*{3cm}// derivative of f with respect to $x_1$}\\ |
---|
3152 | \> {\sf \rule{0.5cm}{0pt}y[0] = sqrt(x[0]*x[0]+x[1]*x[1]+x[2]*x[2]);}\\ |
---|
3153 | \> {\sf \rule{0.5cm}{0pt}y[1] = atan(sqrt(x[0]*x[0]+x[1]*x[1])/x[2]);}\\ |
---|
3154 | \> {\sf \rule{0.5cm}{0pt}y[2] = atan(x[1]/x[0]);}\\ |
---|
3155 | \\ |
---|
3156 | \> {\sf \rule{0.5cm}{0pt}cout $<<$ "y1=" $<<$ y[0].getValue() $<<$ " , y2=" $<<$ y[1].getValue ... ;}\\ |
---|
3157 | \> {\sf \rule{0.5cm}{0pt}cout $<<$ "dy2/dx1 = " $<<$ y[1].getADValue() $<<$ endl;}\\ |
---|
3158 | \> {\sf \rule{0.5cm}{0pt}return 0;}\\ |
---|
3159 | \> {\sf \}} |
---|
3160 | \end{tabbing} |
---|
3161 | %\end{center} |
---|
3162 | }} |
---|
3163 | \caption{Example for tapeless scalar forward mode} |
---|
3164 | \label{fig:modcode} |
---|
3165 | \end{figure} |
---|
3166 | % |
---|
3167 | \subsubsection*{The vector mode} |
---|
3168 | % |
---|
3169 | In scalar mode only one direction element has to be stored per {\sf |
---|
3170 | adouble} whereas a field of $p$ elements is needed in the vector |
---|
3171 | mode to cover the computations for the given $p$ directions. The |
---|
3172 | resulting changes to the source code are described in this section. |
---|
3173 | |
---|
3174 | Similar to tapeless scalar forward mode, the tapeless vector forward |
---|
3175 | mode is used by defining {\sf ADOLC\_TAPELESS}. Furthermore, one has to define |
---|
3176 | an additional preprocessor macro named {\sf NUMBER\_DIRECTIONS}. This |
---|
3177 | macro takes the maximal number of directions to be used within the |
---|
3178 | resulting vector mode. Just as {\sf ADOLC\_TAPELESS} the new macro |
---|
3179 | must be defined before including the \verb#<adolc.h/adouble.h># |
---|
3180 | header file since it is ignored otherwise. |
---|
3181 | |
---|
3182 | In many situations recompiling the source code to get a new number of |
---|
3183 | directions is at least undesirable. ADOL-C offers a function named |
---|
3184 | {\sf setNumDir} to work around this problem partially. Calling this |
---|
3185 | function, ADOL-C does not take the number of directions |
---|
3186 | from the macro {\sf NUMBER\_DIRECTIONS} but from the argument of |
---|
3187 | {\sf setNumDir}. A corresponding source code would contain the following lines: |
---|
3188 | \begin{center} |
---|
3189 | \begin{tabular}{l} |
---|
3190 | {\sf \#define NUMBER\_DIRECTIONS 10}\\ |
---|
3191 | ...\\ |
---|
3192 | {\sf adtl::setNumDir(5);} |
---|
3193 | \end{tabular} |
---|
3194 | \end{center} |
---|
3195 | Note that using this function does not |
---|
3196 | change memory requirements that can be roughly determined by |
---|
3197 | ({\sf NUMBER\_DIRECTIONS}$+1$)*(number of {\sf adouble}s). |
---|
3198 | |
---|
3199 | Compared to the scalar case setting and getting the derivative |
---|
3200 | values, i.e. the directions, is more involved. Instead of |
---|
3201 | working with single {\sf double} values, pointer to fields of {\sf |
---|
3202 | double}s are used as illustrated by the following example: |
---|
3203 | \begin{center} |
---|
3204 | \begin{tabular}{l} |
---|
3205 | {\sf \#define NUMBER\_DIRECTIONS 10}\\ |
---|
3206 | ...\\ |
---|
3207 | {\sf adouble x, y;}\\ |
---|
3208 | {\sf double *ptr=new double[NUMBER\_DIRECTIONS];}\\ |
---|
3209 | ...\\ |
---|
3210 | {\sf x1=2;}\\ |
---|
3211 | {\sf x1.setADValue(ptr);}\\ |
---|
3212 | ...\\ |
---|
3213 | {\sf ptr=y.getADValue();} |
---|
3214 | \end{tabular} |
---|
3215 | \end{center} |
---|
3216 | Additionally, the tapeless vector forward mode of ADOL-C offers two |
---|
3217 | new methods for setting/getting the derivative values. Similar |
---|
3218 | to the scalar case, {\sf double} values are used but due to the vector |
---|
3219 | mode the position of the desired vector element must be supplied in |
---|
3220 | the argument list: |
---|
3221 | \begin{center} |
---|
3222 | \begin{tabular}{l} |
---|
3223 | {\sf \#define NUMBER\_DIRECTIONS 10}\\ |
---|
3224 | ...\\ |
---|
3225 | {\sf adouble x, y;}\\ |
---|
3226 | ...\\ |
---|
3227 | {\sf x1=2;}\\ |
---|
3228 | {\sf x1.setADValue(5,1);\hspace*{3.7cm}// set the 6th point value of x to 1.0}\\ |
---|
3229 | ...\\ |
---|
3230 | {\sf cout $<<$ y.getADValue(3) $<<$ endl;\hspace*{1cm}// print the 4th derivative value of y} |
---|
3231 | \end{tabular} |
---|
3232 | \end{center} |
---|
3233 | The resulting source code containing all changes that are required is |
---|
3234 | shown in \autoref{fig:modcode2} |
---|
3235 | |
---|
3236 | \begin{figure}[!h!t!b] |
---|
3237 | \framebox[\textwidth]{\parbox{\textwidth}{ |
---|
3238 | \begin{tabbing} |
---|
3239 | \hspace*{-1cm} \= \kill |
---|
3240 | \> {\sf \#include $<$iostream$>$}\\ |
---|
3241 | \> {\sf using namespace std;}\\ |
---|
3242 | \\ |
---|
3243 | \> {\sf \#define ADOLC\_TAPELESS}\\ |
---|
3244 | \> {\sf \#define NUMBER\_DIRECTIONS 3}\\ |
---|
3245 | \> {\sf \#include $<$adouble.h$>$}\\ |
---|
3246 | \> {\sf typedef adtl::adouble adouble;}\\ |
---|
3247 | \\ |
---|
3248 | \> {\sf ADOLC\_TAPELESS\_UNIQUE\_INTERNALS;}\\ |
---|
3249 | \\ |
---|
3250 | \> {\sf int main() \{}\\ |
---|
3251 | \> {\sf \rule{0.5cm}{0pt}adouble x[3], y[3];}\\ |
---|
3252 | \\ |
---|
3253 | \> {\sf \rule{0.5cm}{0pt}for (int i=0; i$<$3; ++i) \{}\\ |
---|
3254 | \> {\sf \rule{1cm}{0pt}...\hspace*{3cm}// Initialize $x_i$}\\ |
---|
3255 | \> {\sf \rule{1cm}{0pt}for (int j=0; j$<$3; ++j) if (i==j) x[i].setADValue(j,1);}\\ |
---|
3256 | \> {\sf \rule{0.5cm}{0pt}\}}\\ |
---|
3257 | \\ |
---|
3258 | \> {\sf \rule{0.5cm}{0pt}y[0] = sqrt(x[0]*x[0]+x[1]*x[1]+x[2]*x[2]);}\\ |
---|
3259 | \> {\sf \rule{0.5cm}{0pt}y[1] = atan(sqrt(x[0]*x[0]+x[1]*x[1])/x[2]);}\\ |
---|
3260 | \> {\sf \rule{0.5cm}{0pt}y[2] = atan(x[1]/x[0]);}\\ |
---|
3261 | \\ |
---|
3262 | \> {\sf \rule{0.5cm}{0pt}cout $<<$ "y1=" $<<$ y[0].getValue() $<<$ " , y2=" $<<$ y[1].getValue ... ;}\\ |
---|
3263 | \> {\sf \rule{0.5cm}{0pt}cout $<<$ "jacobian : " $<<$ endl;}\\ |
---|
3264 | \> {\sf \rule{0.5cm}{0pt}for (int i=0; i$<$3; ++i) \{}\\ |
---|
3265 | \> {\sf \rule{1cm}{0pt}for (int j=0; j$<$3; ++j)}\\ |
---|
3266 | \> {\sf \rule{1.5cm}{0pt}cout $<<$ y[i].getADValue(j) $<<$ " ";}\\ |
---|
3267 | \> {\sf \rule{1cm}{0pt}cout $<<$ endl;}\\ |
---|
3268 | \> {\sf \rule{0.5cm}{0pt}\}}\\ |
---|
3269 | \> {\sf \rule{0.5cm}{0pt}return 0;}\\ |
---|
3270 | \> {\sf \}} |
---|
3271 | \end{tabbing} |
---|
3272 | }} |
---|
3273 | \caption{Example for tapeless vector forward mode} |
---|
3274 | \label{fig:modcode2} |
---|
3275 | \end{figure} |
---|
3276 | % |
---|
3277 | \subsection{Compiling and Linking the Source Code} |
---|
3278 | % |
---|
3279 | After incorporating the required changes, one has to compile the |
---|
3280 | source code and link the object files to get the executable. |
---|
3281 | As long as the ADOL-C header files are not included in the absolute path |
---|
3282 | the compile sequence should be similar to the following example: |
---|
3283 | \begin{center} |
---|
3284 | \begin{tabular}{l} |
---|
3285 | {\sf g++ -I/home/username/adolc\_base/include -c tapeless\_scalar.cpp} |
---|
3286 | \end{tabular} |
---|
3287 | \end{center} |
---|
3288 | The \verb#-I# option tells the compiler where to search for the ADOL-C |
---|
3289 | header files. This option can be omitted when the headers are included |
---|
3290 | with absolute path or if ADOL-C is installed in a ``global'' directory. |
---|
3291 | |
---|
3292 | Since the tapeless forward version of ADOL-C is implemented in the |
---|
3293 | header \verb#adouble.h# as complete inline intended version, |
---|
3294 | the object files do not need to be linked against any external ADOL-C |
---|
3295 | code or the ADOL-C library. Therefore, the example started above could be finished with the |
---|
3296 | following command: |
---|
3297 | \begin{center} |
---|
3298 | \begin{tabular}{l} |
---|
3299 | {\sf g++ -o tapeless\_scalar tapeless\_scalar.o} |
---|
3300 | \end{tabular} |
---|
3301 | \end{center} |
---|
3302 | The mentioned source codes {\sf tapeless\_scalar.c} and {\sf tapeless\_vector.c} |
---|
3303 | illustrating the use of the for tapeless scalar and vector mode can be found in |
---|
3304 | the directory {\sf examples}. |
---|
3305 | % |
---|
3306 | \subsection{Concluding Remarks for the Tapeless Forward Mode Variant} |
---|
3307 | % |
---|
3308 | As many other AD methods the tapeless forward mode provided by the |
---|
3309 | ADOL-C package has its own strengths and drawbacks. Please read the |
---|
3310 | following section carefully to become familiar with the things that |
---|
3311 | can occur: |
---|
3312 | \begin{itemize} |
---|
3313 | \item Advantages: |
---|
3314 | \begin{itemize} |
---|
3315 | \item Code speed\\ |
---|
3316 | Increasing computation speed was one of the main aspects in writing |
---|
3317 | the tapeless code. In many cases higher performance can be |
---|
3318 | expected this way. |
---|
3319 | \item Easier linking process\\ |
---|
3320 | As another result from the code inlining the object code does |
---|
3321 | not need to be linked against an ADOL-C library. |
---|
3322 | \item Smaller overall memory requirements\\ |
---|
3323 | Tapeless ADOL-C does not write tapes anymore, as the name |
---|
3324 | implies. Loop ''unrolling'' can be avoided this |
---|
3325 | way. Considered main memory plus disk space as overall memory |
---|
3326 | requirements the tapeless version can be |
---|
3327 | executed in a more efficient way. |
---|
3328 | \end{itemize} |
---|
3329 | \item Drawbacks: |
---|
3330 | \begin{itemize} |
---|
3331 | \item Main memory limitations\\ |
---|
3332 | The ability to compute derivatives to a given function is |
---|
3333 | bounded by the main memory plus swap size when using |
---|
3334 | tapeless ADOL-C. Computation from swap should be avoided anyway |
---|
3335 | as far as possible since it slows down the computing time |
---|
3336 | drastically. Therefore, if the program execution is |
---|
3337 | terminated without error message insufficient memory size can be |
---|
3338 | the reason among other things. The memory requirements $M$ can |
---|
3339 | be determined roughly as followed: |
---|
3340 | \begin{itemize} |
---|
3341 | \item Scalar mode: $M=$(number of {\sf adouble}s)$*2 + M_p$ |
---|
3342 | \item Vector mode: $M=$(number of {\sf adouble}s)*({\sf |
---|
3343 | NUMBER\_DIRECTIONS}$+1) + M_p$ |
---|
3344 | \end{itemize} |
---|
3345 | where the storage size of all non {\sf adouble} based variables is described by $M_p$. |
---|
3346 | \item Compile time\\ |
---|
3347 | As discussed in the previous sections, the tapeless forward mode of |
---|
3348 | the ADOL-C package is implemented as inline intended version. Using |
---|
3349 | this approach results in a higher source code size, since every |
---|
3350 | operation involving at least one {\sf adouble} stands for the |
---|
3351 | operation itself as well as for the corresponding derivative |
---|
3352 | code after the inlining process. Therefore, the compilation time |
---|
3353 | needed for the tapeless version may be higher than that of the tape based code. |
---|
3354 | \item Code Size\\ |
---|
3355 | A second drawback and result of the code inlining is the |
---|
3356 | increase of code sizes for the binaries. The increase |
---|
3357 | factor compared to the corresponding tape based program is |
---|
3358 | difficult to quantify as it is task dependent. Practical results |
---|
3359 | have shown that values between 1.0 and 2.0 can be |
---|
3360 | expected. Factors higher than 2.0 are possible too and even |
---|
3361 | values below 1.0 have been observed. |
---|
3362 | \end{itemize} |
---|
3363 | \end{itemize} |
---|
3364 | % |
---|
3365 | %++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
---|
3366 | \section{Installing and Using ADOL-C} |
---|
3367 | \label{install} |
---|
3368 | % |
---|
3369 | \subsection{Generating the ADOL-C Library} |
---|
3370 | \label{genlib} |
---|
3371 | % |
---|
3372 | The currently built system is best summarized by the ubiquitous gnu |
---|
3373 | install triplet |
---|
3374 | \begin{center} |
---|
3375 | \verb=configure - make - make install= . |
---|
3376 | \end{center} |
---|
3377 | Executing this three steps from the package base directory |
---|
3378 | \verb=</SOMEPATH/=\texttt{\packagetar}\verb=>= will compile the static and the dynamic |
---|
3379 | ADOL-C library with default options and install the package (libraries |
---|
3380 | and headers) into the default installation directory {\tt |
---|
3381 | \verb=<=\$HOME/adolc\_base\verb=>=}. Inside the install directory |
---|
3382 | the subdirectory \verb=include= will contain all the installed header |
---|
3383 | files that may be included by the user program, the subdirectory |
---|
3384 | \verb=lib= will contain the 32-bit compiled library |
---|
3385 | and the subdirectory \verb=lib64= will contain the 64-bit compiled |
---|
3386 | library. Depending on the compiler only one of \verb=lib= or |
---|
3387 | \verb=lib64= may be created. |
---|
3388 | |
---|
3389 | Before doing so the user may modify the header file \verb=usrparms.h= |
---|
3390 | in order to tailor the \mbox{ADOL-C} package to the needs in the |
---|
3391 | particular system environment as discussed in |
---|
3392 | \autoref{Customizing}. The configure procedure which creates the necessary |
---|
3393 | \verb=Makefile=s can be customized by use of some switches. Available |
---|
3394 | options and their meaning can be obtained by executing |
---|
3395 | \verb=./configure --help= from the package base directory. |
---|
3396 | |
---|
3397 | All object files and other intermediately generated files can be |
---|
3398 | removed by the call \verb=make clean=. Uninstalling ADOL-C by |
---|
3399 | executing \verb=make uninstall= is only reasonable after a previous |
---|
3400 | called \verb=make install= and will remove all installed package files |
---|
3401 | but will leave the created directories behind. |
---|
3402 | |
---|
3403 | The sparse drivers are included in the ADOL-C libraries if the |
---|
3404 | \verb=./configure= command is executed with the option |
---|
3405 | \verb=--enable-sparse=. The ColPack library available at |
---|
3406 | \verb=http://www.cscapes.org/coloringpage/software.htm= is required to |
---|
3407 | compute the sparse structures, and is searched for in all the default |
---|
3408 | locations as well as in the subdirectory \verb=<ThirdParty/ColPack/>=. |
---|
3409 | In case the library and its headers are installed in a nonstandard path |
---|
3410 | this may be specified with the \verb?--with-colpack=PATH? option. |
---|
3411 | It is assumed that the library and its header files have the following |
---|
3412 | directory structure: \verb?PATH/include? contains all the header |
---|
3413 | files, |
---|
3414 | \verb?PATH/lib? contains the 32-bit compiled library and |
---|
3415 | \verb?PATH/lib64? contains the 64-bit compiled library. Depending on |
---|
3416 | the compiler used to compile {\sf ADOL-C} one of these libraries will |
---|
3417 | be used for linking. |
---|
3418 | |
---|
3419 | \subsection{Compiling and Linking the Example Programs} |
---|
3420 | % |
---|
3421 | The installation procedure described in \autoref{genlib} also |
---|
3422 | provides the \verb=Makefile=s to compile the example programs in the |
---|
3423 | directories \verb=<=\texttt{\packagetar}\verb=>/ADOL-C/examples= and the |
---|
3424 | additional examples in |
---|
3425 | \verb=<=\texttt{\packagetar}\verb=>/ADOL-C/examples/additional_examples=. However, |
---|
3426 | one has to execute the |
---|
3427 | \verb=configure= command with appropriate options for the ADOL-C package to enable the compilation of |
---|
3428 | examples. Available options are: |
---|
3429 | \begin{center} |
---|
3430 | \begin{tabular}[t]{ll} |
---|
3431 | \verb=--enable-docexa=&build all examples discussed in this manual\\ |
---|
3432 | &(compare \autoref{example})\\ |
---|
3433 | \verb=--enable-addexa=&build all additional examples\\ |
---|
3434 | &(See file \verb=README= in the various subdirectories) |
---|
3435 | \end{tabular} |
---|
3436 | \end{center} |
---|
3437 | |
---|
3438 | Just calling \verb=make= from the packages base directory generates |
---|
3439 | all configured examples and the library if necessary. Compiling from |
---|
3440 | subdirectory \verb=examples= or one of its subfolders is possible |
---|
3441 | too. At least one kind of the ADOL-C library (static or shared) must |
---|
3442 | have been built previously in that case. Hence, building the library |
---|
3443 | is always the first step. |
---|
3444 | |
---|
3445 | For Compiling the library and the documented examples on Windows using |
---|
3446 | Visual Studio please refer to the \verb=<Readme_VC++.txt>= files in |
---|
3447 | the \verb=<windows/>=, \verb=<ThirdParty/ColPack/>= and |
---|
3448 | \verb=<ADOL-C/examples/>= subdirectories. |
---|
3449 | % |
---|
3450 | \subsection{Description of Important Header Files} |
---|
3451 | \label{ssec:DesIH} |
---|
3452 | % |
---|
3453 | The application of the facilities of ADOL-C requires the user |
---|
3454 | source code (program or module) to include appropriate |
---|
3455 | header files where the desired data types and routines are |
---|
3456 | prototyped. The new hierarchy of header files enables the user |
---|
3457 | to take one of two possible ways to access the right interfaces. |
---|
3458 | The first and easy way is recommended to beginners: As indicated in |
---|
3459 | \autoref{globalHeaders} the provided {\em global} header file |
---|
3460 | \verb=<adolc/adolc.h>= can be included by any user code to support all |
---|
3461 | capabilities of ADOL-C depending on the particular programming language |
---|
3462 | of the source. |
---|
3463 | |
---|
3464 | \begin{table}[h] |
---|
3465 | \center \small |
---|
3466 | \begin{tabular}{|p{3.6cm}|p{10.5cm}|}\hline |
---|
3467 | \verb=<adolc/adolc.h>= & |
---|
3468 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3469 | \boldmath $\rightarrow$ \unboldmath |
---|
3470 | & global header file available for easy use of ADOL-C; \\ |
---|
3471 | $\bullet$ & includes all ADOL-C header files depending on |
---|
3472 | whether the users source is C++ or C code. |
---|
3473 | \end{tabular*} |
---|
3474 | \\ \hline |
---|
3475 | \verb=<adolc/usrparms.h>= & |
---|
3476 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3477 | \boldmath $\rightarrow$ \unboldmath |
---|
3478 | & user customization of ADOL-C package (see |
---|
3479 | \autoref{Customizing}); \\ |
---|
3480 | $\bullet$ & after a change of |
---|
3481 | user options the ADOL-C library \verb=libadolc.*= |
---|
3482 | has to be rebuilt (see \autoref{genlib}); \\ |
---|
3483 | $\bullet$ & is included by all ADOL-C header files and thus by all user |
---|
3484 | programs. |
---|
3485 | \end{tabular*} \\ \hline |
---|
3486 | \end{tabular} |
---|
3487 | \caption{Global header files} |
---|
3488 | \label{globalHeaders} |
---|
3489 | \end{table} |
---|
3490 | |
---|
3491 | The second way is meant for the more advanced ADOL-C user: Some source code |
---|
3492 | includes only those interfaces used by the particular application. |
---|
3493 | The respectively needed header files are indicated |
---|
3494 | throughout the manual. |
---|
3495 | Existing application determined dependences between the provided |
---|
3496 | ADOL-C routines are realized by automatic includes of headers in order |
---|
3497 | to maintain easy use. The header files important to the user are described |
---|
3498 | in the \autoref{importantHeaders1} and \autoref{importantHeaders2}. |
---|
3499 | |
---|
3500 | \begin{table}[h] |
---|
3501 | \center \small |
---|
3502 | \begin{tabular}{|p{3.8cm}|p{10.5cm}|}\hline |
---|
3503 | %\multicolumn{2}{|l|}{\bf Tracing/taping}\\ \hline |
---|
3504 | \verb=<adolc/adouble.h>= & |
---|
3505 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3506 | \boldmath $\rightarrow$ \unboldmath |
---|
3507 | & provides the interface to the basic active |
---|
3508 | scalar data type of ADOL-C: {\sf class adouble} |
---|
3509 | (see \autoref{prepar}); |
---|
3510 | % $\bullet$ & includes the header files \verb=<adolc/avector.h>= and \verb=<adolc/taputil.h>=. |
---|
3511 | \end{tabular*} |
---|
3512 | \\ \hline |
---|
3513 | % \verb=<adolc/avector.h>= & |
---|
3514 | %\begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3515 | % \boldmath $\rightarrow$ \unboldmath |
---|
3516 | % & provides the interface to the active vector |
---|
3517 | % and matrix data types of ADOL-C: {\sf class adoublev} |
---|
3518 | % and {\sf class adoublem}, respectively |
---|
3519 | % (see \autoref{arrays}); \\ |
---|
3520 | % $\bullet$ & is included by the header \verb=<adolc/adouble.h>=. |
---|
3521 | %\end{tabular*} |
---|
3522 | %\\ \hline |
---|
3523 | \verb=<adolc/taputil.h>= & |
---|
3524 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3525 | \boldmath $\rightarrow$ \unboldmath |
---|
3526 | & provides functions to start/stop the tracing of |
---|
3527 | active sections (see \autoref{markingActive}) |
---|
3528 | as well as utilities to obtain |
---|
3529 | tape statistics (see \autoref{examiningTape}); \\ |
---|
3530 | $\bullet$ & is included by the header \verb=<adolc/adouble.h>=. |
---|
3531 | \end{tabular*} |
---|
3532 | \\ \hline |
---|
3533 | \end{tabular} |
---|
3534 | \caption{Important header files: tracing/taping} |
---|
3535 | \label{importantHeaders1} |
---|
3536 | \end{table} |
---|
3537 | % |
---|
3538 | \begin{table}[h] |
---|
3539 | \center \small |
---|
3540 | \begin{tabular}{|p{3.8cm}|p{10.5cm}|}\hline |
---|
3541 | %\multicolumn{2}{|l|}{\bf Evaluation of derivatives}\\ \hline |
---|
3542 | \verb=<adolc/interfaces.h>= & |
---|
3543 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3544 | \boldmath $\rightarrow$ \unboldmath |
---|
3545 | & provides interfaces to the {\sf forward} and |
---|
3546 | {\sf reverse} routines as basic versions of derivative |
---|
3547 | evaluation (see \autoref{forw_rev}); \\ |
---|
3548 | $\bullet$ & comprises C++, C, and Fortran-callable versions; \\ |
---|
3549 | $\bullet$ & includes the header \verb=<adolc/sparse/sparsedrivers.h>=; \\ |
---|
3550 | $\bullet$ & is included by the header \verb=<adolc/drivers/odedrivers.h>=. |
---|
3551 | \end{tabular*} |
---|
3552 | \\ \hline |
---|
3553 | \verb=<adolc/drivers.h>= & |
---|
3554 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3555 | \boldmath $\rightarrow$ \unboldmath |
---|
3556 | & provides ``easy to use'' drivers for solving |
---|
3557 | optimization problems and nonlinear equations |
---|
3558 | (see \autoref{optdrivers}); \\ |
---|
3559 | $\bullet$ & comprises C and Fortran-callable versions. |
---|
3560 | \end{tabular*} |
---|
3561 | \\ \hline |
---|
3562 | \begin{minipage}{3cm} |
---|
3563 | \verb=<adolc/sparse/=\newline\verb= sparsedrivers.h>= |
---|
3564 | \end{minipage} & |
---|
3565 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3566 | \boldmath $\rightarrow$ \unboldmath |
---|
3567 | & provides the ``easy to use'' sparse drivers |
---|
3568 | to exploit the sparsity structure of |
---|
3569 | Jacobians (see \autoref{sparse}); \\ |
---|
3570 | \boldmath $\rightarrow$ \unboldmath & provides interfaces to \mbox{C++}-callable versions |
---|
3571 | of {\sf forward} and {\sf reverse} routines |
---|
3572 | propagating bit patterns (see \autoref{ProBit}); \\ |
---|
3573 | |
---|
3574 | $\bullet$ & is included by the header \verb=<adolc/interfaces.h>=. |
---|
3575 | \end{tabular*} |
---|
3576 | \\ \hline |
---|
3577 | \begin{minipage}{3cm} |
---|
3578 | \verb=<adolc/sparse/=\newline\verb= sparse_fo_rev.h>= |
---|
3579 | \end{minipage} & |
---|
3580 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3581 | \boldmath $\rightarrow$ \unboldmath |
---|
3582 | & provides interfaces to the underlying C-callable |
---|
3583 | versions of {\sf forward} and {\sf reverse} routines |
---|
3584 | propagating bit patterns. |
---|
3585 | \end{tabular*} |
---|
3586 | \\ \hline |
---|
3587 | \begin{minipage}{3cm} |
---|
3588 | \verb=<adolc/drivers/=\newline\verb= odedrivers.h>= |
---|
3589 | \end{minipage} & |
---|
3590 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3591 | \boldmath $\rightarrow$ \unboldmath |
---|
3592 | & provides ``easy to use'' drivers for numerical |
---|
3593 | solution of ordinary differential equations |
---|
3594 | (see \autoref{odedrivers}); \\ |
---|
3595 | $\bullet$ & comprises C++, C, and Fortran-callable versions; \\ |
---|
3596 | $\bullet$ & includes the header \verb=<adolc/interfaces.h>=. |
---|
3597 | \end{tabular*} |
---|
3598 | \\ \hline |
---|
3599 | \begin{minipage}{3cm} |
---|
3600 | \verb=<adolc/drivers/=\newline\verb= taylor.h>= |
---|
3601 | \end{minipage} & |
---|
3602 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3603 | \boldmath $\rightarrow$ \unboldmath |
---|
3604 | & provides ``easy to use'' drivers for evaluation |
---|
3605 | of higher order derivative tensors (see |
---|
3606 | \autoref{higherOrderDeriv}) and inverse/implicit function |
---|
3607 | differentiation (see \autoref{implicitInverse});\\ |
---|
3608 | $\bullet$ & comprises C++ and C-callable versions. |
---|
3609 | \end{tabular*} |
---|
3610 | \\ \hline |
---|
3611 | \verb=<adolc/adalloc.h>= & |
---|
3612 | \begin{tabular*}{10.5cm}{cp{9.5cm}} |
---|
3613 | \boldmath $\rightarrow$ \unboldmath |
---|
3614 | & provides C++ and C functions for allocation of |
---|
3615 | vectors, matrices and three dimensional arrays |
---|
3616 | of {\sf double}s. |
---|
3617 | \end{tabular*} |
---|
3618 | \\ \hline |
---|
3619 | \end{tabular} |
---|
3620 | \caption{Important header files: evaluation of derivatives} |
---|
3621 | \label{importantHeaders2} |
---|
3622 | \end{table} |
---|
3623 | % |
---|
3624 | \subsection{Compiling and Linking C/C++ Programs} |
---|
3625 | % |
---|
3626 | To compile a C/C++ program or single module using ADOL-C |
---|
3627 | data types and routines one has to ensure that all necessary |
---|
3628 | header files according to \autoref{ssec:DesIH} are |
---|
3629 | included. All modules involving {\em active} data types as |
---|
3630 | {\sf adouble} |
---|
3631 | %, {\bf adoublev} and {\bf adoublem} |
---|
3632 | have to be compiled as C++. Modules that make use of a previously |
---|
3633 | generated tape to evaluate derivatives can either be programmed in ANSI-C |
---|
3634 | (while avoiding all C++ interfaces) or in C++. Depending |
---|
3635 | on the chosen programming language the header files provide |
---|
3636 | the right ADOL-C prototypes. |
---|
3637 | For linking the resulting object codes the library \verb=libadolc.*= |
---|
3638 | must be used (see \autoref{genlib}). |
---|
3639 | % |
---|
3640 | \subsection{Adding Quadratures as Special Functions} |
---|
3641 | % |
---|
3642 | \label{quadrat} |
---|
3643 | % |
---|
3644 | Suppose an integral |
---|
3645 | \[ f(x) = \int\limits^{x}_{0} g(t) dt \] |
---|
3646 | is evaluated numerically by a user-supplied function |
---|
3647 | \begin{center} |
---|
3648 | {\sf double myintegral(double\& x);} |
---|
3649 | \end{center} |
---|
3650 | Similarly, let us suppose that the integrand itself is evaluated by |
---|
3651 | a user-supplied block of C code {\sf integrand}, which computes a |
---|
3652 | variable with the fixed name {\sf val} from a variable with the fixed |
---|
3653 | name {\sf arg}. In many cases of interest, {\sf integrand} will |
---|
3654 | simply be of the form |
---|
3655 | \begin{center} |
---|
3656 | {\sf \{ val = expression(arg) \}}\enspace . |
---|
3657 | \end{center} |
---|
3658 | In general, the final assignment to {\sf val} may be preceded |
---|
3659 | by several intermediate calculations, possibly involving local |
---|
3660 | active variables of type {\sf adouble}, but no external or static |
---|
3661 | variables of that type. However, {\sf integrand} may involve local |
---|
3662 | or global variables of type {\sf double} or {\sf int}, provided they |
---|
3663 | do not depend on the value of {\sf arg}. The variables {\sf arg} and |
---|
3664 | {\sf val} are declared automatically; and as {\sf integrand} is a block |
---|
3665 | rather than a function, {\sf integrand} should have no header line. |
---|
3666 | |
---|
3667 | Now the function {\sf myintegral} can be overloaded for {\sf adouble} |
---|
3668 | arguments and thus included in the library of elementary functions |
---|
3669 | by the following modifications: |
---|
3670 | \begin{enumerate} |
---|
3671 | \item |
---|
3672 | At the end of the file \verb=<adouble.cpp>=, include the full code |
---|
3673 | defining \\ {\sf double myintegral(double\& x)}, and add the line |
---|
3674 | \begin{center} |
---|
3675 | {\sf extend\_quad(myintegral, integrand); } |
---|
3676 | \end{center} |
---|
3677 | This macro is extended to the definition of |
---|
3678 | {\sf adouble myintegral(adouble\& arg)}. |
---|
3679 | Then remake the library \verb=libadolc.*= (see \autoref{genlib}). |
---|
3680 | \item |
---|
3681 | In the definition of the class |
---|
3682 | {\sf ADOLC\_DLL\_EXPORT adouble} in \verb=<adolc/adouble.h>=, add the statement |
---|
3683 | \begin{center} |
---|
3684 | {\sf friend adouble myintegral(adouble\&)}. |
---|
3685 | \end{center} |
---|
3686 | \end{enumerate} |
---|
3687 | In the first modification, {\sf myintegral} represents the name of the |
---|
3688 | {\sf double} function, whereas {\sf integrand} represents the actual block |
---|
3689 | of C code. |
---|
3690 | |
---|
3691 | For example, in case of the inverse hyperbolic cosine, we have |
---|
3692 | {\sf myintegral} = {\sf acosh}. Then {\sf integrand} can be written as |
---|
3693 | {\sf \{ val = sqrt(arg*arg-1); \}} |
---|
3694 | so that the line |
---|
3695 | \begin{center} |
---|
3696 | {\sf extend\_quad(acosh,val = sqrt(arg*arg-1));} |
---|
3697 | \end{center} |
---|
3698 | can be added to the file \verb=<adouble.cpp>=. |
---|
3699 | A mathematically equivalent but longer representation of |
---|
3700 | {\sf integrand} is |
---|
3701 | \begin{center} |
---|
3702 | \begin{tabbing} |
---|
3703 | {\sf \{ }\hspace{1.0in}\= {\sf \{ adouble} \= temp = \kill |
---|
3704 | \>{\sf \{ adouble} \> {\sf temp = arg;} \\ |
---|
3705 | \> \ \> {\sf temp = temp*temp; } \\ |
---|
3706 | \> \ \> {\sf val = sqrt(temp-1); \}} |
---|
3707 | \end{tabbing} |
---|
3708 | \end{center} |
---|
3709 | The code block {\sf integrand} may call on any elementary function that has already |
---|
3710 | been defined in file \verb=<adouble.cpp>=, so that one may also introduce |
---|
3711 | iterated integrals. |
---|
3712 | % |
---|
3713 | % |
---|
3714 | \section{Example Codes} |
---|
3715 | \label{example} |
---|
3716 | % |
---|
3717 | The following listings are all simplified versions of codes that |
---|
3718 | are contained in the example subdirectory |
---|
3719 | \verb=<=\texttt{\packagetar}\verb=>/ADOL-C/examples= of ADOL-C. In particular, |
---|
3720 | we have left out timings, which are included in the complete codes. |
---|
3721 | % |
---|
3722 | \subsection{Speelpenning's Example ({\tt speelpenning.cpp})} |
---|
3723 | % |
---|
3724 | The first example evaluates the gradient and the Hessian of |
---|
3725 | the function |
---|
3726 | \[ |
---|
3727 | y \; = \; f(x)\; =\; \prod_{i=0}^{n-1} x_i |
---|
3728 | \] |
---|
3729 | using the appropriate drivers {\sf gradient} and {\sf hessian}. |
---|
3730 | |
---|
3731 | \begin{verbatim} |
---|
3732 | #include <adolc/adouble.h> // use of active doubles and taping |
---|
3733 | #include <adolc/drivers/drivers.h> // use of "Easy to Use" drivers |
---|
3734 | // gradient(.) and hessian(.) |
---|
3735 | #include <adolc/taping.h> // use of taping |
---|
3736 | ... |
---|
3737 | void main() { |
---|
3738 | int n,i,j,tape_stats[STAT_SIZE]; |
---|
3739 | cout << "SPEELPENNINGS PRODUCT (ADOL-C Documented Example) \n"; |
---|
3740 | cout << "number of independent variables = ? \n"; |
---|
3741 | cin >> n; |
---|
3742 | double* xp = new double[n]; |
---|
3743 | double yp = 0.0; |
---|
3744 | adouble* x = new adouble[n]; |
---|
3745 | adouble y = 1; |
---|
3746 | for(i=0;i<n;i++) |
---|
3747 | xp[i] = (i+1.0)/(2.0+i); // some initialization |
---|
3748 | trace_on(1); // tag =1, keep=0 by default |
---|
3749 | for(i=0;i<n;i++) { |
---|
3750 | x[i] <<= xp[i]; y *= x[i]; } |
---|
3751 | y >>= yp; |
---|
3752 | delete[] x; |
---|
3753 | trace_off(); |
---|
3754 | tapestats(1,tape_stats); // reading of tape statistics |
---|
3755 | cout<<"maxlive "<<tape_stats[2]<<"\n"; |
---|
3756 | ... // ..... print other tape stats |
---|
3757 | double* g = new double[n]; |
---|
3758 | gradient(1,n,xp,g); // gradient evaluation |
---|
3759 | double** H=(double**)malloc(n*sizeof(double*)); |
---|
3760 | for(i=0;i<n;i++) |
---|
3761 | H[i]=(double*)malloc((i+1)*sizeof(double)); |
---|
3762 | hessian(1,n,xp,H); // H equals (n-1)g since g is |
---|
3763 | double errg = 0; // homogeneous of degree n-1. |
---|
3764 | double errh = 0; |
---|
3765 | for(i=0;i<n;i++) |
---|
3766 | errg += fabs(g[i]-yp/xp[i]); // vanishes analytically. |
---|
3767 | for(i=0;i<n;i++) { |
---|
3768 | for(j=0;j<n;j++) { |
---|
3769 | if (i>j) // lower half of hessian |
---|
3770 | errh += fabs(H[i][j]-g[i]/xp[j]); } } |
---|
3771 | cout << yp-1/(1.0+n) << " error in function \n"; |
---|
3772 | cout << errg <<" error in gradient \n"; |
---|
3773 | cout << errh <<" consistency check \n"; |
---|
3774 | } // end main |
---|
3775 | \end{verbatim} |
---|
3776 | % |
---|
3777 | \subsection{Power Example ({\tt powexam.cpp})} |
---|
3778 | % |
---|
3779 | The second example function evaluates the $n$-th power of a real |
---|
3780 | variable $x$ in |
---|
3781 | $\log_2 n$ multiplications by recursive halving of the exponent. Since |
---|
3782 | there is only one independent variable, the scalar derivative can be |
---|
3783 | computed by |
---|
3784 | using both {\sf forward} and {\sf reverse}, and the |
---|
3785 | results are subsequently compared. |
---|
3786 | \begin{verbatim} |
---|
3787 | #include <adolc/adolc.h> // use of ALL ADOL-C interfaces |
---|
3788 | |
---|
3789 | adouble power(adouble x, int n) { |
---|
3790 | adouble z=1; |
---|
3791 | if (n>0) { // recursion and branches |
---|
3792 | int nh =n/2; // that do not depend on |
---|
3793 | z = power(x,nh); // adoubles are fine !!!! |
---|
3794 | z *= z; |
---|
3795 | if (2*nh != n) |
---|
3796 | z *= x; |
---|
3797 | return z; } // end if |
---|
3798 | else { |
---|
3799 | if (n==0) // the local adouble z dies |
---|
3800 | return z; // as it goes out of scope. |
---|
3801 | else |
---|
3802 | return 1/power(x,-n); } // end else |
---|
3803 | } // end power |
---|
3804 | \end{verbatim} |
---|
3805 | The function {\sf power} above was obtained from the original |
---|
3806 | undifferentiated version by simply changing the type of all |
---|
3807 | {\sf double}s including the return variable to {\sf adouble}s. The new version |
---|
3808 | can now be called from within any active section, as in the following |
---|
3809 | main program. |
---|
3810 | \begin{verbatim} |
---|
3811 | #include ... // as above |
---|
3812 | int main() { |
---|
3813 | int i,n,tag=1; |
---|
3814 | cout <<"COMPUTATION OF N-TH POWER (ADOL-C Documented Example)\n\n"; |
---|
3815 | cout<<"monomial degree=? \n"; // input the desired degree |
---|
3816 | cin >> n; |
---|
3817 | // allocations and initializations |
---|
3818 | double* Y[1]; |
---|
3819 | *Y = new double[n+2]; |
---|
3820 | double* X[1]; // allocate passive variables with |
---|
3821 | *X = new double[n+4]; // extra dimension for derivatives |
---|
3822 | X[0][0] = 0.5; // function value = 0. coefficient |
---|
3823 | X[0][1] = 1.0; // first derivative = 1. coefficient |
---|
3824 | for(i=0;i<n+2;i++) |
---|
3825 | X[0][i+2]=0; // further coefficients |
---|
3826 | double* Z[1]; // used for checking consistency |
---|
3827 | *Z = new double[n+2]; // between forward and reverse |
---|
3828 | adouble y,x; // declare active variables |
---|
3829 | // beginning of active section |
---|
3830 | trace_on(1); // tag = 1 and keep = 0 |
---|
3831 | x <<= X[0][0]; // only one independent var |
---|
3832 | y = power(x,n); // actual function call |
---|
3833 | y >>= Y[0][0]; // only one dependent adouble |
---|
3834 | trace_off(); // no global adouble has died |
---|
3835 | // end of active section |
---|
3836 | double u[1]; // weighting vector |
---|
3837 | u[0]=1; // for reverse call |
---|
3838 | for(i=0;i<n+2;i++) { // note that keep = i+1 in call |
---|
3839 | forward(tag,1,1,i,i+1,X,Y); // evaluate the i-the derivative |
---|
3840 | if (i==0) |
---|
3841 | cout << Y[0][i] << " - " << y.value() << " = " << Y[0][i]-y.value() |
---|
3842 | << " (should be 0)\n"; |
---|
3843 | else |
---|
3844 | cout << Y[0][i] << " - " << Z[0][i] << " = " << Y[0][i]-Z[0][i] |
---|
3845 | << " (should be 0)\n"; |
---|
3846 | reverse(tag,1,1,i,u,Z); // evaluate the (i+1)-st derivative |
---|
3847 | Z[0][i+1]=Z[0][i]/(i+1); } // scale derivative to Taylorcoeff. |
---|
3848 | return 1; |
---|
3849 | } // end main |
---|
3850 | \end{verbatim} |
---|
3851 | Since this example has only one independent and one dependent variable, |
---|
3852 | {\sf forward} and {\sf reverse} have the same complexity and calculate |
---|
3853 | the same scalar derivatives, albeit with a slightly different scaling. |
---|
3854 | By replacing the function {\sf power} with any other univariate test function, |
---|
3855 | one can check that {\sf forward} and {\sf reverse} are at least consistent. |
---|
3856 | In the following example the number of independents is much larger |
---|
3857 | than the number of dependents, which makes the reverse mode preferable. |
---|
3858 | % |
---|
3859 | \subsection{Determinant Example ({\tt detexam.cpp})} |
---|
3860 | % |
---|
3861 | Now let us consider an exponentially expensive calculation, |
---|
3862 | namely, the evaluation of a determinant by recursive expansion |
---|
3863 | along rows. The gradient of the determinant with respect to the |
---|
3864 | matrix elements is simply the adjoint, i.e. the matrix of cofactors. |
---|
3865 | Hence the correctness of the numerical result is easily checked by |
---|
3866 | matrix-vector multiplication. The example illustrates the use |
---|
3867 | of {\sf adouble} arrays and pointers. |
---|
3868 | |
---|
3869 | \begin{verbatim} |
---|
3870 | #include <adolc/adouble.h> // use of active doubles and taping |
---|
3871 | #include <adolc/interfaces.h> // use of basic forward/reverse |
---|
3872 | // interfaces of ADOL-C |
---|
3873 | adouble** A; // A is an n x n matrix |
---|
3874 | int i,n; // k <= n is the order |
---|
3875 | adouble det(int k, int m) { // of the sub-matrix |
---|
3876 | if (m == 0) return 1.0 ; // its column indices |
---|
3877 | else { // are encoded in m |
---|
3878 | adouble* pt = A[k-1]; |
---|
3879 | adouble t = zero; // zero is predefined |
---|
3880 | int s, p =1; |
---|
3881 | if (k%2) s = 1; else s = -1; |
---|
3882 | for(i=0;i<n;i++) { |
---|
3883 | int p1 = 2*p; |
---|
3884 | if (m%p1 >= p) { |
---|
3885 | if (m == p) { |
---|
3886 | if (s>0) t += *pt; else t -= *pt; } |
---|
3887 | else { |
---|
3888 | if (s>0) |
---|
3889 | t += *pt*det(k-1,m-p); // recursive call to det |
---|
3890 | else |
---|
3891 | t -= *pt*det(k-1,m-p); } // recursive call to det |
---|
3892 | s = -s;} |
---|
3893 | ++pt; |
---|
3894 | p = p1;} |
---|
3895 | return t; } |
---|
3896 | } // end det |
---|
3897 | \end{verbatim} |
---|
3898 | As one can see, the overloading mechanism has no problem with pointers |
---|
3899 | and looks exactly the same as the original undifferentiated function |
---|
3900 | except for the change of type from {\sf double} to {\sf adouble}. |
---|
3901 | If the type of the temporary {\sf t} or the pointer {\sf pt} had not been changed, |
---|
3902 | a compile time error would have resulted. Now consider a corresponding |
---|
3903 | calling program. |
---|
3904 | |
---|
3905 | \begin{verbatim} |
---|
3906 | #include ... // as above |
---|
3907 | int main() { |
---|
3908 | int i,j, m=1,tag=1,keep=1; |
---|
3909 | cout << "COMPUTATION OF DETERMINANTS (ADOL-C Documented Example)\n\n"; |
---|
3910 | cout << "order of matrix = ? \n"; // select matrix size |
---|
3911 | cin >> n; |
---|
3912 | A = new adouble*[n]; |
---|
3913 | trace_on(tag,keep); // tag=1=keep |
---|
3914 | double detout=0.0, diag = 1.0; // here keep the intermediates for |
---|
3915 | for(i=0;i<n;i++) { // the subsequent call to reverse |
---|
3916 | m *=2; |
---|
3917 | A[i] = new adouble[n]; // not needed for adoublem |
---|
3918 | adouble* pt = A[i]; |
---|
3919 | for(j=0;j<n;j++) |
---|
3920 | A[i][j] <<= j/(1.0+i); // make all elements of A independent |
---|
3921 | diag += A[i][i].value(); // value() converts to double |
---|
3922 | A[i][i] += 1.0; } |
---|
3923 | det(n,m-1) >>= detout; // actual function call |
---|
3924 | printf("\n %f - %f = %f (should be 0)\n",detout,diag,detout-diag); |
---|
3925 | trace_off(); |
---|
3926 | double u[1]; |
---|
3927 | u[0] = 1.0; |
---|
3928 | double* B = new double[n*n]; |
---|
3929 | reverse(tag,1,n*n,1,u,B); |
---|
3930 | cout <<" \n first base? : "; |
---|
3931 | for (i=0;i<n;i++) { |
---|
3932 | adouble sum = 0; |
---|
3933 | for (j=0;j<n;j++) // the matrix A times the first n |
---|
3934 | sum += A[i][j]*B[j]; // components of the gradient B |
---|
3935 | cout<<sum.value()<<" "; } // must be a Cartesian basis vector |
---|
3936 | return 1; |
---|
3937 | } // end main |
---|
3938 | \end{verbatim} |
---|
3939 | The variable {\sf diag} should be mathematically |
---|
3940 | equal to the determinant, because the |
---|
3941 | matrix {\sf A} is defined as a rank 1 perturbation of the identity. |
---|
3942 | % |
---|
3943 | \subsection{Ordinary Differential Equation Example ({\tt odexam.cpp})} |
---|
3944 | \label{exam:ode} |
---|
3945 | % |
---|
3946 | Here, we consider a nonlinear ordinary differential equation that |
---|
3947 | is a slight modification of the Robertson test problem |
---|
3948 | given in Hairer and Wanner's book on the numerical solution of |
---|
3949 | ODEs \cite{HW}. The following source code computes the corresponding |
---|
3950 | values of $y^{\prime} \in \R^3$: |
---|
3951 | \begin{verbatim} |
---|
3952 | #include <adolc/adouble.h> // use of active doubles and taping |
---|
3953 | #include <adolc/drivers/odedrivers.h> // use of "Easy To use" ODE drivers |
---|
3954 | #include <adolc/adalloc.h> // use of ADOL-C allocation utilities |
---|
3955 | |
---|
3956 | void tracerhs(short int tag, double* py, double* pyprime) { |
---|
3957 | adoublev y(3); // this time we left the parameters |
---|
3958 | adoublev yprime(3); // passive and use the vector types |
---|
3959 | trace_on(tag); |
---|
3960 | y <<= py; // initialize and mark independents |
---|
3961 | yprime[0] = -sin(y[2]) + 1e8*y[2]*(1-1/y[0]); |
---|
3962 | yprime[1] = -10*y[0] + 3e7*y[2]*(1-y[1]); |
---|
3963 | yprime[2] = -yprime[0] - yprime[1]; |
---|
3964 | yprime >>= pyprime; // mark and pass dependents |
---|
3965 | trace_off(tag); |
---|
3966 | } // end tracerhs |
---|
3967 | \end{verbatim} |
---|
3968 | The Jacobian of the right-hand side has large |
---|
3969 | negative eigenvalues, which make the ODE quite stiff. We have added |
---|
3970 | some numerically benign transcendentals to make the differentiation |
---|
3971 | more interesting. |
---|
3972 | The following main program uses {\sf forode} to calculate the Taylor series |
---|
3973 | defined by the ODE at the given point $y_0$ and {\sf reverse} as well |
---|
3974 | as {\sf accode} to compute the Jacobians of the coefficient vectors |
---|
3975 | with respect to $x_0$. |
---|
3976 | \begin{verbatim} |
---|
3977 | #include ....... // as above |
---|
3978 | int main() { |
---|
3979 | int i,j,deg; |
---|
3980 | int n=3; |
---|
3981 | double py[3]; |
---|
3982 | double pyp[3]; |
---|
3983 | cout << "MODIFIED ROBERTSON TEST PROBLEM (ADOL-C Documented Example)\n"; |
---|
3984 | cout << "degree of Taylor series =?\n"; |
---|
3985 | cin >> deg; |
---|
3986 | double **X; |
---|
3987 | X=(double**)malloc(n*sizeof(double*)); |
---|
3988 | for(i=0;i<n;i++) |
---|
3989 | X[i]=(double*)malloc((deg+1)*sizeof(double)); |
---|
3990 | double*** Z=new double**[n]; |
---|
3991 | double*** B=new double**[n]; |
---|
3992 | short** nz = new short*[n]; |
---|
3993 | for(i=0;i<n;i++) { |
---|
3994 | Z[i]=new double*[n]; |
---|
3995 | B[i]=new double*[n]; |
---|
3996 | for(j=0;j<n;j++) { |
---|
3997 | Z[i][j]=new double[deg]; |
---|
3998 | B[i][j]=new double[deg]; } // end for |
---|
3999 | } // end for |
---|
4000 | for(i=0;i<n;i++) { |
---|
4001 | py[i] = (i == 0) ? 1.0 : 0.0; // initialize the base point |
---|
4002 | X[i][0] = py[i]; // and the Taylor coefficient; |
---|
4003 | nz[i] = new short[n]; } // set up sparsity array |
---|
4004 | tracerhs(1,py,pyp); // trace RHS with tag = 1 |
---|
4005 | forode(1,n,deg,X); // compute deg coefficients |
---|
4006 | reverse(1,n,n,deg-1,Z,nz); // U defaults to the identity |
---|
4007 | accode(n,deg-1,Z,B,nz); |
---|
4008 | cout << "nonzero pattern:\n"; |
---|
4009 | for(i=0;i<n;i++) { |
---|
4010 | for(j=0;j<n;j++) |
---|
4011 | cout << nz[i][j]<<"\t"; |
---|
4012 | cout <<"\n"; } // end for |
---|
4013 | return 1; |
---|
4014 | } // end main |
---|
4015 | \end{verbatim} |
---|
4016 | \noindent The pattern {\sf nz} returned by {\sf accode} is |
---|
4017 | \begin{verbatim} |
---|
4018 | 3 -1 4 |
---|
4019 | 1 2 2 |
---|
4020 | 3 2 4 |
---|
4021 | \end{verbatim} |
---|
4022 | The original pattern {\sf nz} returned by {\sf reverse} is the same |
---|
4023 | except that the negative entry $-1$ was zero. |
---|
4024 | % |
---|
4025 | %\subsection {Gaussian Elimination Example ({\tt gaussexam.cpp})} |
---|
4026 | %\label{gaussexam} |
---|
4027 | % |
---|
4028 | %The following example uses conditional assignments to show the usage of a once produced tape |
---|
4029 | %for evaluation at new arguments. The elimination is performed with |
---|
4030 | %column pivoting. |
---|
4031 | %\begin{verbatim} |
---|
4032 | %#include <adolc/adolc.h> // use of ALL ADOL-C interfaces |
---|
4033 | |
---|
4034 | %void gausselim(int n, adoublem& A, adoublev& bv) { |
---|
4035 | %along i; // active integer declaration |
---|
4036 | %adoublev temp(n); // active vector declaration |
---|
4037 | %adouble r,rj,temps; |
---|
4038 | %int j,k; |
---|
4039 | %for(k=0;k<n;k++) { // elimination loop |
---|
4040 | % i = k; |
---|
4041 | % r = fabs(A[k][k]); // initial pivot size |
---|
4042 | % for(j=k+1;j<n;j++) { |
---|
4043 | % rj = fabs(A[j][k]); |
---|
4044 | % condassign(i,rj-r,j); // look for a larger element in the same |
---|
4045 | % condassign(r,rj-r,rj); } // column with conditional assignments |
---|
4046 | % temp = A[i]; // switch rows using active subscripting |
---|
4047 | % A[i] = A[k]; // necessary even if i happens to equal |
---|
4048 | % A[k] = temp; // k during taping |
---|
4049 | % temps = bv[i]; |
---|
4050 | % bv[i]=bv[k]; |
---|
4051 | % bv[k]=temps; |
---|
4052 | % if (!value(A[k][k])) // passive subscripting |
---|
4053 | % exit(1); // matrix singular! |
---|
4054 | % temps= A[k][k]; |
---|
4055 | % A[k] /= temps; |
---|
4056 | % bv[k] /= temps; |
---|
4057 | % for(j=k+1;j<n;j++) { |
---|
4058 | % temps= A[j][k]; |
---|
4059 | % A[j] -= temps*A[k]; // vector operations |
---|
4060 | % bv[j] -= temps*bv[k]; } // endfor |
---|
4061 | %} // end elimination loop |
---|
4062 | %temp=0.0; |
---|
4063 | %for(k=n-1;k>=0;k--) // backsubstitution |
---|
4064 | % temp[k] = (bv[k]-(A[k]*temp))/A[k][k]; |
---|
4065 | %bv=temp; |
---|
4066 | %} // end gausselim |
---|
4067 | %\end{verbatim} |
---|
4068 | %\noindent This function can be called from any program |
---|
4069 | %that suitably initializes |
---|
4070 | %the components of {\sf A} and {\sf bv} |
---|
4071 | %as independents. The resulting tape can be |
---|
4072 | %used to solve any nonsingular linear system of the same size and |
---|
4073 | %to get the sensitivities of the solution with respect to the |
---|
4074 | %system matrix and the right hand side. |
---|
4075 | %\vspace*{-4mm} |
---|
4076 | % |
---|
4077 | \section*{Acknowledgements} |
---|
4078 | % |
---|
4079 | Parts of the ADOL-C source were developed by Andreas |
---|
4080 | Kowarz, Hristo Mitev, Sebastian Schlenkrich, and Olaf |
---|
4081 | Vogel. We are also indebted to George Corliss, |
---|
4082 | Tom Epperly, Bruce Christianson, David Gay, David Juedes, |
---|
4083 | Brad Karp, Koichi Kubota, Bob Olson, Marcela Rosemblun, Dima |
---|
4084 | Shiriaev, Jay Srinivasan, Chuck Tyner, Jean Utke, and Duane Yoder for helping in |
---|
4085 | various ways with the development and documentation of ADOL-C. |
---|
4086 | % |
---|
4087 | \begin{thebibliography}{10} |
---|
4088 | |
---|
4089 | \bibitem{BeKh96} |
---|
4090 | Christian~H. Bischof, Peyvand~M. Khademi, Ali Bouaricha and Alan Carle. |
---|
4091 | \newblock {\em Efficient computation of gradients and Jacobians by dynamic |
---|
4092 | exploitation of sparsity in automatic differentiation}. |
---|
4093 | \newblock Optimization Methods and Software 7(1):1-39, 1996. |
---|
4094 | |
---|
4095 | \bibitem{Chri91a} |
---|
4096 | Bruce Christianson. |
---|
4097 | \newblock {\em Reverse accumulation and accurate rounding error estimates for |
---|
4098 | Taylor series}. |
---|
4099 | \newblock Optimization Methods and Software 1:81--94, 1992. |
---|
4100 | |
---|
4101 | \bibitem{GeMaPo05} |
---|
4102 | Assefaw Gebremedhin, Fredrik Manne, and Alex Pothen. |
---|
4103 | \newblock {\em What color is your {J}acobian? {G}raph coloring for computing |
---|
4104 | derivatives}. |
---|
4105 | \newblock SIAM Review 47(4):629--705, 2005. |
---|
4106 | |
---|
4107 | \bibitem{GePoTaWa06} |
---|
4108 | Assefaw Gebremedhin, Alex Pothen, Arijit Tarafdar and Andrea Walther. |
---|
4109 | {\em Efficient Computation of Sparse Hessians: An Experimental Study |
---|
4110 | using ADOL-C}. Tech. Rep. (2006). To appear in INFORMS Journal on Computing. |
---|
4111 | |
---|
4112 | \bibitem{GePoWa08} Assefaw Gebremedhin, Alex Pothen, and Andrea |
---|
4113 | Walther. |
---|
4114 | {\em Exploiting Sparsity in Jacobian Computation via Coloring and Automatic Differentiation: |
---|
4115 | a Case Study in a Simulated Moving Bed Process}. |
---|
4116 | In Chr. Bischof et al., eds., {\em Proceedings AD 2008 conference}, LNCSE 64, pp. 327 -- 338, Springer (2008). |
---|
4117 | |
---|
4118 | \bibitem{GeTaMaPo07} |
---|
4119 | Assefaw Gebremedhin, Arijit Tarafdar, Fredrik Manne, and Alex Pothen, |
---|
4120 | {\em New Acyclic and Star Coloring Algorithms with Applications to Hessian Computation}. |
---|
4121 | SIAM Journal on Scientific Computing 29(3):1042--1072, 2007. |
---|
4122 | |
---|
4123 | |
---|
4124 | \bibitem{GrWa08} |
---|
4125 | Andreas Griewank and Andrea Walther: {\em Evaluating Derivatives, Principles and Techniques of |
---|
4126 | Algorithmic Differentiation. Second edition}. SIAM, 2008. |
---|
4127 | |
---|
4128 | |
---|
4129 | \bibitem{Griewank97} |
---|
4130 | Andreas Griewank, Jean Utke, and Andrea Walther. |
---|
4131 | \newblock {\em Evaluating higher derivative tensors by forward propagation |
---|
4132 | of univariate Taylor series}. |
---|
4133 | \newblock Mathematics of Computation, 69:1117--1130, 2000. |
---|
4134 | |
---|
4135 | \bibitem{GrWa00} |
---|
4136 | Andreas Griewank and Andrea Walther. {\em Revolve: An Implementation of Checkpointing for the Reverse |
---|
4137 | or Adjoint Mode of Computational Differentiation}, |
---|
4138 | ACM Transaction on Mathematical Software 26:19--45, 2000. |
---|
4139 | |
---|
4140 | \bibitem{HW} |
---|
4141 | Ernst Hairer and Gerhard Wanner. |
---|
4142 | {\it Solving Ordinary Differential Equations II.\/} |
---|
4143 | Springer-Verlag, Berlin, 1991. |
---|
4144 | |
---|
4145 | \bibitem{Knuth73} |
---|
4146 | Donald~E. Knuth. |
---|
4147 | \newblock {\em The Art of Computer Programming. Second edition.} |
---|
4148 | \newblock Addison-Wesley, Reading, 1973. |
---|
4149 | |
---|
4150 | \bibitem{Wa05a} |
---|
4151 | Andrea Walther. |
---|
4152 | \newblock {\em Computing Sparse Hessians with Automatic Differentiation}. |
---|
4153 | \newblock Transaction on Mathematical Software, 34(1), Artikel 3 (2008). |
---|
4154 | \end{thebibliography} |
---|
4155 | \end{document} |
---|
4156 | |
---|