Skip to main content

Full text of "DTIC AD0616381: DYNAMIC PROGRAMMING AND CLASSICAL ANALYSIS"

See other formats


AD616381 


U 


wmuuhx corv wf i,^ 0 f 01 

^onf'-ON WHJ. bt ^ * 
"?S22^ RY osas or DDC. 





Rep'oduced by 

The RAND C o i p o r a t i o n • Santa Monica • C a I > ♦ o » n i a 


The views expressed ti this paper are not necessarily those of the Corporation 





CLEARINGHOUSE FOR FEDERAL SCIENTIFIC AND TECHNICAL INFORMATION. CFSTI 

INPUT SECTION 410.11 


LIMITATIONS IN REPRODUCTION QUALITY OF TECHNICAL ABSTRACT BULLETIN 
OOCUMENTS. DEFENSE DOCUMENTATION CENTER (DOC) 


J I. AVAILABLE ONLY FOR REFERENCE USE AT DOC FIELD SERVICES. 
COPY IS NOT AVAILABLE FOR PUBLIC SALE. 


^ 2. AVAILABLE COPY MILL NOT PERMIT FULLY LEGIBLE REPRODUCTION. 
REPRODUCTION WILL BE MADE IF REQUESTED BY USERS OF DDC. 


X' A. COPY IS AVAILABLE FOR PUBLIC SALE. 


B. COPY IS NOT AVAILABLE FOR PUBLIC SALE. 


3 LIMITED NUMBER OF COPIES CONTAINING COLOR OTHER THAN BLACK 

AND WHITE ARE AVAILABLE UNT'jL STOCK IS EXHAUSTED. REPRODUCTIONS 
WILL BE MADE IN BLACK AND WHITE ONLY. 


TSL-I2I-2 65 


DATE PROCESSED: 


V 






PROCESSOR: . / 


«■ * £ 7 ' 



SUMMARY 


P-1804 

5-28-59 

ii 


Over the Dost ten yearo, research In the field of dynamic 
programming has assumed many different forms. Sometimes, the 
emphasis has been upon Questions of formulation in analytic 

terms and concepts, sometimes upon the problems of existence 
and uniqueness of solutions of the functional equations derived 
from the underlying processes, occasionally upon the actual 
analytic structure of the solutions of these equations, some¬ 
times upon the computational aspects; and sometimes upon the 
applications—to control processes, to trajectories of various 
types, to operations research, tc mathematical economics. 

Inevitably, the result of this quasi-ergodlc behavior has 
been tc ignore a number of significant problems, and to treat 
a number of others In cavalier fashion. In this exposition, 
we wish to focus attention upon a number of interesting, 
difficult, and significant questions in analysis which arise 
naturally out of the functional equation technique of dynamic 
programming. Our aim la to show that this theory constitutes 
a natural extension of classical Investigations and that the 
corresponding problems are natural generallzatIona of problems 
of classical analysis. 



P-1804 

0 - 28-59 

1 


DYNAMIC PROGRAMMING AND CLASSICAL ANALYSIS 

Richard Bellman 


1. Introduction 

Over the last ten years, research in the field of dynamic 
programming has assumed many different forms. Sometimes, the 
emphasis has been upon questions of formulation in analytic 
terms and concepts, [1,2]; sometimes upon the problems of 
existence and uniqueness of solutions of the functional 
equations derived from the underlying processes, [2,5,4]; 
occasionally upon the actual analytic structure of the solu¬ 
tions of these equations, [5,6,7,8,9]; sometimes upon the 
computational aspects, [lO,ll]; and sometimes upon the appli¬ 
cations; to control processes, [2,12,15]; to trajectories of 
various types, [15,14] ; to operations research, [lp,16,17]; to 
mathematical economics, [18,19] • 

Inevitably, the result of this quasi—ergodlc behavior has 
been to Ignore a number of significant problems, and to treat 
a number of ethers In cavalier fashion. In this exposition, we 
wl3h to focus attention upon a number of Interesting, difficult, 
and significant questions in analysis which arise naturally out 
of the functional equation technique of dynamic programming. 

Our aim is to show that this theory constitutes a natural 
extension of classical, investigations and that the correspond¬ 
ing problems are natural generalizations of problems of 
classical analysis. 

As always in studying new areas, there is the hope that 



P-1804 


light throw]', lr, VhU v-1'.v.ln territory will bo refiectad back 
upon still hidden parts .f t’».- s cIcihhVc-jI domain. 

We shall restrict our attention here to deterministic 
processes, reserving for other times any discussion of more 
complex and arcane processes arising fi-ora the study of 
stochastic and adaptive processes, [2]. These new areas of 
analysis where many different sub—disciplines such as algebra, 
topology, differential equations and probability theory merge 
and lose their separate identity in f 'll-embraclng problems 
offer bountiful and boundless regions for research. The reader 
familiar with the concepts and problems discussed here can 
readily construct for himself corresponding questions involving 
uncertainty. 


?. Mnl ti.~t: Dec 1*4on PrccesHes 

Pyruir..r ; ro^i^nm ing is « mathematical theory of multistage 

decls 4 .r. prccer.sea. Txj uOttona «> r tM. theory may be P'und in 
[l] and [?J . Th t r v le:r m . w will he very much n :v 

neartir.yfui wl 4 ;: ir, the ccrt>»xt of dynamic programming. The 
rerder whc wishes, i owever, may \ -'jr . i e h 1‘ insertions of moti¬ 
vation and n’Jiir.l the p ir blem? *hr.t follow a? n ni’cdrw pulled 
cut ^ the bj;?. 

j. The Calculus of Variations 

Perhaps the most interesting and important example of a 
multistage lecislon process of continuous type is the calculus 
of variations. Consider the scalar variational problem of 
maximizing the integral 



( 1 ) 


J(y) -,/ T Hx.yjdt, 

u 0 


P-1804 

9 - 28-59 

3 


over all functions y, where x and y are related by means 
of the dl f fpr**nM «^’j.itlon 

(2) St ' 0! ’'’ y) ' X ' 0) ’ =• 

Introducing the function 


(M 


f(c,T' - Max J'y), 


we obtain from the principle of optimality, [lj , the nonlinear 
partial dl ffer rt nt 1 ?.? oq’ntior. 


( 4 ) 


jijr • Max P(c,v) ♦ G (c, v )ji- 


with the lnitla^ condition 


J 


f { C ,0 y * 0 . 


See [l ,2] . 

We ohall uso tula equation as a pivot for some of tno 
subsequent problems we ona*i preoent. 


4. Existence and Uniqueness 

The equation in ,3.4) Is derived in a formal fashion, 
much as the Euler equation la customarily obtained. The first 
problem wo pose la that of finding conditions upon the 
functions P(x,y) and Q(x,y) which ensure the existence and 
uniqueness of a solution of tno unconventional nonlinear partial 



differential equation of (3.4). 


P-1804 

9-28-89 

4 




One way to approach this problem is to employ the reeulte 
of the classical calculus of variations. These enable us to 
establish the existence of a solution of the original variation¬ 
al problem, under certain rather rigid conditions upon P(x,y) 
and 0(x,y). 

This is not a particularly satisfactory procedure for a 
number of reasons. In the first place, we would like to use 
Equation (3.4) to resolve the original variational problem. 
Secondly, we want to use analogues of (3.4) to study variational 
problems which partially or wholly escape the classical theory. 
This will be the substance of the following section. 

3. Constraints 

A question of much analytic interest, and one which in any 
case is rudely thrust upon us by a great number of feedback 
control and trajectory processes, is that of solving the 
variational problem of (3.1) and (3-2) under the additional 
constraint 

(i) |y(t;| < k, o ^ t < t. 

It is certainly surprising that a simple condition of this 
nature should so effectively stymie the usual approach, but 
nonetheless, true. For a detailed discussion, see [2,20] . 

As a result of the presence of the foregoing restriction, 
the equation of (3.4) is replaced by the equation 



P-1804 

£- 28-59 

5 


< 2 ' [ P(C ’ V) + 0 ( c ' v ) 39 - 

One might surmise that the presence of the constraint would 
simplify the investigation of existence and uniqueness of 
solution, as it certainly does the computational solution of 
equations of this nature. However, equations of the foregoing 
type have not been rigorously investigated as yet. 

6. Discrete Version—I 

A standard route to the goal of existence of a solution of 
a functional equation Involving derivatives is uy way of 
difference equations. This approach is of the utmost 
importance in connection with the numerical solution of these 

equat ior.L . 

Let us then in place of (3*4) write the recurrence relation 

(l' ~ - f . m Max r?( c ,v) ♦ . f C c " 6 f T l 

2b v L d0 

where c and T assume respectively values of the form + kS, 
lb. Kb b —* 0, 6 —♦ 0, this relation formally approaches 
that of (3.4). 

The problem now is that of determining the connection be¬ 
tween the solution of (1) and the possible solution of (3.4), 
first under the assumption that (3.4) has a solution and 
secondly in the hope of using the solution of (1) to establish 
the existence of a solution of (3.4). 

Closely associated with questions of this nature is the 


P-1804 

9 - 28 - 5 ^ 

6 


question of numerical stability of a computational algorithm 
such as (1). Finally, let us note that this is only one of a 
large set of possible discrete approximations to (3.4). 

7. Discrete Version—II 

To obtain a discrete version of a different type, we use 
an idea of some importance as far as approximation techniques 
are concerned. In place of approximating to the exact equation 
describing the process, we can employ the exact equation for an 
approximating process. 

Hence, in place of the continuous decision process 
described by the equations of (4.1) and (4.2), let us consider 
the problem of determining the sequence jy^ which 
minimizes the function 




N-l 

(1) 

J K (y) 

* JJ'tvy-JA. 



k-0 

where 

x,, and 

A 

are related by means of the equation 

(2) 

x k+l “ 

' \ ’ a '-\’ y K' A ’ *0 = °> * " 0<1 ' 2 ’ ••• 

Here 

< 

X 

1 

X 

. y k - y (xfl) . n& - t. 


Setting 


(3) f N (c) - Min J N (y), 

y 

we obtain the recurrence relation 

( 4 ) f„(c) - Max 

v 


F(c,v)& + f N_i( c ♦ . 



r-1304 

9-28-59 

7 


with 

(5 ) f 0 (c) - Max F(c,v)A. 

J v 

Recurrence relations of this nature have been quite 
successful in the obtaining of computational solutions; see 
[10,14,21] . 

Observe that as A —+ 0, the equation in (4) formally 
reduces to that of (5*4). Yet, unlike (5.1), it involves only 
one small quantity A, and no ratio of small quantities such 
as A/6. T3iis la a most important point in connection with 
numerical stability. 

A small amount of work has be^n done on the subject of the 
rorve^hnce of the solution of (4) to the solution of (3.4); 
see [22,1,23#24] . Much remains to be done. 

8. An Application 

We have mentioned above the possibility of applying ideas 
and techniques developed in the new field of multistage 
decision processes to classical equations. Let us give an 
example of this. 

Consider the partial differential equation 
(1 ) u t - uu x> u(x,0} - g(x), 

an equation which pussesces a shock discontinuity. In place of 
the usual finite difference scheme, a la (6.1), let us borrow 
the approach of (7.4^ and use the relation 



P-1804 

>- 28-^9 


(2) u(x,t ♦ A) - u(u ♦ u(x,t'iA,t), 

as an approximation to (1). 

Excellent results have been obtained, even In the 
Immediate neighborhood of the shock, using this algorithm; see 
[253* No work has been done, however, on the question of tne 
convergence of the solution of (2) to the solution of (1) as 
& —* 0, for this equation or for more general equations. For 
another application of this Idea, see [26] . 

9. Approximation In Policy Space 

Another attack on the basic problem of establlsning 
existence and uniqueness of solutions of the equation In (4.7, 
or (o.2) is by means of the method of successive approximations. 
In place of the usual approach, let us Invoke the technique of 
approximation In policy space ; see [\ , 2 ] . 

We begin by guessing an Initial policy , v^ » v^(c,T), 
and using this policy to determine a return function 
fg • fQ(c,T), by means of the linear partial differential 
equation 

(1) yr " F(c *V * 0 (c ' v o , ?c fi ' f o (c ' 0; " °* 

Having determined fQ(c,T', let us determine the function 
v^ by the condition that It maximize the function 



P-1604 

9-26-59 

9 

Using this new policy—function, let us determine the new return 
function f 1 by way of the linear partial differential equation 
b T 

(3) ■ P(c,v^) ♦ Ofc.v^^l, f 1 (c,0'i • 0. 


Continuing in this fashion, we obtain a sequence of functions 
^ f^ | and iv^J. I n view of the manner in which is deter¬ 

mined, it is clear that 




3f 0 

W 


F(c,v Q ) 


af o 

* Q ( c » v 0^^c~ 


2f, 


< F(c,v 1 j + 


Prom this, we would expect to find that f Q (c,T) < f 1 (c,T) for 
T > 0, and this is actually the case. Oenerally, the merit of 
this approximation method is that it yields monotonic behavior, 


(3) f 0 (c,T) < f 1 (c,T ; < ••• < f n (c,T) < •••. 

important problem is to determine conditions under 
which this sequence converges to a solution of (.5.4}, and, of 
course, to investigate th* application of this technique to 
more general equations. Some preliminary results for a much 
simpler problem may be found in [l] , Chapter 11. 


10. Positive Operators 

A crucial result in the foregoing procedure is the con¬ 
clusion f 1 > fg for T > 0 as a consequence of the equality 



P-1804 

9-28-59 

10 


and the inequality 
3f 

(2 J » v i ^ ♦ G(c » v l j Jc^' 

granted common initial conditions at T « 0. 

Questions of this nature are part of the theory of positive 
operators. Oiven a relation Au 2 0» where A is an operator, 
we wish to determine when tnts implies u > 0. Thi) type of 
investigation was initiate! by Caplygin, [27]; see [28,29,30j 
for further results and references. Again much remains to be 
done In this field. 

11. Quaslllnearlzation 

Let us now indie it* another aprl ‘ oat i< >n of new technlpj'*?' 
developed in the theory of dynamic programming to classical 
analysis. We have Just seen, in ^9 , that equations of the 
special form of (3.4 1 can be approached along a route which is 
not open to the equations of classical analysis. In view of 
the raonotonlclty of approx.LT.ctlo 0 , 1 '»«?ry '»aiu:-*bie analytic and 
computational aid, it may be worth devoting some effort t the 
question of converting an equation of conventional type into 
one of the form of (3.4). 

What wti are doln* Is transforming an equation arising from 
a descriptive process into one which arises from a variational 
process Tills, of course, is a familiar Idea in analysis and 
one of great power and versatility. The way in which we do it 
is, however, new. 

To give a simple example of the technique which can be 



P-1804 

9 - 28-59 

11 


used, consider the Rlccatl equation 

(1) u* - u 2 + a(t), u (0) - c . 

Write 

(2) u 2 - Max (2uv - v 2 ), 

v 

so that (1) assumes the form 

(3) u' - Max |^2uv - v 2 ♦ a(t)J , u(0) ■ e. 

Consider the related linear equation 
(4 ; IT' - 2Uv - v 2 + a(t), U(C) - c, 

whose solution we write In the form 
(5) U - T(v.t). 

We suspect that 

(5) u - Max U • Max T(v,t), 

v v 

a fact which la a consequence of the positivity of the operator 
d/dt - 2v, In the sense of §10. Since we can obtain an 
explicit representation for T(v # t) In terms of Integral 
equations, (o) furnishes an Interesting representation for the 
solution of (1). 

Oenerally, If f(u,t) Is convex as a function of u for 
0 < t < tQ# we can write 



(7) 


P-1304 
9-28-59 
1? 


f(u,t) - Max [f(v f ti ♦ ( u - vjf y (v f t)J, 

and if f (u,t) is concave, we can write 

(8) f(u,t) - Min ff(v,t) 4 (u - v)f (v,t) . 

v 

'Riese representations enable us to treat differential equations 
of the form 

(9) u' - f*u,t ), 

and, more generally, to transform functional equations of the 
form 

(10) Au - f (u,t ), 

where A la an operator, into quasllinear equations of the 

forra 

(11; Au - Max 

v 

Having obtained this form, we can employ approximation in 
policy space, and proceed as above. This representation also 
has important computational advantages. For a more detailed 
discussion, with many examples, see [j?B, 29 j- 

12. Semi—Qroup Theory 

The modem theory of semi—groups of transformations, [^l}, 
deals with functional equations of the form 


: 


f ( V , t ) 4 ( U - V i f 


( V , t )j 


T 


P-lbO* 

9 - 23^9 

19 


(1) 3T " Au ' 

where A Is a linear operator. A particular equation of thla 
type is the linear partial differential equation of (9.1). The 
functional equations associated with dynamic programming pro— 
ceases of continuous type have the form 

(2) - Max [A(v)u + b(v)] . 


In view of tne fact that (2) contains (1), we can expect 
a diffusion of knowledge in both directions. In the first place, 
in view of the quaalllnearlty of (2), we may expect that a large 
part of sem.*,.-group theory can be applied to the question of the 
existence of solutions of (2). In this way we would hope to 
obtain far stronger results in the calculus of variations than 
those currently available. In particular, we would aspire to a 
more complete theory for variational problems with constraints. 

Secondly, we may expect to utilize the results of the 
linear theory, and some of the results and techniques of 
dynamic programming, tc obtain a theory of nonlinear equations 
whicn can be written in the font, of (2). 


13. Multidimensional Variational Problems 

Consider tne problem of maximizing the Integral- 


( 1 ) 


J(u^ •Jpt (u,u x ,u y )dA, 


where the integration is over the interior of a two-dimensional 



P-1804 
9 - 28-39 
14 


region R, and the value of u Is prescribed on the boundary 
B of R, 

(2) u - g(P), P * B. 

In a procedure completely analogous to the one—dimensional case, 
we may introduce the functional 

(3) f(g(P);R) - Max J(u). 

u 


Considering a sequence of shrinking regions, it Is not 
difficult to obtain a functional equation for f(g(P);R). The 
derivatives that occur will now be functional derivatives, cf. 

Although this technique ha3 been used to obtain the 
Hadamard variational formula for the Green’s function of a 
region, and other results, [35,54,53,36], no work has been done 
on the existence and uniqueness of solutions of equations of 
this nature. 

An Interesting side problem associated with variational 
questions of this nature Is that of determining the functional 
form of f(g(P);R). Sven for the classical problem involving 
the Dirlchlet functional, 


(») 


J(u) 


4 




u>A, 


deep and subtle analysis Is required; see Osborn, ([57], where 
further references may be found. 



P-i604 

9-28—59 

15 


14 . Stability and Asymptotic Behavior 

Aa soon as the fundamental questions of existence and 
uniqueness of solution have been disposed of, we can turn to 
the deeper and more Interesting problems of the analytic 
structure of the solution. In particular, we would like to 
examine the stability properties of the solution and its 
asymptotic behavior as t -» oo . A small amount of work has 
been done in this direction, but no systematic theory has been 
constructed; see 'i7,38,59] . 

A theory of this nature can be based upon an extension of 
the present theory of positive operators; see [40,41,42]. 

15. Implicit Variational Problems 

Sc far we have considered variational processes of fixed 
duration. It is of interest to consider processes in which 
the upper limit T depends upon the policy used, and those of 
still more general nature . Those are examples of implicit 
variational problems. 

One of the most important examples of this type of problem 
la furnished by "bang—bang" control. Suppose that we have a 
system S ruled by a vector differential equation 

(1) - g(*,y), x(0) - c, 

where the components of y are subject to the restrictions 

\& / l y ± V -) 1 js ^ ^ ^ " 1 » 2, ... ,N, 


or 



P—1804 
9-2S-69 
lo 


(b) y 1 (t) - + k 1 , 1 - 

We wish to choose y In such a way as to minimize the time 
required to force tne system from the initial state c to 
nother state, say 0. 

Por the linear case, 

(3) • Ax -f y, x(Q) - c, 

a threat deal has b r r*n a^ne U31 n _j k variety >f technl :ju'*3; see 
[43,^4,4for further' references. 

10. Purt’ne r Directions 

It is easy to pose a number of additional questions of 
interest if we admit two person processes, multistage games; 
cf. [4,46], and If we Introduce stochastic and adaptive pro¬ 
cesses in general; [1,2,39*47] . 

The range of investigation Is now so broad, and so 
different in many ways from the classical tableau, that we 
feel It better to present 113 cuss ions of this nature in 
separate publications. 



P-1 ft 04 
9—26-69 

17 


RSRI-lRtfNC 




1. P. VI Ir.fij , JynamV F'ix r i’anmlng , Princeton Univ . Press, 

Princeton, ri7 77 , 19**. 

2 . -, Dyrm.. 1 r jgrammlng and Adaptive Control Pro— 

cesses , to appear. 

3. -, ”50106 functional equations In the theory of 

dynamic programming—I: functions of points and point 
transformation," Trans, Amer. Math. Soc .. vol. 80, 1955# 

PP. 51—71. 

4 . -, "Functional equations in the theory of dynamic 

programming—III," Rend. Clr. Mate. Palermo , ser. 2, 
tomo 5# 1958, pp. 1-23. 

6. -, "A variational problem with constraints In 

dynamic programming," J. Soc. Ind, Appl. Math ., vol. 4, 
1950, pp. 48—ol. ' ’ ' " 

o. -, "On a class of variational problems," Q. Appl. 

Math ., vol. 14, 1957, pp. 353-359. 

7 . -, "Functional equations in the theory of dynamic 

programming—V: positivity and quas 1-1 inearity," Proc . 

Hat. Acad. 3c1. USA , vol. 4l, 1955, PP. 74>-748. 

8 . R. Bellman, I. Ollcksberg, and 0. Gross, "On the optimal 

Inventory equation," Management Scl ., vol. 2, 1955, 
pp. 8>-104 . 

9 . R. Bellman, "On some applications of dynamic programming to 

matrix theory," Illinois J. Math ., vol. 1, 1957, 

PP. 297-501. 

10. R. Bellman and S. Dreyfus, Computational Aspects of Dynamic 

Programming, Princeton Univ. Press, Princeton, k. j 
to appear. 

11. -, "A bottleneck situation Involving 

Interdependent Industries," Naval Research Log. Q .. vol. 5 , 

1958 , pp. 21 - 28 . 

12. R. Bellman, S. Dreyfus, and R. Kalaba, "Dynamic programming 

trajectories and space travel," Proc. Fourth Symp. on 
Astronautics , Los Angeles, 1959# to appear. 

15- R. Bellman and S. Dreyfus, "An application of dynamic pro¬ 
gramming to the determination of optimal satellite 
trajectories," J. British Interplanetary Soc ., to appear. 



P-1804 

9 - 28-69 

18 


14. T. Cartalno and S. Dreyfus, "Application of dynamic pro¬ 

gramming to the airplane minimum time—to—climb problem," 
Aero, Sng. Re^ ., 19L>7 • 

15. S. Dreyfus, "a note on an Industrial replacement process," 

Operational Research Q ., December, 1957. 


10. R. Bellman, "Equipment replacement policy," J. Sue. Ind . 
Appl. Math ., vol. 3, 1955, pp. 153-136. 

17. R. Howard, Discrete Dynamic Programming Processes , Thesis, 

Massachusetts Institute of Technology, -. 

18. R. Beckwith, Analytic and Computational Aspects of Dynamic 

Programming Processes of Ttlgh Dimension , Ph.D. TheBla. 
Purdue University] 1 ) 59 . 

19. R. Bellman, I. Qlicksberg, and 0. Gross, "On the optimal 

Inventory equation," Management Science, vol. 2. 1955. 
pp. 83-104. 

20. R. Bellman, "Dynamic programming and its application to tt.e 

variational problems In mathematical economics," Proc . 
S ymp. In Calc il *s of Variations and A^licatlona, Arner. 
Math. Soc7, Chicago, April, i"$£b, pp. II 5 —published 
by McGraw-Hill Book Co., Inc., New York. 

21. R. Bellman and S. Dreyfus, "On the computational solution 

of dynamic programming processes—I: on a tactical air 
warfare model of Mengel," J. Oper. Re3 ., vol. o, 1956, 
pp. o5-7 1 . 

22. R. Bellman, "Functional equations in the theory of dynamic 

programming—VI: a direct convergence proof,' Ann. Math ., 
vol. 03 , 1957, pp . 215-225. 

23* H. Osoorn, "The problem of continuu .u programs," Pacif ic 
J. Math ., vol. o, 1950, pp. 721-/ , 51. 

24. W. Fleming, Discrete Approximations toSeme Continuous 

Dynamic Programming Process es, The RAND Corporation, 
Research Memurand in RM-15C1, June 2, 195:'. 

25. R. Bellman, I. Cherry, an! 0. M. Wing, "A note on the 

nomerlcal Integration of a class of nonlinear hyperbolic 
equations," Q. App1. Math ., vol. lo, 1958, pp. 151—163. 

2o. R. Bellman, "On the nonnegativity of solutions of the heat 
equation," B 11. Unlone Mateiaatlco , vol. 1?, *h'' T , 

; : . 529- 25- 



P-1804 

9 - 28-59 

19 


27. S. Caplygln, A Hew Method i‘or the Approximate Integration 

of Solution of Differential Equations , Princeton unlv. 
?ress, Prlnoeton, ft. J., to appear; translated by 
R. Bellman and R. Kalaba. 

28. R. Bellman, "Functional equation* In the theory of dynamic 

programming—V: positivity and quail-linearity," Proc. 

Nat. Acad. Sci. USA , vol. 41, 1955, pp. 743-746. 

29. R. Kalaba, "On nonlinear differential equations, the maxi¬ 

mum operation and monotone convergence," J. Nath, and 
Mech., vol. 6, 1959, PP- 519-574. 

30. E. F. Beckenbach and R. Bellman, Inequalities , vol. 1, 

Krgebnlsse der Math., to appear. 

31. B. Hllle and R. Phillips, Functional analysis and semi¬ 

group s , Amer. Math. Soc. tiolloq. Publ., vol. 3i, 1958. 

32. R. Bellman, "Dynamic programming and a new formalism in the 

theory of integral equations, Proc. Nat. Acad. Sci. USA , 
vol. 41, 1955, PP- 31-34. 

33* R- Bellman and H. Osborn, "Dynamic programming and the 
variation of Oreen's functions," J. Math, and Mech., 
vol. 7, 1958, pp. 81-86. 

34. R. Bellman, "Functional equations in the theory of dynamic 
programming—VIII: the variation of Oreen's functions 
for the one-dimensional case," Proc. Nat. Acad. Sci. USA, 
vol. 43, 1957, pp. 839-841. ~— 

35* P • Bellman and S. Lehman, "Functional equations in the theory 
of dynamic programming—IX: variational analysis, analytic 
continuation, and imbedding of operators," Proc. Nat. Acad. 
Sci. USA , vol. 44, 1958, pp. 905-907. 

36. -, "Functional equations in the 

theory of dynamic programming—X: resolvents, characteris¬ 
tic functions and values," Duke Math. J .. to appear. 

37. H. Osborn, "The Dlrlchlet functional," J. Math. Analysis and 

Appl ., to appear. 

38. R. Bellman, "A Markovian decision process," J. Math, and 

Mech ., vol. 6, 1957, pp. 679-t>84. ~ —* 

39. -, "Directions of Mathematical research in non¬ 

linear circuit theory," PQCT Trans ., to appear. 

40. M. Kreln and M. A. Rutman, Linear Operators Leaving 

Invariant a Cone in Banach Space , Amer. Math. Soc. 
Translation No. 26, 1950. 



P-1804 

9-28-89 

20 


41. T. S. Harris, Branching Processes , Jtrgebnlsse der Math., 

to appsar. 

42. R. Bellman, Introduction to Matrix Analysis , McGraw-Hill 

Book Co ., Inc., New Vork. 19^9* 

4). R. Bellman, 1. Glickaberg, and 0. Oross, "On the 'bang-bang' 
control problem," Q. Appl. Math ., vol. 14, 19bb, 

pp. 11-18. 

44. J. P. LaSalle, "On time optimal control systems," Proc. 

Nat. Acad. Scl. USA , vol. 4?j, 1989, pp. 87>677. 

4^. R. Bellman and J. M. Richardson, "On the application of 
dynamic programming to a class of implicit variational 
problems," Q. Appl. Math ., to appear. 

4b. Studies in Qame Theory—-!, II, III , Annals of Math. Studies, 
Princeton University Press, Princeton, N. J. 

47. R. Bellman and R. Kalaba, "A Mathematical theory of adaptive 
control processes," Proc. Nat. Acad. Scl. USA, to appear.