Texto 
Rev. Acad. Canar. Cienc., XIX (Núms. 12), 3747 (2007) (publicado en septiembre de 2008)
CONTROL THEORETIC OPTIMIZATION IN GENERAL
DELAYDIFFERENTIAL EQUATIONS WITH SMALL DELAY
T. E. Aman* & D. K. Bhattacharya
Department of Pure Mathematics
University of Calcutta, India
ABSTRACT. The paper considers optimal control problem involving
small delay. It considers separately the role of constant ( discrete)
delay, variable delay and distributed delay in connection with optima!
control problem. In each case, necessary conditions of optimality
are obtained and it is shown that the conditions agree, in particular
case, with the corresponding known conditions when there
is no delay. Lastly examples are given for optimal control problems
involving all sorts of delay and it is shown how the aforesaid
conditions can be applied to solve such problems.
Key words and phrases: Pontryagin's maximum principie, Constrained
optima! control problems, Delay differential equation.
0. INTRODUCTION
So far as real world continuous phenomena are concerned, delaydifferential
equations are more befitting than ordinary differential equations.
·Now control problems associated with ordinary differential equations
representing dynamical systems are well studied in the literature.
But similar control problems modeled by differential equations involving
time delay are not so well studied. Hence it is essential to consider
control problems with time delay. With this in view, the present paper
suitably generalizes, in the context of all types of delaydifferential
• Gangarampur College, Dakshin Dinajpur, INDIA.
37
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
equations, the wellknown Pontryagin's maximum principle, which is an
essential tool for solving an optimal control problem.
l. GENERAL FORM OF THE CONTROL PROBLEM INVOLVING SMALL
CONSTANT DELAY
Problem 1
Let the dynamical system be given by
± = f(t ,x(t) ,x7(t) , u(t) ,u7,(t)) (1)
where t E [O, ti] e R, x(t) E Rn, u(t) E Rm, X7(t) = x(t  T) E Rn,
U71(t) = u(t  T 1) E Rm and f : R X Rn X Rn X Rm X Rm 7 Rn is a
continuously differentiable mapping.
Let the objective criteria be given by
l t¡
Maximize J(u) = (t1, x(t1)) + F(t , x(t), X7(t), u(t), U71(t))dt
o (2)
where : R x Rn ____, R, F : R X Rn X Rn x Rm x Rm ____, R are also
differentiable mappings.
Theorem 1.1
A necessary condition that (x*, u*) is a solution of the control problem
I is that there exists a costate vector p( t) E Rn such that
pT(t1) = (~)t=t1 , pT + ~~ + g~: =O and
( oH + oH ) ou = T oH f21 + 7 1 Ü oH
ou OUT! ot OXr ot our'
at u = u* where
H(t, x, XT) u, U71 ,p) = F(t , x, X7, u, U71) + PT f(t, x, XT) u, U71)
Proof. We first introduce the costate vector p(t) to construct the augmented
functional la (u), as given by
Applying the rule of integration by parts, we get
l a(u) = (t1, x(t1))  [pT x]~1 + J~1 [H(t, x, Xn u, U71,p) + pT x]dt
38
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
where the Hamiltonian function H(t, x, xT) u , U7 1,p) is given by
H(t, X, XTl U, U7 1,p) = F(t, X, XTl U, U 7 1) + pT J(t, X, XTl U, U7 1)
Now the variation of la(u) is given by
f;J = (ª 1  aH + aH + p·T = o and 1 ax tt1 ' ax ax,,.
aH U = T aH !li_ au ax,,. at
[i. e. aaHu = T aaxH,,. !atui at u = u*]
(2) If the delay occurs only in the control vector u(t) , then the above
condition becomes:
pT (t ) _ a 1 aH aH ·T _ Ü d 1  ax t=l1' ax + ax,,. + p  an
(aaHu + aau,H,.1 )ü = T1Ü aauH,,.1 at u= u*
(3) If there is no delay, then the above result agrees with the condition
given by Pontryagin's maximum principle.
Example 1.1
Consider the problem
Max J(u) = fo1 (x; + u;)dt
subject to
x = ax7 + bu7 + ex + du
Key to solve the problem
Here the Hamiltonian function H is given by
H = ( x; + u;) + p( ax7 + bu7 + ex + du)
Thus
aH 2 aH aH 2 b oH d ¡; = x7 + ap, ,, = pe, ¡; = u7 + p, ,, = p uXT ux uUT uu
Hence from the above theorem, it follows that the necessary conditions
that u* is an optimal solution of the problem are
aaHx + aaxH,,. +p=O ' p(l) =Ü ' (ªaHu + aau,,H.1 )Ü=TaaxH,,. !atl i+T'üoªuH,,.1 at u=u*
i.e. we have p +(a+ e)p = 2xn which is a linear differential equation
of first order with the terminal condition p(l) =O. Let p* be a solution
of this equation. Then from the last condition, we have the differential
equation (2u~ + pb + pd)ü = Tü(2u7 + bp) where we put p = p*. Solving
40
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
this equation we get u =u*. Finally x* is obtained as a solution of the
differential equation, x = ax,,. + bu,,. + ex + du, when u is replaced by
this value of u* obtained as above.
2. GENERAL FORM OF CONTROL PROBLEM WITH VARIABLE DELAY
Let the delay be variable, and be a function T(t) of t, which is differentiable
throughout the interval [O, t 1].
Problem 11
Let the dynamical system be given by
x = f(t, x(t), x,,.(t), u(t), u,,.1 (t)) (3)
where t E [O, t1] C R, x(t) E Rn, u(t) E Rm, x,,.(t) = x(t  T(t)) E Rn,
u,,.1 (t) = u(t  T1(t)) E Rm and J: R X Rn X Rn X Rm X Rm+ Rn is
a continuously differentiable mapping and T : [O, t 1] + (ó, ó) e R and
T1 : [O, t 1] + (ó1 , ó1) e R are also differentiable, where ó, ó1 are taken
very small.
Let the objective criteria be given by
Maximize J(u) = (t1,x(t1)) + ¡ti F(t ,x(t),x,,.(t),u(t),u,,.1 (t))dt
lo (4)
where : R X Rn + R , F : R X Rn X Rn X Rm X Rm + R are also
differentiable mappings.
Theorem 2.1
A necessary condition that (x*, u*) is a solution of the control problem
I I is that there exists a costate vector p( t) E Rn such that
p T(t 1 )  (0a2x. ) t=t¡' p· T + aaHx + axa,H(t )  Ü a t u  u * an d
(ªH + _2jf_ )u= T(t)_Qjf_fil + _QJf_ Ji+ _Q!f_ui1 + T1(t)ü aH
au au,.¡ (t) axr(t) al axr(t) au.¡ (t) au,1
at u = u* where
H(t , X, Xr(t), U, U,,1 (t) 1P) = F(t , X, Xr(t), U, U,,1 (t)) +pT J(t , X, Xr(t)i U, U,,1 (t))·
Proof. We first introduce the costate vector p(t) to construct the augmented
functional la (u), as given by
la(u) =
(t1, x(t1)) + fJ1 [F(t, X, Xr(t) 1 u, U,,1 (t)) + PT f(t , X, Xr(t), u, U,,1 (t))  xjdt
41
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
Applying the rule of integration by parts, we get
where the Hamiltonian function H(t, x, X7 (t), u, U71 (t) ,P) is defined by
H(t, x, XT(t), u, UT1(t),P) = F(t, x, XT(t), u, UT1(t)) + PT f(t , x, XT(t), u, UT¡(t))
N ow the variation of la (u) is given by
bla = (~  PT)bx f t=t1
+JOrt 1¡0""H"i' )Ui: X + ,,8 Hu Xi:T (t) + "8"H"i') Ui: U + "'8 Hu Ui:T 1(l) + p·T iu: x jdt UX UXT(t) UU UUTl (t)
Now applying Taylor's theorem with T very small, we have
XT(t)(t) = x(t  T(t)) = x(t)  T(t)x(t) = x(t)  T(t)f
as x = f. This implies that bx7 = bx  T(t)bf  fbT.
Again applying Taylor's theorem, with T1 very small, we get
UT¡(t)(t) = u(t  T1(t)) = u(t)  T1(t)ú(t)
Thus bU71 (t) =bu  T1(t)üc5t  úbT1.
bl = (0É  pT)bx 1  a ax tt¡ + JrOt i [ªaHx bx + OaXHT(t ) (bx  T(t)bf  fbT) +
~uUH bu+"u'U aTH1 ( t) (bu  T1(t)üc5t  úbT1) + i7bx]dt
= (ªrP  pT)bx J _ ax tt1 + JrOt i [ªaHx bx + _OJX}TJ(tj)_ bx + aaHu bu+ O2UT.!¡ f(t_) bu 
T(t)_OJX}TJ(tj)_ bf  T1(t)ü_OUQTlf (ft_) bt  _OJX}TJ(jt_) fbT  _OUQTflf _ÚbT1 + pTbx]dt (t)
= (ªraPx pT)bx 1 + rt1[(ªH + _J}Jj_ +p·T)bx + (ªH + 2.!f_)bu _ tt1 JO ax OXT(l) au OUTl (t)
T(t)_OJX}TJ(lj)_ bj  OaXHT(l ) fbT  OaUHTl úbT1  T1(t)üO2U.T!¡f _c5t]dt (t)
We now choose p( t) so that
PT = ía2!xÉ at t = t 1 and consider aaHx + _OJX}TJ(jt_) + p·T =O.
Then under this choice of p(t), bla equals to
Íiot1 [(ªaHu +&2u.T!l f_ )buT(t)_J}Jj_bfT1(t)ü_Qff_c5t _ _J}Jj_ fbT aH úbT1]dt (t) OXT(t) &uTl (t) OXT(t) &uTl
Assuming that la is maximum corresponding to u= u*, we have bla =O
at u= u*; thus we have,
42
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
Taking the limit as ót + O, we get
(8H + _QJf_)ú = r(t)_§.lf_fll + _§.lf_f+ + _QJf_úi + T (t)ü_QJf_ at 8u 8u.,.1 (t) 8xr(t) 8t 8x.,.(t) 8u.,.1 ( t ) 1 1 8u.,.1 (t)
u= u*
This proves the theorem. o
Sorne special cases:
If the delay terms are constants, then +=O and i 1 =O. Consequently
the last result of the Theorem 2.1 becomes
(ªH + fJH )ú = r(t) fJH aj + T1(t)ü fJH
au OUTI (t) OXT(t) at OUTI (t)
Thus the results coincide with the corresponding results of control problem
with constant delay as given in Theorem l. l.
Example 2.1
Consider the problem
subject to
Max J(u) = fo\x;(t) + u;(t))dt
. K
x = axT(t) + buT(t) +ex+ du, r(t) = 
t
Key to solve the problem
Here the Hamiltonian function H is given by
H = (x;(t) + u;(t)) + p(ax7 (t) + bU7 (t) +ex+ du)
Thus
88XHr(t ) = 2xT(t) + ap, 88XH = pe, 8U8rH(t ) = 2UT(t) + bp, 88UH = pd
43
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
Thus from the above theorem, we have the necessary conditions that u*
is an optimal solution of the problem are
&H + 2.JL + p = Ü p(l) = Ü
&x &x'T(t) '
and at u= u*
fJH fJH . fJH fJf fJH . fJH . . .. fJH ( + )u= T(t) + jT + UT1 + T1(t)u
fJu 8ur1 (t) fJxr(t) fJt fJXrt 8ur1 (t) 8ur1 (t)
This amounts to the solution of the linear differential equation of first
order given by p+ (a+c)p = 2Xr(t) with the terminal condition p(l) =
O; this can be solved. Let p* be one such solution. Then from the last
condition, we have
K
(2Ur(t) + pb + pd)u = TÜ(2Ur(t) + bp)  (2X7 (t) + 2U7 (t) + ap + bp)j{i
Then solving this differential equation, we can have u*. Finally corresponding
this u*, x* can be obtained from x = axr(t) + bur(t) + ex + du.
3. GENERAL FORM OF CONTROL PROBLEM WITH DISTRIBUTIVE
DELA Y
Problem 111
lot1
Maximize l x(u) = 1>(t ,x(t1)) + F(t ,x(t) ,u(t))dt \l(x,u) E X
o (5)
where X= {(x,u): corresponding to each u(t), x(t) is an integral curve
of
i; = j(t, x(t), u(t)) + ¡~r g(t, x(t), u(t))dt} (6)
T is the distributive delay; u(t) E Rm, x(t) E Rn are all C1functions
on [O, T]; X= Xo when t =O; J: R X Rn X Rm+ Rn, : R X Rn+ R,
F : R x Rn x Rm + R are all C1maps.
44
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
Remark l. Problems with distributive delay have vast applications in
various fields of Biomathematics. Further the mostly used form of delay
diff erential equations with distributed delay is taken as
x(t) = f(t, x(t) , u(t)) + j~00 e ktg(t, x(t), u(t))dt
This form is considered in arder to explain that the dynamics depends
on density of populations since its very inception. But practically the
dynamics starts only at some preceding instant of time and continues till
date. This is why we consider the form of delay differential equations
with distributed delay as given by
x(t) = f(t , x(t), u(t)) + ¡~T g(t, x(t), u(t))dt (7)
Theorem 3.1
A control problem with distributed delay as given by (7) can be reduced
to a problem with constant discrete delay.
Proof. Let y= y(t) E Rn, t E [O, t1] be such that y(t) = fL7 g(t, x(t), u(t))d
then from the rule of differentiation under the sign of integral, we get
y(t) = g(t, x(t), u(t))  g(t  T, x(t  T), u(t  T)).
Thus the differential equation x(t) = f(t, x, u) + ILT g(t, x, u)dt with
distributive delay reduces to the following pair x(t) = f(t, x(t), u(t)) +
y(t) = f1(t, x(t), x7(t) , u(t), u7 (t)) say, and
y(t) = g(t, x(t), u(t))g(tT, x(tT), u(tT)) = f2(t, x(t), X7(t), u(t) , u7(t)
say, where we take x = ( ~ ) , X7(t) = x(tT), U7 = u(tT). Now if we
define J = ( j~ ) l then the systeffi beCOffieS x(t) = f (t, x (t) , XT(t), u(t), U7(1
further ~(t, x(t1) ) = (t, x(t1)) and renaming the objective criteria as
l x(u), we get
l x(u) = ~(t, x(t1)) + fot1 F(t, x(t), x7 (t), u(t), uT(t))dt
Thus the problem reduces to
Maximize l x(u) = ~(t, x(t1)) + fot 1 F(t , x(t), x7 (t), u(t), uT(t))dt
SUbject to x(t) = f(t , x(t), XT(t), u(t) , UT(t)).
This is a problem with constant delay. D
45
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
Remark 2. This theorem shows that a control problem with distributed
delay can also be solved, as the corresponding problem with constant
delay is salvable.
Example 3.1
Max J(u) = fo1 (x2 + u2)dt
subject to
x = ¡~T (2x + u)dt
Proceeding as in the above theorem, we ha ve x = y = f 1 ,
y. = 2 x+u2 xTuT = f 2, F = x 2 +u2 , f = ( fJ i ) = ( 2 + y2 \
2 X u  XT  UT )
and the Hamiltonian is given by
H = x2 + u2 + P1Y + P2(2x +u  2xT  uT)
where p1 (1) =O= p2(1); also the aforesaid conditions give p1 = 2x+2p2,
P2 = Pi. Solving we get P1 = c1 cosvf2t + c2sinv/2t  y and P2
c3cosv/2t + c4sinv/2t  x. Again we have
(f)H + f)H )ü = T f)H aj+ T 1Ü f)H at u= u*
au OUT' OXT at OUT'
i.e. 2uü + Tp2ü = O.
This equation is not a very standard one. Anyway if it is not easily
solvable, sorne numerical solution may be tried. Finally with this numerical
solution u= u*, the rest of the procedure as considered in the
earlier examples may be repeated to solve this optimal control problem
with distributed delay.
This gives a rough outline to solve Problem III.
REFERENCES
l. Athans,M and P.L.Falb : Optima[ control: An introduction to the theory and its
applications. New York: MacgrawHill (1966).
2. Bernet, S : Introduction to Mathematical Control theory. Clarendon press, Oxford
(1975).
3. Fridman E. : Robust sample data H control of Linear singularly perturbed system.
IEEE Trans. On automatic control, Vol51, no3(2006).
46
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
4. Lee, E.B. and L. Markus : Foundations of optimal control theory. John Willey
and sons, Inc. New York (1968).
5. Naghshtabizi N. and Hespanha J. : Delay impulshive system. 44th Alletron conference
on communication, control and computing, Sept. 2729,2006.
6. Pontryagin, L.S. , V.S. Boltyanski, R.V. Gamkrelidze, and E.F. Ischenko : The
mathematical theory of optimal process. Wiley Interscience, New York (1962).
47
© Del documento, de los autores. Digitalización realizada por ULPGC. Biblioteca Universitaria, 2017
