Chapman & Hall/CRC Mathematical and Computational Biology Series Optimal Control Applied to Biological Models C6404_FM.indd 1 3/20/07 10:21:48 AM CHAPMAN & HALL/CRC Mathematical and Computational Biology Series Aims and scope: This series aims to capture new developments and summarize what is known over the whole spectrum of mathematical and computational biology and medicine. It seeks to encourage the integration of mathematical, statistical and computational methods into biology by publishing a broad range of textbooks, reference works and handbooks. The titles included in the series are meant to appeal to students, researchers and professionals in the mathematical, statistical and computational sciences, fundamental biology and bioengineering, as well as interdisciplinary researchers involved in the field. The inclusion of concrete examples and applications, and programming techniques and examples, is highly encouraged.
Series Editors Alison M. Etheridge Department of Statistics University of Oxford Louis J. Gross Department of Ecology and Evolutionary Biology University of Tennessee Suzanne Lenhart Department of Mathematics University of Tennessee Philip K. Maini Mathematical Institute University of Oxford Shoba Ranganathan Research Institute of Biotechnology Macquarie University Hershel M.
Safer Weizmann Institute of Science Bioinformatics & Bio Computing Eberhard O. Voit The Wallace H. Couter Department of Biomedical Engineering Georgia Tech and Emory University Proposals for the series should be submitted to one of the series editors above or directly to: CRC Press, Taylor & Francis Group 24-25 Blades Court Deodar Road London SW15 2NU UK C6404_FM.indd 2 3/20/07 10:21:49 AM Published Titles Cancer Modelling and Simulation Luigi Preziosi Computational Biology: A Statistical Mechanics Perspective Ralf Blossey Computational Neuroscience: A Comprehensive Approach Jianfeng Feng Data Analysis Tools for DNA Microarrays Sorin Draghici Differential Equations and Mathematical Biology D. Sleeman Exactly Solvable Models of Biological Invasion Sergei V.
Petrovskii and Bai-Lian Li Introduction to Bioinformatics Anna Tramontano An Introduction to Systems Biology: Design Principles of Biological Circuits Uri Alon Knowledge Discovery in Proteomics Igor Jurisica and Dennis Wigle Modeling and Simulation of Capsules and Biological Cells C. Pozrikidis Niche Modeling: Predictions from Statistical Distributions David Stockwell Normal Mode Analysis: Theory and Applications to Biological and Chemical Systems Qiang Cui and Ivet Bahar Optimal Control Applied to Biological Models Suzanne Lenhart and John T. Workman Stochastic Modelling for Systems Biology Darren J. Wilkinson The Ten Most Wanted Solutions in Protein Bioinformatics Anna Tramontano C6404_FM.indd 3 3/20/07 10:21:49 AM C6404_FM.indd 4 3/20/07 10:21:49 AM Chapman & Hall/CRC Mathematical and Computational Biology Series Optimal Control Applied to Biological Models Suzanne Lenhart John T.
Workman Boca Raton London New York Chapman & Hall/CRC is an imprint of the Taylor & Francis Group, an informa business C6404_FM.indd 5 3/20/07 10:21:49 AM CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2007 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U. Government works Version Date: 20130920 International Standard Book Number-13: 978-1-4200-1141-8 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained.
If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U. Copyright Law, no part of this book may be reprinted, reproduced, transmit- ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.
com (http://www.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.com and the CRC Press Web site at http://www.com Contents Preface xi 1 Basic Optimal Control Problems 1 1.2 The Basic Problem and Necessary Conditions .3 Pontryagin’s Maximum Principle. 18 2 Existence and Other Solution Properties 21 2.1 Existence and Uniqueness Results .2 Interpretation of the Adjoint .3 Principle of Optimality .4 The Hamiltonian and Autonomous Problems. 35 3 State Conditions at the Final Time 37 3.2 States with Fixed Endpoints.
46 4 Forward-Backward Sweep Method 49 5 Lab 1: Introductory Example 57 6 Lab 2: Mold and Fungicide 63 7 Lab 3: Bacteria 67 8 Bounded Controls 71 8. 83 9 Lab 4: Bounded Case 85 10 Lab 5: Cancer 89 11 Lab 6: Fish Harvesting 93 viii 12 Optimal Control of Several Variables 97 12.2 Linear Quadratic Regulator Problems .3 Higher Order Differential Equations. 113 13 Lab 7: Epidemic Model 117 14 Lab 8: HIV Treatment 123 15 Lab 9: Bear Populations 129 16 Lab 10: Glucose Model 135 17 Linear Dependence on the Control 139 17.1 Bang-Bang Controls. 151 18 Lab 11: Timber Harvesting 153 19 Lab 12: Bioreactor 157 20 Free Terminal Time Problems 163 20.2 Time Optimal Control.
173 21 Adapted Forward-Backward Sweep 175 21.2 One State with Fixed Endpoints .3 Nonlinear Payoff Terms .4 Free Terminal Time. 187 22 Lab 13: Predator-Prey Model 189 23 Discrete Time Models 193 23. 202 24 Lab 14: Invasive Plant Species 205 ix 25 Partial Differential Equation Models 211 25.1 Existence of an Optimal Control .2 Sensitivities and Necessary Conditions .3 Uniqueness of the Optimal Control .7 Predator-Prey Example .9 Controlling Boundary Terms. 234 26 Other Approaches and Extensions 237 References 245 Index 259 Preface Consider a system in some application, where the dynamics are captured by a model, whether it be ordinary differential equations (ODEs), partial differ- ential equations (PDEs), or discrete difference equations.
Suppose also this system has a variable, or variables, which can be controlled from the outside. The question which naturally arises is how exactly to control this element in order to produce the “best” outcome, as measured by some predetermined goal or goals. The mathematical theory behind answering these questions, often called optimal control theory or dynamic optimization, has found ap- plication in a myriad of fields, from the biological sciences, to economics, to business and management, to physics and engineering. The goal of this text is two fold.
First, we wish to present the reader with an introductory, but thorough, development of the mathematical aspects of optimal control theory. This is done in a “graded” way, as the most basic prob- lem, with a continuous time ODE, is examined in Chapter 1, and increasingly more complicated problems are handled as the book progresses. This includes variations of the initial conditions, imposed bounds on the control, multiple states and controls, linear dependence on the control, and free terminal time. Optimal control of discrete systems and optimal control of partial differential equations are also introduced.
The second goal is to give the reader an insight into application of optimal control theory to biological models. Several different kinds of applications are presented here, including disease models of immunology and epidemic types, management decisions in harvesting and resource allocation models, and more. These are presented in the interactive “lab” sections, which we feel is a novel feature of this text. The MATLAB codes on which the labs are based are included, in addition to a user-friendly interface, which will allow everyone, even those with no prior MATLAB knowledge, to access them.
The underlying numerical methods are also developed in the text. This book is designed for use as a textbook for advanced undergraduate or beginning graduate students. It would be suitable for a one-semester course. It can also be used by anyone who wants to learn optimal control theory for application to specific models.
Mathematically, only a basic knowledge of multi-variable calculus and simple ordinary differential equations is needed for the bulk of the text. Some prior knowledge of PDEs is required for the (optional) chapter on this subject. The reader should also be familiar with mathematical models and how they are used. This book is not intended as a course in mathematical modeling.
xii Each so-called “theory” chapter has several fully-worked examples and ends with a group of exercises. There are also, throughout the book, more open- ended and thought provoking questions dealing with specific models or appli- cations. The reader is advised to take advantage of both kinds of exercises. We view this book as an introduction; the last chapter provides some infor- mation about more advanced topics.
We have also tried to provide references for further reading. This includes papers and other texts where one can find additional information on theoretical, numerical, or biological questions. We recall the impact of the tools of dynamic programming on the field of behavioral ecology resulting from the work of Clark, Mangel, Houston, and McNamara [34, 86, 136]. We hope that some biologists will consider using the tools introduced here for new applications.
The idea for this book came while working on materials for the short course Optimal Control Theory in Application to Biology. This short course, spon- sored by the National Institutes of Health, took place at the University of Tennessee in the summer of 2003. The authors would like to take this opportunity to thank several people who have helped immensely during the preparation of this book: Chuck Collins, for his numerical guidance and all our chats; Mike Saum and Hem Raj Joshi, for their technical expertise; Elsa Schaefer and Lou Gross, for their helpful suggestions; and Peter Andreae, Wandi Ding, Renee Fister, Elizabeth Martin, Vladimir Protopopescu, and Raj Soni for their help in various ways. We would also like to acknowledge the many authors, on whose work several of the examples and labs are based.
To download the MATLAB m-files needed for the labs, go to www.edu/∼lenhart/mfiles. Send any questions or comments about this book to lenhart@math. Suzanne Lenhart University of Tennessee Oak Ridge National Laboratory and John T. Workman Cornell University Chapter 1 Basic Optimal Control Problems We present a motivating idea of optimal control theory in a classic applica- tion from King and Roughgarden [104] on allocation between vegetative and reproductive growth for annual plants.
This plant growth model formulated by Cohen [36] divides the plant into two parts: the vegetative part, consist- ing of leaves, stems, and roots, and the reproductive part. The products of photosynthesis (growth) are partitioned into these parts, and the rate of pho- tosynthesis is assumed to be proportional to the weight of the vegetative part. Let x1 (t) be the weight of the vegetative part at time t and x2 (t) the weight of the reproductive part. Consider the following ordinary differential equation model: x01 (t) = u(t)x1 (t), x02 (t) = (1 − u(t))x2 (t), 0 ≤ u(t) ≤ 1, x1 (0) > 0, x2 (0) ≥ 0, where the function u(t) is the fraction of the photosynthate partitioned to vegetative growth.
The natural evolution of the plant should encourage max- imal growth of the reproductive part in order to ensure effective reproduction. Therefore, the goal is to find a partitioning pattern control u(t) which maxi- mizes the functional Z T ln(x2 (t)) dt. 0 The maximum season length is the upper bound T on the time interval, and it is assumed that all season lengths from zero to a fixed maximum have equal probability of occurrence.