net Probabilistic Methods of Signal and System Analysis THIRD EDITION www.net Probabilistic Methods of Signal and System Analysis Third Edition www.net THE OXFORD SERIES IN ELECTRICAL AND COMPUTER ENGINEERING SERIES EDITORS Adel S. Sedra, Series Editor, Electrical Engineering Michael R. Lightner, Series Editor, Computer Engineering SERIES TITLES Allen and Holberg, CMOS Analog Circuit Design Bobi,:ow, Elementary Linear Circuit Analysis, 2nd Ed. Bobrow, Fundamentals of Electrical Engineering, 2nd Ed. Campbell, The Science and Engineering of Microelectronic Fabrication Chen, Analog & Digital Control System Design Chen, Linear System Theory and Design, 3rd Ed. Chen, System and Signal Analysis, 2nd Ed. Comer, Digital Logic and State Machine Design, 3rd Ed.net Cooper and McGillem, Probabilistic Methods of Signal and System Analysis, 3rd Ed. Franco, Electric Circuits Fundr;imentals Fortney, Principles Of Electronics: Analog & Digital Granzow, Digital Transmission Lines Guru and Hiziroglu, Electric Machinery & Transformers, 2nd Ed. Boole and Boole, A Modern Short Course In Engineering Electromagnetics Jones, Introduction to Optical Fiber Communication Systems Krein, Elements of Power Electronics Kuo, Digital Control Systems, 3rd Ed. Lathi, Modern Digital and Analog Communications Systems, 3rd Ed. McGillem and Cooper, Continuous and Discrete· Signal and System Analysis, 3rd Ed. Miner, Lines and Electromagnetic Fields for Engineers Roberts and Sedra, SPICE, 2nd Ed. Roulston, An Introduction to the Physics of Semiconductor Devices Sadiku, Elements of Electromagnetics, 2nd Ed. Santina, Stubberud, and Hostetter, Digital Control System Design, 2nd Ed. Schwarz, Electromagnetics for Engineers Schwarz and Oldham, Electrical Engineering: An Introduction, 2nd Ed. Sedra and Smith, Microelectronic Circuits, 4th Ed. Stefani, Savant, Shahian, and Hostetter, Design of Feedback Control Systems, 3rd Ed. Van Valkenburg, Anµlog Filter Design Warner and Grung,i Semiconductor Device Electronics Wolovich, Automatic Control Systems Yariv, Optical Electronics in Modem Communications, 5th Ed.net CONTENTS ��� , Preface xi 1 Introduction to Probabil ity 1 1-1 Engineering Applications of Probability www.net 1-2 Random Experiments and Events 5 1-3 Definitions of Probability 7 1-4 The Relative-Frequency Approach 8 1-5 Elementary Set Theory 13 1-6 The Axiomatic Approach 19 1-7 Conditional Probability 22 1-8 Independence 27 1-9 Combined Experiments 29 1-10 Bernoulli Trials 31 1-11 Applications of Bernoulli Trials 35 Problems 38 References 50 2 Random Variables sz 2-1 Concept of a Random Variable 52 2-2 Distribution Functions 54 2-3 Density Functions 57 2-4 Mean Values and Moments 63 2-5 The Gaussian Random Variable 67 2-6 Density Functions Related to Gaussian 77 2- 7 Other Probability Density Functions 87 2-8 Conditional Probability Distribution andJ)ensity Functions 97 2-9 Examples and Applications I 02 www.net vi CONT E NTS Problems 109 References 119 3 S everal Random Variab l es 120 3-1 Two Random Variables 120 3-2 Conditional Probability-Revisited 124 3-3 Statistical Independence 130 3-4 Correlation between Random Variables 132 3-5 Density Function of the Sum of Two Random Variables 136 3-6 Probability Density Function of a Function of Two Random Variables 142 www.net 3-7 The Characteristic Function 148 Problems 152 References 158 4 El em ents of Statistics 159 4-1 'Introduction 159 4-2 Sampling Theory-The Sample Mean 160 4-3 Sampling Theory-The Sample Variance 166 4-4 Sampling Distributions and Confidence Intervals 169 4-5 Hypothesis Testing 173 4-6 Curve Fitt_ing and Linear Regression 177 4-7 Correlation between Two Sets of Data 182 Problems 184 References 188 5 Random Processes 189 5-1 Introduction 189 5-2 Continqous and Discrete Random Processes 191 5-3 Deterministic and Nondetermipistic Random Processes 194 5-4 Stationary and Nonstationary Random Processes 195 5-5 Ergodic and Nonergodic Random Processes 197 5-6 Measurement of Process Parameters 199 5-7 Smoothing Data with a Moving Window Average 203 Problems 205 References 208 www.net C O NTE NTS vii 6 Correlation Functions 209 6-1 Introduction 209 6-2 Example: Autocorrelation Function of a Binary Process 213 6-3 Properties of Autocorrelation Functions 216 6-4 Measurement of Autocorrelation Functions 220 6.5 Examples of Autocorrelation Functions 227 6-6 Crosscorrelation Functions 230 6-7 Properties of Crosscorrelation Functions 232 6-8 Examples and Applications of Crosscorrelation Functions 234 6-9 Correlation Matrices for Sampled Functions 240 Problems 245 www.net References 256 7 Spectral Density 2s1 7-1 Introduction 257 7-2 Relation of Spectral Density to the Fourier Transform 259 7-3 Properties of Spectral Density 263 7-4 Spectral Density and the Complex Frequency Plane 271 7-5 Mean-Square Values from Spectral Density 274 7-6 Relation of Spectral Density to the Autocorrelation Function 281 7-7 White Noise 287 7-8 Cross-Spectral Density 189 7-9 Autocorrelation Function Estimate of Spectral Density 292 7-10 Periodogram Estimate of Spectral Density 301 7-11 Examples and Applications of Spectral Density 309 Problems 315 References 322 8 Respo, n se of Linear Systems to Random Inputs 323 8-1 Introduction 323 8-2 Analysis in the Time Domain 324 8-3 Mean and Mean-Square Value of System Output 326 8-4 Autocorrelation Function of System Output 330 8-5 Crosscorrelation between Input and Output 335 8-6 Examples of Time-Domain System Analysis 339 8-7 Analysis in the Frequency Domain 345 8-8 Spectral Density at the System Output 346 www.net viii C O N T E NTS 8-9 Cross-Spectral Densities between Input and Output 350 8-10 Examples of Frequency-Domain Analysis 352 8-11 Nurp.erical Computation of System Output 359 Problems 368 , References 380 9 Opti m um Li near System s 381 9-1 Introduction 381 9-2 Criteria of Optimality · 382 9-3 Restrictions on the Optimum System 384 9-4 Optimization by Parameter Adjustment 385 9-5 Systems That Maximize Signal-to-Noise Ratio 395 www.net 9-6 Systems That Minimize Mean-Square Error 402 Problems 412 References 418 Appendices A Mathematical Tab l es 419 A-1 Trigonometric Identities 419 A-2 Indefinite Integrals 420 A-3 Definite Integrals 421 A-4 Fourier Transform Operations 422 A-5 Fourier Transforms 423 A-6 One-Sided Laplace Transforms. 423 B Freq uently Encount e red Probab i l ity Distri b utions 425 B-1 Discrete Probability Functions 425 B-2 Continuous Distributions 427 C B i n o m i al Coeffi cients 43 1 D Nor mal Probabil ity Distri b ution Function 432 E The Q-Function 434 -; Student's t Distri bution Function 436 G Com p uter Com p utati ons 438 www.net C O N T E NTS Ix H Table of Co rrelation Function-Spectral Density Pai rs 466 Conto u r Integration 467 Index 475 www.net PREFACE ------ The goals of the Third Edition are essentially the same as those of the earlier editions, viz., to provide an introduction to the applications of probability theory to the solution of problems arising in the analysis of signals and systems that is appropriate for engineering students at the junior or senior level. However, it may also serve graduate students and engineers as a concise review of material that they previously encountered in widely scattered sources.net This edition differs from the first and second in several respects. In this edition use of the computer is introduced both in text examples and in selected problems. The computer examples are carried out using MATLAB 1 and the problems are such that they can be handled with the Student Edition of MATLAB as well as with other computer mathematics applications. to the introduction of computer usage in solving problems involving statistics and random processe�. other changes have also been made. In particular, a number of new sections have been added, virtually all of the exercises have been modified or changed, a number of the problems have been modified, and a number of new problems have been added. Since this is an engineering text, the treatment is heuristic rather than rigorous, and the student will find many examples of the application of these concepts to engineering problems. However, it is not completely devoid of the mathematical subtleties, and considerable attention has been devoted to pointing out some of the difficulties that make a more advanced study of the subject essential if one is to master it. The authors believe that the educational process is best served by repeated exposure to difficult subject matter; this text is intended to be the first exposure to probability and random processes and, we hope, not the last. The book is not comprehensive, but deals selectively with those topics that the authors have found most useful in the solution of engineering problems. A brief discussion of some of the significant features of this book will help set the stage for a discussion of the various ways it can be used. Elementary concepts of discrete probability are introduced in Chapter 1: first from the intuitive standpoint of the relative frequency approach and then from the more rigorous standpoint of axiomatic probability. Simple examples illustrate all these concepts and are more meaningful to engineers than are the traditional examples of selecting red and white balls from urns. The concept of a random variable is introduced in Chapter 2 along with the ideas of probability distribution and density functions, mean values, and conditional probability. A significant feature of this chapter is an extensive discussion of MATLAB is the registered trademark of The MathWorks, Inc.net xii P R E FA C E. many different probability density functions and the physical situations in which they may occur. Chapter 3 extends the random variable concept to situations involving two or more random variables and introduces the concepts of statistical independence and correlation. In Chapter 4, sampling theory, as applied to statistical estimation, is considered in some detail and a thorough discussion of sample mean and sample varianoe is given. The distribution of the sample is described and the use of confidence intervals in making statistical decisions is both considered and illustrated by many examples of hypothesis testing. The problem of fitting smooth curves to experimental data is analyzed, and the use of linear regression is illustrated by practical examples. The problem of determining the correlation between data sets is examiried. A general discussion of random processes and their classification is given in Chapter 5. The emphasis here is on selecting probability models that are useful in solving engineering problems. Accordingly, a great deal of attention is devoted to the physical significance of the various process classifications, with no attempt at mathematical rigor. A unique feature of this chapter, which is www.net continued in subsequent chapters, is an introduction to the practical problem of estimating the mean of a random process from an observed sample function. The technique of smoothing data with a moving window is discussed. Properties and applications of autocorrelation and crosscorrelation functions are discussed in Chapter 6. Many examples are presented in an attempt to develop some insight into the nature of correlation functions. The important problem of estimating autocorrelation functions · is discussed in some detail and illustrated with several computer examples. Chapter 7 turns to a frequency-domain representation of random processes by introducing the concept of spectral density. Unlike most texts, which simply define spectral density as the Fourier transform of the correlation function, a more fundamental approach is adopted here iri order to bring out the physical significance of the concept. This chapter is the most difficult one in the book, but the authors believe the material should be presented in this way. Methods of estimating the spectral density from the autocorrelation function and from the periodogram are developed and illustrated with appropriate computer-based examples. The use of window functions to improve estimates is illustrated as well as the use of the computer to carry out integration of the spectral density using both the real and complex frequency representations. Chapter 8 utilizes the concepts of correlation functions and spectral density to analyze the response of linear systems to random inputs. In a sense, this chapter is a culmination of all that preceded it, and is particularly significant to engineers who must use these concepts. It contains many examples that are relevant to engineering probiems and emphasizes the need for mathematical models that are both realistic and manageable.lmtation of system output through simulation is examined and illustrated with computer examples, Chapter 9 extends the concepts of systems analysis to consider systems that are optimum in some sense. Both the Classical matched filter for known signals and the Wiener filter for random signals are considered from an elementary standpoint.
