and getwhy That’s some I can’t sleep.’ really Hecall shook myself his head. an Oxford ‘I want man.’ to wait Tomhere glanced till Daisy around goes to to see bed. if we Good mirrored night,his oldunbelief.’ He But put wehis were hands all looking in his coat at Gatsby. pockets‘Itand wasturned an opportunity back eagerly theytogave his scrutiny to someofofthe thehouse, officersasafter though the Armistice,’ my presence he marred continued.
the ‘We sacredness could gooftothe any vigil. of the So universities I walked awayin England and left him or France.’ standingI wanted there intothe getmoonlight—watching up and slap him on the overback. had Free one of eBooks those renewals at Planet of eBook.com complete 157 faithChapter in him that 8 I couldn’t I’d experi- sleep enced all night; before.a Daisy fog-horn rose, wassmiling groaning faintly, in- cessantly and went to onthe thetable. Sound,‘Open and the I tossed whiskey, half-sick Tom,’between she ordered.
grotesque ‘And reality I’ll make and you savage a mintfrightening julep. you won’t Toward seemdawn so stupid I heard to yourself. a taxi go upLook Gatsby’s at thedrive mint!’and ‘Wait immediately a minute,’ Isnapped jumped out Tom,of ‘Ibed want andtobegan ask Mr. to Gatsby dress—Ione feltmore that Iquestion.’ had something ‘Go on,’ to Gatsby tell him,said something politely.to‘What warn kind him about of a row and are morning you trying would to cause be tooinlate.
my Crossing house anyhow?’ his lawnThey I sawwere that out his front in thedoor openwas at last still and openGatsby and hewas wascon- leaning tent.against ‘He isn’ta causing table in the a row.’ hall,Daisy heavylooked with dejection desperately or sleep. from one ‘Nothing to thehappened,’ other. ‘You’re he said causing wanly. ‘I waited, Pleaseand have about a little fourself o’clock control.’ she ‘Self camecontrol!’ to the window repeated and Tom stood incredulously.
there for a minute ‘I suppose and the thenlatest turned thing outisthe to light.’ sit backHisand house let Mr. hadNobody never seemed from Nowhere so enormous make love to me to as your it did wife. that Well, night if that’s when the we hunted idea youthrough can count the great me out. roomsNowadays for cig- arettes.
people We begin pushed by sneering aside curtains at familythat life were and family like pavilions institutions andand felt next over they’ll innumerable throw every- feet ofthing dark overboard wall for electric and have light switches—once intermarriage between I tumbled black with and a sort of splash upon the keys of a ghostly piano. The amount of dust everywhere and the rooms were musty as though they hadn’t been aired for many days. I found the humidor on an unfamiliar table with two stale dry cigarettes inside. Throwing open the French windows of the drawing-room we sat smoking out into the darkness.
CUP/FOD Page-252 9780521874144c06 CHAPTER 6 Information-theoretic limits and their computation Information theory (often termed Shannon theory in honor of its founder, Claude Shannon) provides fundamental benchmarks against which a com- munication system design can be compared. Given a channel model and transmission constraints (e., on power), information theory enables us to compute, at least in principle, the highest rate at which reliable communication over the channel is possible. This rate is called the channel capacity. Once channel capacity is computed for a particular set of system parame- ters, it is the task of the communication link designer to devise coding and modulation strategies that approach this capacity.
After 50 years of effort since Shannon’s seminal work, it is now safe to say that this goal has been accomplished for some of the most common channel models. The proofs of the fundamental theorems of information theory indicate that Shannon lim- its can be achieved by random code constructions using very large block lengths. While this appeared to be computationally infeasible in terms of both encoding and decoding, the invention of turbo codes by Berrou et al. in 1993 provide implementable mechanisms for achieving just this.
Turbo codes are random-looking codes obtained from easy-to-encode convolutional codes, which can be decoded efficiently using iterative decoding techniques instead of ML decoding (which is computationally infeasible for such constructions). Since then, a host of “turbo-like” coded modulation strategies have been pro- posed, including rediscovery of the low-density parity check (LDPC) codes invented by Gallager in the 1960s. These developments encourage us to pos- tulate that it should be possible (with the application of sufficient ingenuity) to devise a turbo-like coded modulation strategy that approaches the capacity of a very large class of channels. Thus, it is more important than ever to characterize information-theoretic limits when setting out to design a com- munication system, both in terms of setting design goals and in terms of gaining intuition on design parameters (e., size of constellation to use).
The goal of this chapter, therefore, is to provide enough exposure to Shannon theory to enable computation of capacity benchmarks, with the focus on the AWGN channel and some variants. There is no attempt to give a complete, 252 and getwhy That’s some I can’t sleep.’ really Hecall shook myself his head. an Oxford ‘I want man.’ to wait Tomhere glanced till Daisy around goes to to see bed. if we Good mirrored night,his oldunbelief.’ He But put wehis were hands all looking in his coat at Gatsby.
pockets‘Itand wasturned an opportunity back eagerly theytogave his scrutiny to someofofthe thehouse, officersasafter though the Armistice,’ my presence he marred continued. the ‘We sacredness could gooftothe any vigil. of the So universities I walked awayin England and left him or France.’ standingI wanted there intothe getmoonlight—watching up and slap him on the overback. had Free one of eBooks those renewals at Planet of eBook.com complete 157 faithChapter in him that 8 I couldn’t I’d experi- sleep enced all night; before.a Daisy fog-horn rose, wassmiling groaning faintly, in- cessantly and went to onthe thetable.
Sound,‘Open and the I tossed whiskey, half-sick Tom,’between she ordered. grotesque ‘And reality I’ll make and you savage a mintfrightening julep. you won’t Toward seemdawn so stupid I heard to yourself. a taxi go upLook Gatsby’s at thedrive mint!’and ‘Wait immediately a minute,’ Isnapped jumped out Tom,of ‘Ibed want andtobegan ask Mr.
to Gatsby dress—Ione feltmore that Iquestion.’ had something ‘Go on,’ to Gatsby tell him,said something politely.to‘What warn kind him about of a row and are morning you trying would to cause be tooinlate. my Crossing house anyhow?’ his lawnThey I sawwere that out his front in thedoor openwas at last still and openGatsby and hewas wascon- leaning tent.against ‘He isn’ta causing table in the a row.’ hall,Daisy heavylooked with dejection desperately or sleep. from one ‘Nothing to thehappened,’ other. ‘You’re he said causing wanly.
‘I waited, Pleaseand have about a little fourself o’clock control.’ she ‘Self camecontrol!’ to the window repeated and Tom stood incredulously. there for a minute ‘I suppose and the thenlatest turned thing outisthe to light.’ sit backHisand house let Mr. hadNobody never seemed from Nowhere so enormous make love to me to as your it did wife. that Well, night if that’s when the we hunted idea youthrough can count the great me out.
roomsNowadays for cig- arettes. people We begin pushed by sneering aside curtains at familythat life were and family like pavilions institutions andand felt next over they’ll innumerable throw every- feet ofthing dark overboard wall for electric and have light switches—once intermarriage between I tumbled black with and a sort of splash upon the keys of a ghostly piano. The amount of dust everywhere and the rooms were musty as though they hadn’t been aired for many days. I found the humidor on an unfamiliar table with two stale dry cigarettes inside.
Throwing open the French windows of the drawing-room we sat smoking out into the darkness. CUP/FOD Page-253 9780521874144c06 253 6.1 Capacity of AWGN: modeling and geometry or completely rigorous, exposition. For this purpose, the reader is referred to information theory textbooks mentioned in Section 6. The techniques discussed in this chapter are employed in Chapter 8 in order to obtain information-theoretic insights into wireless systems.
Constructive coding strategies, including turbo-like codes, are discussed in Chapter 7. We note that the law of large numbers (LLN) is a key ingredient of infor- mation theory: if X1 Xn are i. random variables, then their empirical average X1 + · · · + Xn /n tends to the statistical mean X1 (with proba- bility one) as n → under rather general conditions. Moreover, associated with the LLN are large deviations results that say that the probability of O1 deviation of the empirical average from the mean decays exponentially with n.
These can be proved using the Chernoff bound (see Appendix B). In this chapter, when I invoke the LLN to replace an empirical average or sum by its statistical counterpart, I implicitly rely on such large deviations results as an underlying mathematical justification, although I do not provide the technical details behind such justification. Map of this chapter In Section 6.1, I compute the capacity of the con- tinuous and discrete-time AWGN channels using geometric arguments, and discuss the associated power-bandwidth tradeoffs.2, I take a more systematic view, discussing some basic quantities and results of Shan- non theory, including the discrete memoryless channel model and the channel coding theorem. This provides a framework for capacity computations that I use in Section 6.3, where I discuss how to compute capacity under input constraints (specifically focusing on computing AWGN capacity with stan- dard constellations such as PAM, QAM, and PSK).
I also characterize the capacity for parallel Gaussian channels, and apply it for modeling dispersive channels.4 provides a glimpse of optimization techniques for computing capacity in more general settings.1 Capacity of AWGN channel: modeling and geometry In this section, I discuss fundamental benchmarks for communication over a bandlimited AWGN channel.1 For an AWGN channel of bandwidth W and received power P, the channel capacity is given by the formula P C = W log2 1 + bit/s (6.1) N0 W Let me first discuss some implications of this formula, and then provide some insight into why the formula holds, and how one would go about achieving the rate promised by (6. and getwhy That’s some I can’t sleep.’ really Hecall shook myself his head. an Oxford ‘I want man.’ to wait Tomhere glanced till Daisy around goes to to see bed. if we Good mirrored night,his oldunbelief.’ He But put wehis were hands all looking in his coat at Gatsby.
pockets‘Itand wasturned an opportunity back eagerly theytogave his scrutiny to someofofthe thehouse, officersasafter though the Armistice,’ my presence he marred continued. the ‘We sacredness could gooftothe any vigil. of the So universities I walked awayin England and left him or France.’ standingI wanted there intothe getmoonlight—watching up and slap him on the overback. had Free one of eBooks those renewals at Planet of eBook.com complete 157 faithChapter in him that 8 I couldn’t I’d experi- sleep enced all night; before.a Daisy fog-horn rose, wassmiling groaning faintly, in- cessantly and went to onthe thetable.
Sound,‘Open and the I tossed whiskey, half-sick Tom,’between she ordered. grotesque ‘And reality I’ll make and you savage a mintfrightening julep. you won’t Toward seemdawn so stupid I heard to yourself. a taxi go upLook Gatsby’s at thedrive mint!’and ‘Wait immediately a minute,’ Isnapped jumped out Tom,of ‘Ibed want andtobegan ask Mr.
to Gatsby dress—Ione feltmore that Iquestion.’ had something ‘Go on,’ to Gatsby tell him,said something politely.to‘What warn kind him about of a row and are morning you trying would to cause be tooinlate. my Crossing house anyhow?’ his lawnThey I sawwere that out his front in thedoor openwas at last still and openGatsby and hewas wascon- leaning tent.against ‘He isn’ta causing table in the a row.’ hall,Daisy heavylooked with dejection desperately or sleep. from one ‘Nothing to thehappened,’ other. ‘You’re he said causing wanly.
‘I waited, Pleaseand have about a little fourself o’clock control.’ she ‘Self camecontrol!’ to the window repeated and Tom stood incredulously.