1 The linear regression model 1 The linear regression model: an overview 2 Functional forms of regression models 3 Qualitative explanatory variables regression models 2 The linear regression model: an overview As noted in the Preface, one of the important tools of econometrics is the linear re- gression model (LRM). In this chapter we discuss the general nature of the LRM and provide the background that will be used to illustrate the various examples discussed in this book. We do not provide proofs, for they can be found in many textbooks.1 The linear regression model The LRM in its general form may be written as: Yj =B1 +B2X2i +B3 X 3i + .1) The variable Yis known as the dependent variable. or regressand, and the X variables are known as the explanatory variables, predictors, covariates, or regressors, and U is known as a random, or stochastic, error term.
The subscript i denotes the ith observa- tion. For ease of exposition. we will write Eq.2) where BX is a short form for Bl + B 2 X 2i +B3 X 3i + .1), or its short form (1.2), is known as the population or true model. It consists of two components: (1) a deterministic component, BX, and (2) a nonsystematic, or random component, Uj.
As shown below, BX can be interpreted as the conditional mean of Yb E(Yj IX), conditional upon the given X values.2) states that an individual Yi value is equal to the mean value of the population of which he or she is a member plus or minus a random term. The concept of popula- tion is general and refers to a well-defined entity (people, firms, cities, states, coun- tries. and so on) that is the focus of a statistical or econometric analysis. if Y represents family expenditure on food and X represents family income.2) states that the food expenditure of an individual family is equal to the mean food expenditure of all the families with the same level of income, plus or minus 1 See, for example, Damodar N.
Gujarati and Dawn C. Porter, Basic Econometrics, 5th edn, McGraw-Hili, New York, 2009 (henceforward, GujaratiiPorter text); Jeffrey M. Wooldridge, Introductory Econometrics: A Modern Approach, 4th edn, South- Western, USA, 2009; James H. Stock and Mark W.
Watson, Introduction to Econometrics, 2nd edn, Pearson, Boston, 2007; and R. Carter Hill, William E. Griffiths and Guay C. Lim, Principles o/Econometrics, 3rd edn, John Wiley & Sons, New York, 2008.
2 Recall from introductory statistics that the unconditional expected, or mean, value of Ii is denoted as E(Y;. but the conditional mean, conditional on given X, is denoted asE(YIX). l The linear regression model: an overview 3 a random component that may vary from individual to individual and that may depend on several factors.1) Bl is known as the intercept and B2 to Bk are known as the slope coeffi- cients. Collectively, they are called regression coefficients or regression parameters.
In regression analysis our primary objective is to explain the mean, or average, behav- ior of Y in relation to the regressors, that is, how mean Y responds to changes in the values of the X variables. An individual Yvalue will hover around its mean value. It should be emphasized that the causal relationship between Yand the Xs, if any, should be based on the relevant theory. Each slope coefficient measures the (partial) rate of change in the mean value of Y for a unit change in the value of a regressor, holding the values of all other regressors constant, hence the adjective partial.
How many regressors are included in the model depends on the nature of the problem and will vary from problem to problem. The error term ui is a catchall for all those variables that cannot be introduced in the model for a variety of reasons. However, the average influence of these variables on the regressand is assumed to be negligible. The nature of the Y variable It is generally assumed that Yis a random variable.
It can be measured on four different scales: ratio scale, interval scale, ordinal scale, and nominal scale. Ratio scale: A ratio scale variable has three properties: (1) ratio of two variables, (2) distance between two variables, and (3) ordering of variables. On a ratio scale if, say, Ytakes two values, Y1 and Y2, the ratio Y!iY2 and the distance (Y2 - Y1) are meaning- ful quantities, as are comparisons or ordering such as Y2 :0; Y1 or Y2 ~ Y1. Most eco- nomic variables belong to this category.
Thus we can talk about whether GDP is greater this year than the last year, or whether the ratio of GDP this year to the GDP last year is greater than or less than one. Interval scale: Interval scale variables do not satisfy the first property of ratio scale variables. For example, the distance between two time periods, say, 2007 and 2000 (2007 - 2000) is meaningful, but not the ratio 2007/2000. Ordinal scale: Variables on this scale satisfy the ordering property of the ratio scale, but not the other two properties.
For examples, grading systems, such as A, B, C, or income classification, such as low income, middle income, and high income, are or- dinal scale variables, but quantities such as grade A divided by grade B are not meaningful. Nominal scale: Variables in this category do not have any of the features of the ratio scale variables. Variables such as gender, marital status, and religion are nominal scale variables. Such variables are often called dummy or categorical variables.
They are often "quantified" as 1 or 0,1 indicating the presence of an attribute and 0 indicating its absence. Thus, we can" quantify" gender as male = 1 and female = 0, or vice versa. Although most economic variables are measured on a ratio or interval scale, there are situations where ordinal scale and nominal scale variables need to be considered. That requires specialized econometric techniques that go beyond the standard LRM.
We will have several examples in Part III of this book that will illustrate some of the specialized techniques. 4 The linear regression model) The nature of X variables or regressors The regressors can also be measured on anyone of the scales we have just discussed, although in many applications the regressors are measured on ratio or interval scales. In the standard, or dassicallinear regression model (CLRM), which we will discuss shortly, it is assumed that the regressors are nonrandom, in the sense that their values are fixed in repeated sampling. As a result, our regression analysis is conditional, that is, conditional on the given values of the regressors.
We can allow the regressors to be random like the Yvariable, but in that case care needs to be exercised in the interpretation of the results. We will illustrate this point in Chapter 7 and consider it in some depth in Chapter 19. The nature of the stochastic error term, U The stochastic error term is a catchall that includes all those variables that cannot be readily quantified. It may represent variables that cannot be included in the model for lack of data availability, or errors of measurement in the data, or intrinsic randomness in human behavior.
Whatever the source of the random term u, it is assumed that the average effect of the error term on the regressand is marginal at best However, we will have more to say about this shortly. The nature of regression coefficients, the Bs In the CLRM it is assumed that the regression coefficients are some fixed numbers and not random, even though we do not know their actual values. It is the objective of re- gression analysis to estimate their values on the basis of sample data. A branch of sta- tistics known as Bayesian statistics treats the regression coefficients as random.
In this book we will not pursue the Bayesian approach to the linear regression models. a The meaning of linear regression For our purpose the term "linear" in the linear regression model refers to linearity in the regression coefficients, the Bs, and not linearity in the Y and X variables. For in- stance, the Y and X variables can be logarithmic (e. In X 2 ), or reciprocal (l/Xa) or raised to a power (e.
Xi), where In stands for natural logarithm, that is, logarithm to the base e. 4 Linearity in the B coefficients means that they are not raised to any power (e. Bi) or are divided by other coefficients (e. B2IBa) or transformed, such as In B 4.
There are occasions where we may have to consider regression models that are not linear in the regression coefficients. 5 a Consult, for instance, Gary Koop, Bayesian Econometrics, John Wiley & Sons, West Sussex, England, 200a. 4 By contrast, logarithm to base 10 is called common log. But there is a fixed relationship between the common and natural logs, which is: Inc X = 2.
5 Since this is a specialized topic requiring advanced mathematics, we will not cover it in this book. But for an accessible discussion, see Gujarati/Porter, op cit. l The linear regression model: an overview 5 1.2 The nature and sources of data To conduct regression analysis, we need data. There are generally three types of data that are available for analysis: (1) time series, (2) cross-sectional, and (3) pooled or panel (a special kind of pooled data).
Time series data A time series is a set of observations that a variable takes at different times, such as daily (e. stock prices, weather reports), weekly (e. money supply), monthly (e. the unemployment rate; the consumer price index CPI), quarterly (e.
government budgets), quinquenially or every five years (e. the census of manu- factures), or decennially or every ten years (e. the census of population). Sometimes data are collected both quarterly and annually (e.
So-called high-frequency data are collected over an extremely short period of time. In flash trading in stock and foreign exchange markets such high-frequency data have now become common. Since successive observations in time series data may be correlated, they pose spe- cial problems for regressions involving time series data, particularly, the problem of autocorrelation. In Chapter 6 we will illustrate this problem with appropriate examples.
Time series data pose another problem, namely, that they may not be stationary. Loosely speaking, a time series data set is stationary if its mean and variance do not vary systematically over time. In Chapter 13 we examine the nature of stationary and nonstationary time series and show the special estimation problems created by the latter. If we are dealing with time series data, we will denote the observation subscript by t (e.
Cross-sectional data Cross-sectional data are data on one or more variables collected at the same point in time. Examples are the census of population conducted by the Census Bureau, opinion polls conducted by various polling organizations, and temperature at a given time in several places, to name a few. Like time series data, cross-section data have their particular problems, particularly the problem of heterogeneity. For example, if you collect data on wages in several firms in a given industry at the same point in time, heterogeneity arises because the data may contain small, medium, and large size firms with their individual characteris- tics.
We show in Chapter 5 how the size or scale effect of heterogeneous units can be taken into account. Cross-sectional data will be denoted by the subscript i (e. Yi> XJ Panel, longitudinal or micro-panel data Panel data combines features of both cross-section and time series data. For example, to estimate a production function we may have data on several firms (the cross-sec- tional aspect) over a period of time (the time series aspect).
Panel data poses several challenges for regression analysis. In Chapter 17 we present examples of panel data regression models. Panel observations will be denoted by the double subscript it (e.