PART III THE CONVERGENCE OF MANAGEMENT, POLITICS, AND LAW IN THE PUBLIC SECTOR Chapter 8 Policy Analysis and Implementation Evaluation Chapter 9 Regulatory Administration: An Illustration of Management, Politics, and Law in the Public Sector 361 CHAPTER 8 POLICY ANALYSIS AND IMPLEMENTATION EVALUATION Key Learning Objectives 1. Know the difference between policy outputs and policy outcomes or impacts. Understand the three different research designs for analyzing policy outcomes or impacts. Understand the managerial, political, and legal perspectives on evaluating policy implementation.
362 Chapter 8 Policy Analysis and Implementation Evaluation 363 Public administration’s myriad impacts on the economy and society have fostered concern about how well public policies work and how their imple- mentation could be improved. This chapter focuses on two ways of judging policies: policy analysis and implementation evaluation. Policy analysis can be prospective or retrospective. It seeks to determine how social, economic, and other problems will be affected by different governmental policy choices.
It also considers the extent to which a policy achieves its objectives and why it is successful or falls short. Implementation evaluation focuses on whether implementation maximizes appropriate values. The traditional managerial, political, and legal perspectives tend to agree that implementation can be prob- lematic if it gives too much discretionary authority to individual administra- tors. However, the new public management favors broad discretion.
Each of these perspectives may be more suitable for evaluating policy in a particu- lar area of public administration, such as overhead administrative functions, sociotherapy, or regulation. Public administration is an activist part of government. It is a means by which government seeks to intervene in aspects of the economy, society, and polity. When called on to be the arm of governmental intervention in these spheres of life, public administrators are required to implement policy.
The question of how successfully public policy is being implemented inevitably arises. And, if the answer is, “Not successfully enough!” is the problem with the implementation, the policy, or both? In essence, seeking answers to such questions is policy analysis and implementation evaluation. However, in an age of hollowed out government, these questions are increasingly difficult to answer as nonprofit and for-profit organizations are increasingly being con- tracted by government to deliver public services. Public administrators not only need to be substantive policy experts, they must also be good at contract management.
➻ THE GROWING CONCERN WITH POLICY ANALYSIS Although these questions seem normal enough, systematic policy analysis became a standard public administrative function only in the 1960s and 1970s. In part this was due to a shift in the nature of administrative inter- vention in the 1960s that made public administration more salient in the workplace, in neighborhoods, in families, and in society. President Lyndon Johnson’s “Great Society” program, for instance, rested on the premise that public administration could intervene successfully in a wide range of aspects of life to promote greater equality of opportunity among the citizenry. How- ever, assessments of implementation tend to depend on one’s perspectives.
What one person considers a service provided by public administration, another may consider a constraint. Whereas an employee may view pub- lic administrative intervention to protect him or her from toxic substances in the workplace as a valuable governmental service, the employer may view it as administrative interference in legitimate business considerations. 364 Part III The Convergence of Management, Politics, and Law in the Public Sector As public administration provides more services to people, it also engages in more extensive regulatory activities. By the 1970s it was evident that many individuals and groups thought that the administrative state had gone too far in regulating their activities.
Some policies for broad social change, conceived in hope in the 1960s, appeared to fail. Perhaps even worse, they seemed to linger on well after their inadequacies were widely understood. In truth, many of the policies being implemented by public administrators were established by legislatures, but nonetheless opposi- tion often focused on “the bureaucrats.” Public administrative intervention in the economy, society, and polity makes policy analysis more salient. The more resources the administrative state consumes, and the more it intervenes in our lives, the more we want it to adopt successful strategies and to know about the effects it is having.
The growing desire to evaluate policy design and implementation coincided with the rapid development of analytic techniques, especially in the social sciences, that could be used to assess the impacts of administrative interventions and ascertain the costs and benefits associated with public administrative implementation of policies. In one sense, the basic question is at least as old as Woodrow Wilson’s call in the 1880s for a new study of administration that could “discover, first, what government can properly and successfully do.”1 The development of new analytical approaches, statistical techniques, and social scientific methodolo- gies gave impetus to the development of public policy analysis as an area of study and applied research. Some broad aspects of these methodologies are outlined later in this chapter. By way of introduction, it should also be mentioned that in a practical sense retrospective policy analysis was strengthened by a number of related administrative developments.
Among these developments were a congres- sional requirement that in many policy areas a percentage of a program’s budget be set aside for program evaluation; the enactment of the Freedom of Information Act of 1966, which facilitated greater public access to infor- mation that makes policy analysis feasible; the development of program budgeting; and the growing tendency to place “sunset” clauses in enabling legislation. Policy analysis can help by showing whether a program is having the intended impact in a cost-effective manner and with a favorable benefit- cost ratio. By the 1980s, policy analysis had become central to much federal rule making as well. ➻ APPROACHES TO ANALYZING PUBLIC POLICY It is sometimes useful to distinguish between two aspects of public policy.
One is called policy output, the other policy impact or policy outcome. They are linked, but they can be separated for analytical reasons to avoid con- fusion.1 for a description of the different components of this linkage.) Chapter 8 Policy Analysis and Implementation Evaluation 365 8.1 PROGRAM LOGIC MODEL logic model is a diagram of the relationship between the factors of a performance measurement A system. It shows a linear relationship between inputs, activities, outputs, and outcomes. The elements of these boxes are defined as follows: Inputs Activities Outputs Outcomes Inputs: These are the raw materials of a program or service.
Examples include employee time, space to run a substance abuse program, or the equipment and material needed to maintain a public bicycle/jogging trail. Activities: This element of the logic model captures the workload of the program. For example, this can include the number of clients who enter a substance abuse program or the number of bicycle/ jogging trail miles that are in need of repair. Outputs: These are the things that the program has done.
Output measures capture the amount of goods or services actually supplied. For example, such measures could include the number of clients that completed the substance abuse program or the number of bicycle/jogging trail miles actually repaired. Outcomes: As Harry Hatry defines outcomes, they are “the events, occurrences, or changes in conditions, behaviors, or attitudes that indicate progress toward achievement of the mission and objectives of the program.” Outcomes are not what the program did, but the impacts of the program. These measures get at how well the organization is achieving its mission.
Continuing with our examples, an outcome for the substance abuse program could be the number of patients that continue to be clean and sober three months after leaving the program. For the bicycle/jogging trail example, it could be the increase in usage after the trails have been repaired, or taking a longer time frame, the extent to which, if any, the health of users improves or is better than that of nonusers. Obviously, outcomes depend on a variety of factors over which public administrators may have little control. For example, with respect to the trail, usage may depend partly on weather.
Source: Adapted from Harry P. Hatry, Performance Measurement: Getting Results (Washington, DC: Urban Institute Press, 1999), 12–15. The formulation of public policy involves establishing the objectives to be attained and at least sketchily outlining the general means to be used in seeking to achieve these objectives. For instance, to return to a familiar exam- ple, if a legislature decides that equal employment opportunity is a desirable objective, it may enact a statute intended to regulate the behavior of employers to facilitate the achievement of this aim.
The statute may also vest responsibil- ity for implementing or enforcing its provisions in an administrative agency, such as an equal employment opportunity commission. Presumably, if the leg- islature is serious in its intentions, it will allocate sufficient financial and per- sonnel resources to the administering agency and also grant it the legal powers to accomplish its mission. The formulation of public policy is often highly politicized and may involve hotly contested elections, votes in the legislature, and extensive 366 Part III The Convergence of Management, Politics, and Law in the Public Sector lobbying activities. The outgrowth of policy formulation is the policy output—that is, an official statement of governmental intent, delineation of powers and methods, and allocation of resources.
Policy outputs can involve tangible and/or symbolic activity. Statutes, congressional resolu- tions, presidential proclamations, and the allocation of staff and funds are policy outputs. It is widely accepted that public administrators also make policy outputs, such as rules and strategic plans. The creation of policy outputs by administra- tive agencies violates a major tenet of the traditional managerial approach—that administration be separate from politics.
The new public management (NPM) also focuses on “how government should work, not what it should do.”2 Policy outputs are central to politics and administration. They are statements of the goals of the polity and are prerequisites to the attainment of these goals through administrative action. However, from the perspective of policy analysis, it is crucial not to confuse policy outputs with policy out- comes. The outputs do not tell us much about performance or the achieve- ment of stated objectives.
The outputs are essentially activities. It is hoped that they are positively related to the effective achievement of a policy objec- tive. But today, only a naive political observer would assume that a govern- mental purpose is achieved because a statute is enacted, an administrative agency is empowered, or funds are spent. Too much has been learned about the limits of governmental action to assume that the output necessarily has the intended outcome.
Policy outcomes, by contrast, are concerned with performance: What effect is the policy output having on the intended target? Is the objective being achieved? If not, why not? If so, is achievement related to administration? At what cost? With what side effects? These are the types of questions that are most pertinent when we engage in policy analysis with regard to the activi- ties of public administrative agencies. Unfortunately, the answers we seek are sometimes elusive. This is due partly to several factors: the limitations of analytical techniques, problems of measurement, lack of information, and the complexity of so many governmental programs. Outcome Analysis Before the outcome of a policy can be ascertained, it is necessary to identify the content of that policy in operational terms.
In other words, the policy must have some specific content that lends itself to observation. Many poli- cies do that. For instance, a policy of reducing taxes to stimulate economic growth is something that can be analyzed. But many policies are too vague in their objectives or too resistant to measurement to be systematically observed.
Policies to promote some aspects of the public interest or justice, where there are competing visions of what the public interest is or justice requires, are examples.