---

Definition of Public Policy Implementation

Once the government has legitimized some form of public policy such as a law, statute, edict, rule, or regulation, the stipulations of that policy must be put into action, administered, and enforced to bring about the desired change sought by the policy-makers. This task defaults to the government executive and necessitates the designation of a government agency as having the responsibility for the new policy. Theoretically the responsible agency is given the requisite resources and authority to ensure that the new policy is carried out as intended, but in reality, this does not always occur.

As discussed in the first four lessons of this course, public policy is implemented to effect some change in the behavior of a target population and it can normally be assumed that this change will ameliorate some public problem. Therefore, it stands to reason that unless the stipulations of a given policy are actually carried out, the problem will persist. As soon as the tenets of the new policy are implemented, a detailed policy evaluation can be conducted to determine if the desired results are being obtained and if not, why not, and what needs to be changed. More will be discussed concerning policy evaluation later in this lesson.

Administrative agencies accomplish most of the day-to-day work of government therefore they have the most immediate and direct impact on the daily lives of citizens than do any other government entities (Anderson, 1990). Dye indicates that implementation ìinvolves all of the activities designed to carry out the policies enacted by the legislative branch. These activities include the creation of new organizations ñ departments, agencies, bureaus, and so on ñ or the assignment of new responsibilities to existing organizations. These organizations must translate laws into operational rules and regulations. They must hire personnel, draw up contracts, spend money, and perform tasks. All of these activities involve decisions by bureaucrats ñ decisions that determine policy (2005, p. 52).î One critical aspect of policy implementation is the high degree of discretion afforded to the bureaucrats and agency procedures to transform laws into action as outlined by Dye above. Administrative decision-making has a significant impact the determination of who receives benefits and who is restricted as a result of the implementation of any policy. Administrative decision-making also has a far reaching impact on society as a result of the promulgation of agency regulations, contracting, licensing, inspections, enforcement, adjudication, and the actual discretion for agencies to interpret their own agency rules.

 

Definition of Public Policy Implementation, continued

Peters notes that ìbureaucrats play and important role in interpreting legislation and making regulations to put it into effect; they also make important decisions while applying laws and regulations to individual cases (2007, p. 95).î For now, it is important to understand that government agencies at all levels of government are responsible for the implementation of policy, even if the actual providers of government services are private contractors. The institutional entities that are accountable to the people for the administration and outcome of public policy initiatives are administrative agencies and ultimately those elected legislative officials who enacted the policy in the first place and delegated its execution to the government bureaucracy. In the case of administrative agency rules and regulations, elected officials are still ultimately responsible to the people for the professionalism, efficiency, effectiveness, equality, and equity of the services provided by these agencies. Unfortunately, Congress and the president frequently pass vague, ambiguous, and ìloftyî legislation purposely leaving out the details which will need to be filled in by bureaucracy. This legislative technique allows both the Congress and the president to have some degree of plausible deniability if the policy is poorly implemented or if the implementation is a failure altogether. Essentially, this tendency by elected officials to refuse to take responsibility for both the enactment and implementation of laws is an abdication of legislative and executive authority, and many argue that this behavior bestows far too much power on administrative agencies by fiat.

What these few paragraphs indicate is that policy implementation is not a clear cut automatic process that occurs as soon as legislation is passed. Policy is not self-executing. In fact a ìlarge number of factors may limit the ability of a political system to put policies into effect. Rarely will all the factors affect any single policy, but all must be considered when designing a policy and attempting to translate it into real services for citizens. Any one of the factors may be sufficient to cause failure or suboptimal performance by a policy, and all may have to be in good order for the policy to work. In short, it is much easier to prevent a policy from working than it is to make it effective (Peters, 2007, p. 104).î The next few sections along with the assigned course readings will highlight the essential role of policy implementation in the overall policy-making and evaluation process. Additionally, the significance of several key actors involved with policy implementation, and some important challenges that routinely interfere with effective and efficient policy implementation will be discussed.

 

The Relationship between Policy Design and Policy Implementation

Regardless of how well intentioned, or how well formulated, or how universally supported in the adoption phase of the policy process, a public policy cannot begin to change the behavior of a target population or solve a specific public problem until someone or some organization implements the policy. Ideally, ìeach policy includes a design of how a public problem will be resolved. This design will define, in varying detail, the goal(s) of the policy, the set of policy instruments to be used, the agency responsible for implementation, possible timetables, and the target population. The point of policy design is to match the correct set of instruments with the identified problem (Theodoulou and Kofinis, 2004, pp. 168-169),î and to ultimately solve that problem. However, as we have seen throughout this course, it is highly unlikely that any adopted public policy or agency rule or regulation will ever achieve such clarity primarily because of the necessity for policy feasibility and acceptability. The competition inherent in the political stream of the policy process dictates the accommodation of many interests which inevitably dilutes policy details resulting in vague and ambiguous and often confusing legislation.

Frequently, the political give-and-take of the various policy actors in the policy process prevents a thorough understanding, and all too often a misidentification, of the true public problem at hand. Unfortunately, as solutions are developed in the policy formulation stage, improper or inadequate policy instruments are proposed that will have little if any impact. Consequently, policy-makers must be extremely cognizant of the fact that the ills of poorly designed policy cannot be miraculously healed by administrative agencies as they attempt to interpret and implement imperfect legislation. The following text block provides some key considerations that policy-makers should take into account as they work through the policy formulation process. The important point to remember is that the limited understanding of the causality of most social problems makes an in-depth evaluation of potential solutions nearly impossible. Limited understanding and knowledge coupled with the impact of diverse political considerations means that the potential for policy success is limited. Therefore, policy-makers must seriously consider the difficulties and vagaries of policy implementation if they intend to enable any policy solution to have the optimum opportunity to be as successful as bounded rationality allows:

 

The Bureaucracy ñ The Major Policy Actor of the Implementation Process

As is the case with all of the stages of the policy process there are a number of institutions and actors who are involved with policy implementation. Some of these participants are directly engaged in the actual administration and implementation of policy while others attempt to influence the direction that public policy will take as a result of how it is administered and implemented. Administrative agencies represent the former while the president, the Congress, the courts, and whole host of interest groups represent the later. The bureaucracy is delegated a significant degree of power during the implementation stage of the policy process because of its ìdiscretion in interpreting the actual intent, method, and scope of a policy decision. This means that the bureaucracy can, to some extent, redefine the intent of a policy action ñ thereby becoming de facto policy-makers in the process (Theodoulou and Kofinis, 2004, p. 171).î Since many of the specific details of public policy are defined through bureaucratic discretion, decision-making, rule making, regulation, and adjudication it is no wonder that many interest groups persevere in their efforts to mold the ultimate impact that public policy can achieve as a result of how policy is actually implemented.

 

Challenges and Pitfalls of Policy Implementation

The implementation stage of the policy process is by definition an operational phase where policy is actually translated into action with the hope of solving some public problem. Theodoulou and Kofinis identify three key challenges which they believe can routinely impede the effective implementation of public policy. Those three challenges are:

Clarity of Policy Goals:

ìClear policy or program goals help specify the ends or objectives desired from the policy action. Ideally, policies should be formulated with consideration of what the actual specific goals of the policy areÖA goal stated with clarity and specificity not only provides direction but also improves the basis by which policies can be evaluated, for accountability, efficiency, and effectiveness (2004, p. 183).î

Information Intelligence:

ìInformation intelligence refers to the strategic necessity for constant feedback as to how implementation is progressing, as well asÖpreliminary assessments of impactsÖGiven the increasing reliance on multiple bureaucracies, information and feedback is essential to gauge the degree of interagency and intergovernmental cooperation and conflict, which may be affecting implementationÖInformation intelligence requires a high level of communication and feedback not only from the agencies involvedÖbut alsoÖfrom the population affected by the policy action. Implementation can be improved only if the implementers have insight as to how a policy or program is affecting a target population (2004, p. 184).î

Strategic Planning:

ìThe purpose of strategic planning, within the context of policy implementation, is to highlight the importance of assessing the capacity of an agency to meet specific implementation tasks and goals mandated by the policy decision. Essentially, strategic planning is a tool with which the agency can evaluate its ability to achieve the goals of the policy, as well as plan for how the policy will be executed (2004, pp. 184-185).î

It is important to note that these three key implementation challenges as briefly outlined above do not represent the universe of potential problems that can be encountered during the implementation stage. Peters identifies many other pathologies and variables which can equally impede successful policy implementation. Some of the policy distracters discussed by Peters are similar to those identified by Theodoulou and Kofinis and they include: the vagaries of the legislation, the number and diversity of competing interest groups involved in the policy process, organizational disunity of the institutional actors, standard operating procedures employed by administrative agencies, poor organizational communication both internal and external, the perceived inability of administrative agencies to learn from prior experiences in a reasonably quick period of time, inter-organizational politics and conflict, and the difficulties experienced with vertical and horizontal implementation structures (Peters, 2007). The importance of the knowledge of these challenges is that it helps us to understand that policy implementation is not easy to begin with, and it can be made much more difficult if no attention or consideration is given to these potential pitfalls and the subsequent obstacles that they can engender if they are ignored.

 

Definition of Public Policy Evaluation

Once public policy has been operationalized through the formal adoption of laws, rules, or regulations, and the bureaucracy has taken action to implement the policy, some form of evaluation needs to be accomplished to determine if the policy has achieved the desired outcome or impact. Public policy represents the expenditure of limited public resources and or restrictions on certain types of individual or organizational behavior. Consequently, the public has a right to expect that their government officials are accountable for the validity, efficiency, and effectiveness of those policies. Policy evaluation is therefore an absolutely critical stage in the policy process whereby ìwe can determine whether a policyís effects are intended or unintended and whether the results are positive or negative for the target population and society as a whole (Theodoulou and Kofinis, 2004, p. 191).î In essence, policy evaluation is the process used to determine what the consequences of public policy are and what has and has not been achieved.

At first glance, policy evaluation appears to be a straightforward concept however a closer inspection of the process reveals that policy evaluation can be equally as political and divisive as any other stage of the policy process. Policy evaluation provides additional opportunities for the myriad political interest groups and policy actors to attempt to influence the life of a specific policy. Favorable evaluations of the impact of a given policy will tend to perpetuate the implementation and life-cycle while unfavorable evaluations may give rise to change or possibly policy termination. Depending on the proclivity of any interest group or policy actor, the perception of how well a policy or program is performing or being implemented can have far-reaching impact.

The retrospective analysis of any public policy or government action is bounded by a number of ìreal-world constraints, such as time, budget, ethical considerations, and policy restrictions (Theodoulou and Kofinis, 2004, p. 193),î as well as political ideologies, values, experiences, measurement instruments, goal clarity, and institutional biases. The key to understanding and interpreting the results of any policy evaluation is that some degree of bias is inherent in the process. However, this shortcoming should not prevent efforts to produce fair and unbiased policy evaluation products, at least as much as possible. The objective of policy evaluation is to discover policy flaws and to attempt to correct them given all of the limitations incumbent in the overarching policy process. ìIn its simplest form, evaluating a public program involves cataloging the goals of the program, measuring the degree to which the goals have been achieved, and, perhaps, suggesting changes that might bring the performance of the organization more in line with the stated purposes of the program (Peters, 2007, p. 163).î

 

Four Types of Policy Evaluation

Most models of policy evaluation ground their analytical perspective in the logical process used to determine the disparity between what was conceptualized by the initial policy goals and what has actually been accomplished by the policy or program as implemented. However, many other models focus their analysis on different objectives such as what is the true purpose of the evaluation, what is the role of the evaluator in the process, how broad or narrow should the scope of the evaluation be, and finally how should the evaluation be organized and conducted, for example what measurement instruments should be employed to determine success or failure. Theodoulou and Kofinis (2004) identify four generic types of the most commonly used policy evaluation typologies and they are: process evaluation, outcome evaluation, impact evaluation, and cost-benefit analysis.

 

Process Evaluation:

As its name implies this type of evaluation analyzes how well a policy or program is being administered. This type of evaluation is employed more often by program managers to determine what can be done to improve the implementation, the aspects of service delivery, of the program. It does not directly address whether or not the policy or program is achieving the desired outcome or impact on the target population.

 

Outcome Evaluation:

Theodoulou and Kofinis somewhat confuse the concepts of outputs and outcomes in their discussion of outcome evaluation. Outputs are measures of government activity such as the number of tax returns processed or the number of social security checks sent out each month. Whereas outcomes are normally considered to be the impact that a policy has on a target population, for example, did the policy produce the desired behavioral change initially sought. In this case, the authors state that outcome evaluation is concerned with outputs. For example, if the stated goal of a welfare policy is to reduce the number of people receiving welfare benefits then a determination is made to see if less people are receiving welfare benefits after program implementation then before. However, what this type of evaluation does not indicate is what happened to the people who used to receive the welfare benefits and who have been forced off the system because they are no longer eligible. Did they find employment? Did they find other means of charity? Have they migrated to a life of crime? Outcome evaluation as described by Theodoulou and Kofinis focuses more on the readily available and tangible results of policy. The actual impact of the policy is the subject of the next type of evaluation.

 

Impact Evaluation:

This type of evaluation is what is more commonly perceived as a policy evaluation. The objective of this type of evaluation is to determine whether or not a given public policy or program is in fact achieving the intended impact as visualized by the various policy actors who either supported or opposed the given policy. Using the welfare example once again as discussed above this type of evaluation would answer the question what happened to the people who used to receive the welfare benefits and who have been forced off the system because they are no longer eligible. Did they find employment? Did they find other means of charity? Have they migrated to a life of crime? ìIn comparison [with outcome evaluation], impact evaluation is concerned with assessing whether the target population is being affected in any way by the introduction and implementation of the policy. There is also concern with the impact of the program on the original problem being addressedÖfor it is important for both [policyólevel managers and policy designers] to ascertain whether target populations are appropriately receiving delivery of a program (Theodoulou and Kofinis, 2004, 194).î

 

Cost-Benefit Analysis:

Simply stated, a cost-benefit analysis is the comparison of the costs associated with a policy or program to the benefits generated by the policy. Continuing with the welfare example cited above in the two previous definitions, the tangible cost of a new welfare policy with a goal to reduce the number of welfare recipients could be accurately evaluated to include agency operating costs and the actual monetary cost of the benefits that are provided to welfare recipients. However, the cost-benefit analysis technique is controversial because it is extremely difficult to calculate the intangible costs such as those borne by those individuals who are no longer eligible to receive welfare benefits as a result of the new policy to restrict the number of people on the welfare roles. Additionally, the intangible costs borne by society such as the potential for increased crime or increased public health care costs or the long-term loss of employment opportunity and participation in the market place by those individuals who may have been able to pull themselves out of poverty if they were still eligible for welfare assistance are extremely difficult to calculate. All too often the cost-benefit analysis technique is used because actual real-world costs are easy to obtain, quantify, evaluate, and contrast against a variety of metrics or other policies or programs. Unfortunately, many intangible benefits, such as the advantages gleaned by a well-educated society, may not be readily visible for many years to come, and some intangible benefits are impossible to quantify such as the quality of life. Policy evaluators must constantly be aware that the costs and benefits used in any evaluation may not accurately, if at all, represent the real impact of a given policy or program. Instead, a cost-benefit analysis should be employed as one of several methods used to determine the efficacy or efficiency of government action.

 

Seven Important Barriers to Effective Policy Evaluation

ìAs increasing pressures are brought to bear on the public sector to perform its role more effectively and efficiently, evaluation will probably become an even greater source of conflict. Negative evaluations of a programís effectiveness and efficiency now will be more likely to lead to the programís termination than in more affluent times. The content of an evaluation, the values that are contained in it, and event he organization performing the evaluation will all affect the final assessment. Evaluation research is now a major industry involving numerous consulting firms (ëbeltway banditsí), universities, and organizations within government itself. These evaluative organizations will have their own perspectives on what is right and wrong in policy and will bring those values with them when they perform an analysis (Peters, 2007, p. 175).î

As such, Peters identifies seven important barriers to effective policy evaluation that can impede the seemingly simple but operationally difficult process of determining what actually occurred as a result of government action and more specifically determining the level of performance of a public policy of program. It is equally important for both policy evaluators and the consumers of policy evaluation to understand the basic nature of these difficulties. Specifically, anyone engaged in policy evaluation must appreciate the degree of bias and unintentional, albeit sometimes intentional, confusion that can occur as a result of the difference between what is being measured during a policy evaluation and what policy-makers thought the policy should achieve. All too often evaluators, administrators, and the public focus on measurement statistics that are easy to obtain but have no real relationship between what has been accomplished compared to what the original intent of the public policy was. The following text block identifies the seven barriers as enumerated by Peters and it is followed by a brief description of each.

1. Goal Specification and Goal Change:

As outlined in Lessons 3 and 4, problem identification and policy formulation are extremely difficult for a variety of reasons. Not the least of which include political bias, incomplete information (bounded rationality), misunderstanding of the nature of the problem, individual value bias (social construct theory), plurality of competing interests, and inadequate or inappropriate understanding of potential policy alternatives. As a result of these significant limitations, adopted policy is frequently comprised of vague language, ambiguous, contradictory, impractical, or impossible policy goals. Consequently, the administrative agency tasked with policy implementation may need to fill in the gaps, which may or may not coincide with the intentions of policy-makers. Finally, lawmakers may not provide sufficient if any resources to adequately support a policy initiative. This requires bureaucrats to make programmatic decisions, which will unequivocally impact the efficiency and effectiveness of any policy or program.

2. Measurement:

Once policy goals have theoretically been identified, communicated, and programs implemented then some type of measurement instrument must be developed to ascertain the extent to which the goals have been achieved. However, most public problems such as national defense, education, poverty, health care, crime, urban and highway planning, and environmental policy are comprised of policy goals that are extremely difficult to measure directly. Consequently, many surrogate metrics are used to circumscribe the level of effectiveness of programs. Frequently these metrics evaluate outputs such as arrest or conviction rates for example because it is nearly impossible to determine if the ìrightî criminal was arrested and convicted, which of course is the goal. If we use the welfare reform example once again we can see that the mere reduction of the number of people receiving welfare benefits because of new program limitations does not mean that poverty has been eliminated. It simply means that less people are eligible to receive benefits and less people are receiving them. Those individuals who are no longer eligible to receive benefits may very well still be living in poverty and may even been worse off then before because they no longer have government assistance.

Many other factors can also significantly impair the adequate and appropriate measurement of program and policy goals. For example, the time span that many policies require for their full impact to be felt can be a major problem when elected officials or other policy actors are looking for quick answers to support their own policy positions. The awareness of the time span problem often results in the formulation of public policy that will generate quick and measurable results in the short-term but do not necessarily address the real problem over the long haul. Other measurement problems include the inability to adequately isolate contributing variables of a major public issue such as health care. For example, in poverty stricken areas poor health may be the result of many factors such as poor nutrition, inadequate housing, minimal education, or poor sanitation in addition to a lack of access to quality health care. Therefore, public programs that simply provide some minimal level of access to medical care may have little impact on the overall health of a community because of these other present and contributing factors. However, measurement statistics will frequently focus on the increased availability to health care providers or the number of patient contacts, etc. This data may be valid statistics but it alone does not evaluate the level of success of eliminating the real problem of poor health in a particular community. The important point to remember is that voluminous measurement and statistical data does not in and of itself prove anything if the measurement instruments are evaluating indicators that are not directly related to the problem at hand.

3. Targets:

The target population whose behavior is the object of policy action is in many cases as difficult to identify and evaluate as is identifying the problem and formulating policy in the first place. ìPrograms that have significant effects on the population as a whole may not have the desired effects on the more specific target population. For example, the Medicare program was intended, in part, to benefit less-affluent older people, although all the elderly are eligible for it. However, although the health of the elderly population in general has improvedÖthe health of the neediest elderly has not improved commensurately. And as the program has been implemented, substantial coinsurance has been required, along with substantial deductibles if the insured enters a hospital, so that it is difficult for the neediest elderly citizens to participate (Peters, 2007, p. 170).î In many cases, the need to achieve political feasibility during the policy formulation and legitimation stages of the policy process results in the approval of expanded eligibility for benefits beyond the target population as indicated in the Medicare example just cited. Therefore the remedial intent of the initial policy is diffused amongst a larger group of recipients diluting the ultimate impact of the policy, and making its evaluation more complicated.

 

4. Efficiency and Effectiveness:

The discussion of these two concepts was outlined is Lesson 1 and is provided again here as a review.

The concept of effectiveness can be seen as an emphasis on whether or not a given policy was able to get the job done. The primary question asked when attempting to determine an evaluation of effectiveness is: Did a given policy achieve its stated objectives? Theodoulou and Kofinis believe that the concept of ìeffectiveness helps define whether a specific program or service is achieving the intended effect and degree of change in behavior sought by the public policy (2004, p. 17).î Of course a fundamental assumption underlying the concept of effectiveness is that the stated objectives of a given public policy actually address the problem for which the policy was created. On the surface this assumption seems obvious, however many problems especially social problems ìhave no definitive formulation and hence no agreed-upon criteria to tell when a solution has been found; the choice of a definition of a problem, in fact typically determines its ësolutioní (Harmon and Mayer, 1986, p. 9).î Many problems faced by institutional policy actors are social in nature and clear cause and effect relationships are difficult to determine and more often than not vehemently disputed. For example, poverty, crime, education, civil rights, immigration, and the environment are just a few of the multitude of policy areas where vigorous and often rancorous debate over the nature of problem identification and policy alternatives is routine. While the academic definition of effectiveness appears to be fairly straightforward and unambiguous, its application can be anything but unequivocal.

Efficiency, on the other hand, ìcan be understood in terms of whether a government program or service is operating at the most optimal level in terms of resources ñ such as time, dollars, or human resources. Put differently, efficiency highlights the importance of whether programs and services are wasting resources (Theodoulou and Kofinis, 2004, p. 17).î Whether in oneís personal life, or in the business world, or in government all resources are finite, consequently, it is a normative desire to minimize waste and to get the most benefit from the expenditure of those limited resources.

While efficiency and effectiveness represent two separate and distinct evaluation criterion they are not mutually exclusive. For example, a U.S. GAO report discussing Foreign Food Aid indicates that inadequately planned food and transportation procurement procedures resulted in increased logistics costs thereby reducing the quantity, quality, and timeliness of delivery of food aid to stricken areas throughout the world (U.S. GAO, Oct. 2007). Unfortunately, this is an excellent example of how inefficient procedures reduce the amount of intended aid by increasing the cost of service delivery. Additionally, the GAO report indicates that aid workers have experienced difficulty in effectively delivering the food aid that they do receive via the inefficient delivery system. For example, in many cases throughout the world the target population can be found in war torn areas such as Darfur, Sudan, where 460,000 refugees were unable to receive food aid in July 2006 due to excessive violence. This is a case where U.S. humanitarian organizations were ineffective in the attainment of their goals to deliver even the limited food aid that was available to those in need.

5. Values and Evaluation:

What an individual experiences throughout a lifetime of perceptions and experiences creates a strong set of values and beliefs about the world around them. However, this awareness is tempered by limited and incomplete knowledge, but this limitation is frequently unacknowledged by most policy actors throughout the policy-making process. Additionally, value systems are not consistent between individuals, even between individuals who have similar experiences and perceptions. What this means is that each evaluator and reviewer filters evaluation data through their own personal set of values. Organizational members and members of the various professions share similar values all of which can come into conflict with each other and with the values of the client and the public. ìAssessing a policy is thus not a simple matter of relating a set of known facts about outcomes to a given set of values. As in all aspects of the policy process, the values themselves may be the major source of conflict, and rational argumentation and policy analysis merely the ammunition (Peters, 2007, p. 173).î

6. Politics:

A significant obstacle to the fair and unbiased evaluation of any policy or program is the political context within which the evaluation is conducted. There will always be political interests who support the findings of an evaluation and those who oppose them regardless of how objectively accurate the data may be. Those who support a given policy or program tend to be supportive of positive evaluations and contest negative findings while those who oppose a specific policy or program tend to endorse negative evaluation results and dispute positive findings. Additionally, many evaluations are performed on short notice, or are undertaken for more ulterior motives such as to provide ìempiricalî evidence lending support for decisions that have already been made. The underlying consideration with this evaluation impediment is the realization that frequently evaluations are conducted for many reasons other than the basic altruistic purpose of ensuring the effective and efficient execution of policy goals.

7. Increasing Requirements for Evaluation:

The increased emphasis for greater government performance and accountability, which has escalated significantly over the past 20 years, has concurrently intensified the demand for program and policy evaluation. Unfortunately, this increased demand for ìobjectiveî evaluation data reinforces many of the evaluation impediments highlighted above. For example, the testing requirements of the ìNo Child Left Behind Act (NCLB)î have generated considerable debate as to whether or not the test scores required by the program actually represent any indication of improved teaching standards and performance or whether they are even correlated to any increased level of learning on the part of the students. Many opponents of the testing requirements of NCLB state that the educational concentration of the school year is now narrowly focused on the three tested subjects at the expense of other important educational programs such as science, philosophy, history, and the arts. These opponents also emphasize the old adage that states: ìwhat gets measured, gets done.î The point of course is this, just because some output variable is being measured does not mean that the variable has any relationship to the root cause of the public problem or that it is even an accurate representation of program success. In the case of the NCLB the real problem is not poor test scores, it is less than desired intellectual prowess. In fact, some opponents of the NCLB believe that through the limited emphasis on reading, writing, and arithmetic skills, that the NCLB program may in fact be degrading the cognitive capability of future generations of American citizens.

 

Essential Activities in the Evaluation Process

 

Internal and External Policy Evaluators

One of the primary decisions to be made before conducting any type of policy evaluation is to make a determination of who should conduct the evaluation; organizational members, or someone external to the organization. Each option has its own advantages and disadvantages, and as long as the consumers of the evaluation data are aware of who conducted the evaluation and are also aware of the potential limitations of each alternative then neither option has any particular advantage over the other. The specific circumstances surrounding the necessity for any evaluation will have a significant impact on which alternative will be chosen. For example, the time available, the availability of funding, the availability of subject matter expertise, receptivity to the findings of the evaluation, and the availability of administrative support.

Internal evaluations have numerous advantages particularly their low cost. Additionally, internal evaluators tend to have greater familiarity with the organization itself, the evaluated program and associated policies, the various organizational stakeholders, and of course the targeted population. The major disadvantage to the conduct of an internal evaluation is its actual and perceived bias. There is actual bias because it is human nature for organizational members to perceive program flaws through the lens of their common organizational experiences. Consequently subsequent findings may be diluted or may completely overlook significant problem areas. There is perceived bias inherent in internal evaluations because the subsequent findings will always be suspect for the very reasons just discussed for actual bias.

External evaluations also have a number of advantages the primary of which is the minimization of bias, at least internal bias. ìThe major advantage of external evaluation is that it is perceived to be impartial because evaluators supposedly have no stake in the outcome of the evaluation (Theodoulou and Kofinis, 2004, p. 197).î Of course we must always be aware that everyone has some interest and some form of bias. The key is to minimize bias as much as possible and to be aware of its existence when evaluations are being developed and the subsequent data is being analyzed. An additional advantage to using external evaluators is that they are not part of the culture of the organization, program, or policy being evaluated and their professionalism, previous experiences, education, values, and perceptions enables them to offer a totally different perspective to problem identification and potential remediation. One of the major disadvantages to the use of external evaluators is that they are costly, especially when compared to the option of conducting an internal evaluation, assuming that reasonably competent people are available within the organization. Another major disadvantage is time. External evaluations take more time because the evaluators need to immerse themselves into the organization or program to learn as much as they can about what the organization, programs, and policies that they are evaluating. Additionally, the evaluators may experience resistance from organizational members who may view them as a threat and therefore withhold critical information. Finally, the findings of external evaluations may not be well-received by stakeholders who have something to loose, or the findings may be dismissed as being superficial due to a lack of program or policy understanding.

 

Definition of Public Policy Change and Termination

One of the main reasons for doing policy evaluation in the first place is to determine the difference between policy goals and objectives and the subsequent impact that a given policy as implemented has achieved, and then to do something about it. There are essentially three options: maintain the status quo if the policy is working as planned; make adjustments to the policy, in other words make changes; or the most drastic, and rare, change of all is to terminate the program or repeal the policy. ìWhen a policy is replaced or modified in some respect or repealed in parts, then policy change has occurred. Policies are rarely maintained exactly as adopted. Change inevitably starts to occur as soon as a policy is implemented because of the intrinsic ambiguity of legislation. Thus, policies are constantly evolving and the policy cycle is an ongoing dynamic process (Theodoulou and Kofinis, 2004, p. 202).î Of course, these options represent apolitical motivations. Once politics is added to policy change or termination dynamic then a whole host of policy actor motivations can once again be seen as attempting to influence the ultimate outcome of policy.

It is important to understand that policy change does not occur in a vacuum. Proposed changes will essentially go through some variation of the preceding six stages of the policy process: problem identification, agenda-setting, policy formulation, policy adoption, policy implementation, policy evaluation and then once again back to the policy change or termination stage in a nearly never ending cycle.

 

Factors Necessitating Policy Change

ìPolicy change and termination are both troublesome endeavors for government. Policies are developed to address certain problems or issues: however, they rarely do so and are often perceived to have failedÖAlthough it is far more common for policy to change than to be terminated (Theodoulou and Kofinis, 2004, p. 209).î

 

Why Policy Termination is Rare

 

Reasons for Policy Termination

 

Summary

Once the government has legitimized some form of public policy such as a law, statute, edict, rule, or regulation, the stipulations of that policy must be put into action, administered, and enforced to bring about the desired change sought by the policy-makers. As discussed in the first four lessons of this course, public policy is implemented to effect some change in the behavior of a target population and it can normally be assumed that this change will ameliorate some public problem. Regardless of how well intentioned, or how well formulated, or how universally supported in the adoption phase of the policy process, a public policy cannot begin to change the behavior of a target population or solve a specific public problem until someone or some organization implements the policy.

Frequently, the political give-and-take of the various policy actors in the policy process prevents a thorough understanding, and all too often a misidentification, of the true public problem at hand. Unfortunately, as solutions are developed in the policy formulation stage, improper or inadequate policy instruments are proposed that will have little if any impact. Consequently, policy-makers must be extremely cognizant of the fact that the ills of poorly designed policy cannot be miraculously healed by administrative agencies as they attempt to interpret and implement imperfect legislation.

As is the case with all of the stages of the policy process there are a number of institutions and actors who are involved with policy implementation. Some of these participants are directly engaged in the actual administration and implementation of policy while others attempt to influence the direction that public policy will take as a result of how it is administered and implemented. Administrative agencies represent the former while the president, the Congress, the courts, and whole host of interest groups represent the later. The bureaucracy is delegated a significant degree of power during the implementation stage of the policy process. The implementation stage of the policy process is by definition an operational phase where policy is actually translated into action with the hope of solving some public problem. However, there are several impediments to successful policy implementation and they include:

1. Clarity of Policy Goals
2. Information Intelligence
3. Strategic Planning

Once public policy has been operationalized through the formal adoption of laws, rules, or regulations, and the bureaucracy has taken action to implement the policy, some form of evaluation needs to be accomplished to determine if the policy has achieved the desired outcome or impact. Most models of policy evaluation ground their analytical perspective in the logical process used to determine the disparity between what was conceptualized by the initial policy goals and what has actually been accomplished by the policy or program as implemented. Theodoulou and Kofinis (2004) identify four generic types of the most commonly used policy evaluation typologies and they are:

1. Process Evaluation
2. Outcome Evaluation
3. Impact Evaluation
4. Cost-benefit Analysis.

One of the primary decisions to be made before conducting any type of policy evaluation is to make a determination of who should conduct the evaluation; organizational members, or someone external to the organization. Each option has its own advantages and disadvantages, and as long as the consumers of the evaluation data are aware of who conducted the evaluation and are also aware of the potential limitations of each alternative then neither option has any particular advantage over the other.

One of the main reasons for doing policy evaluation in the first place is to determine the difference between policy goals and objectives and the subsequent impact that a given policy as implemented has achieved, and then to do something about it. There are essentially three options: maintain the status quo if the policy is working as planned; make adjustments to the policy, in other words make changes; or the most drastic, and rare, change of all is to terminate the program or repeal the policy. It is important to understand that policy change does not occur in a vacuum. Proposed changes will essentially go through some variation of the preceding six stages of the policy process: problem identification, agenda-setting, policy formulation, policy adoption, policy implementation, policy evaluation and then once again back to the policy change or termination stage in a nearly never ending cycle.