4. Efficiency and Effectiveness:

The discussion of these two concepts was outlined is Lesson 1 and is provided again here as a review.

The concept of effectiveness can be seen as an emphasis on whether or not a given policy was able to get the job done. The primary question asked when attempting to determine an evaluation of effectiveness is: Did a given policy achieve its stated objectives? Theodoulou and Kofinis believe that the concept of ìeffectiveness helps define whether a specific program or service is achieving the intended effect and degree of change in behavior sought by the public policy (2004, p. 17).î Of course a fundamental assumption underlying the concept of effectiveness is that the stated objectives of a given public policy actually address the problem for which the policy was created. On the surface this assumption seems obvious, however many problems especially social problems ìhave no definitive formulation and hence no agreed-upon criteria to tell when a solution has been found; the choice of a definition of a problem, in fact typically determines its ësolutioní (Harmon and Mayer, 1986, p. 9).î Many problems faced by institutional policy actors are social in nature and clear cause and effect relationships are difficult to determine and more often than not vehemently disputed. For example, poverty, crime, education, civil rights, immigration, and the environment are just a few of the multitude of policy areas where vigorous and often rancorous debate over the nature of problem identification and policy alternatives is routine. While the academic definition of effectiveness appears to be fairly straightforward and unambiguous, its application can be anything but unequivocal.

Efficiency, on the other hand, ìcan be understood in terms of whether a government program or service is operating at the most optimal level in terms of resources ñ such as time, dollars, or human resources. Put differently, efficiency highlights the importance of whether programs and services are wasting resources (Theodoulou and Kofinis, 2004, p. 17).î Whether in oneís personal life, or in the business world, or in government all resources are finite, consequently, it is a normative desire to minimize waste and to get the most benefit from the expenditure of those limited resources.

While efficiency and effectiveness represent two separate and distinct evaluation criterion they are not mutually exclusive. For example, a U.S. GAO report discussing Foreign Food Aid indicates that inadequately planned food and transportation procurement procedures resulted in increased logistics costs thereby reducing the quantity, quality, and timeliness of delivery of food aid to stricken areas throughout the world (U.S. GAO, Oct. 2007). Unfortunately, this is an excellent example of how inefficient procedures reduce the amount of intended aid by increasing the cost of service delivery. Additionally, the GAO report indicates that aid workers have experienced difficulty in effectively delivering the food aid that they do receive via the inefficient delivery system. For example, in many cases throughout the world the target population can be found in war torn areas such as Darfur, Sudan, where 460,000 refugees were unable to receive food aid in July 2006 due to excessive violence. This is a case where U.S. humanitarian organizations were ineffective in the attainment of their goals to deliver even the limited food aid that was available to those in need.

5. Values and Evaluation:

What an individual experiences throughout a lifetime of perceptions and experiences creates a strong set of values and beliefs about the world around them. However, this awareness is tempered by limited and incomplete knowledge, but this limitation is frequently unacknowledged by most policy actors throughout the policy-making process. Additionally, value systems are not consistent between individuals, even between individuals who have similar experiences and perceptions. What this means is that each evaluator and reviewer filters evaluation data through their own personal set of values. Organizational members and members of the various professions share similar values all of which can come into conflict with each other and with the values of the client and the public. ìAssessing a policy is thus not a simple matter of relating a set of known facts about outcomes to a given set of values. As in all aspects of the policy process, the values themselves may be the major source of conflict, and rational argumentation and policy analysis merely the ammunition (Peters, 2007, p. 173).î

6. Politics:

A significant obstacle to the fair and unbiased evaluation of any policy or program is the political context within which the evaluation is conducted. There will always be political interests who support the findings of an evaluation and those who oppose them regardless of how objectively accurate the data may be. Those who support a given policy or program tend to be supportive of positive evaluations and contest negative findings while those who oppose a specific policy or program tend to endorse negative evaluation results and dispute positive findings. Additionally, many evaluations are performed on short notice, or are undertaken for more ulterior motives such as to provide ìempiricalî evidence lending support for decisions that have already been made. The underlying consideration with this evaluation impediment is the realization that frequently evaluations are conducted for many reasons other than the basic altruistic purpose of ensuring the effective and efficient execution of policy goals.

7. Increasing Requirements for Evaluation:

The increased emphasis for greater government performance and accountability, which has escalated significantly over the past 20 years, has concurrently intensified the demand for program and policy evaluation. Unfortunately, this increased demand for ìobjectiveî evaluation data reinforces many of the evaluation impediments highlighted above. For example, the testing requirements of the ìNo Child Left Behind Act (NCLB)î have generated considerable debate as to whether or not the test scores required by the program actually represent any indication of improved teaching standards and performance or whether they are even correlated to any increased level of learning on the part of the students. Many opponents of the testing requirements of NCLB state that the educational concentration of the school year is now narrowly focused on the three tested subjects at the expense of other important educational programs such as science, philosophy, history, and the arts. These opponents also emphasize the old adage that states: ìwhat gets measured, gets done.î The point of course is this, just because some output variable is being measured does not mean that the variable has any relationship to the root cause of the public problem or that it is even an accurate representation of program success. In the case of the NCLB the real problem is not poor test scores, it is less than desired intellectual prowess. In fact, some opponents of the NCLB believe that through the limited emphasis on reading, writing, and arithmetic skills, that the NCLB program may in fact be degrading the cognitive capability of future generations of American citizens.