A lot is written about evaluation in education - a great deal of which is misleading and confused. Many informal educators such as youth workers and social pedagogues are suspicious of evaluation because they see it as something that is imposed from outside. It is a thing that we are asked to do; or that people impose on us. As Gitlin and Smyth (1989) comment, from its Latin origin meaning 'to strengthen' or to empower, the term evaluation has taken a numerical turn - it is now largely about the measurement of things - and in the process can easily slip into becoming an end rather than a means. In this discussion of evaluation we will be focusing on how we can bring questions of value (rather than numerical worth) back into the centre of the process. Evaluation is part and parcel of educating. To be informal educators we are constantly called upon to make judgements, to make theory, and to discern whether what is happening is for the good. We have, in Elliot W. Eisner’s words, to be connoisseurs and critics. In this piece we explore some important dimensions of this process; the theories involved; the significance of viewing ourselves as action researchers; and some issues and possibilities around evaluation in informal and community education, youth work and social pedagogy. However, first we need to spend a little bit of time on the notion of evaluation itself.
Much of the current interest in evaluation theory and practice can be directly linked to the expansion of government programmes (often described as the 'New Deal') during the 1930s in the United States and the implementation of various initiatives during the 1960s (such as Kennedy's 'War on Poverty') (see Shadish, Cork and Leviton 1991). From the 1960s-on 'evaluation' grew as an activity, a specialist field of employment with its own professional bodies, and as a body of theory. With large sums of state money flowing into new agencies (with projects and programmes often controlled or influenced by people previously excluded from such political power) officials and politicians looked to increased monitoring and review both to curb what they saw as 'abuses', and to increase the effectiveness and efficiency of their programmes. A less charitable reading would be that they were both increasingly concerned with micro-managing initiatives and in controlling the activities of new agencies and groups. Their efforts were aided in this by developments in social scientific research. Of special note here are the activities of Kurt Lewin and the interest in action research after the Second World War.
As a starter I want to offer an orienting definition:
Evaluation is the systematic exploration and judgement of working processes, experiences and outcomes. It pays special attention to aims, values, perceptions, needs and resources.
There are several things that need to be said about this.
First, evaluation entails gathering, ordering and making judgments about information in a methodical way. It is a research process.
Second, evaluation is something more than monitoring. Monitoring is largely about 'watching' or keeping track and may well involve things like performance indicators. Evaluation involves making careful judgements about the worth, significance and meaning of phenomenon.
Third, evaluation is very sophisticated. There is no simple way of making good judgements. It involves, for example, developing criteria or standards that are both meaningful and honour the work and those involved.
Fourth, evaluation operates at a number of levels. It is used to explore and judge practice and programmes and projects (see below).
Last, evaluation if it is to have any meaning must look at the people involved, the processes and any outcomes we can identify. Appreciating and getting of flavour of these involves dialogue. This makes the focus enquiry rather than measurement - although some measurement might be involved (Rowlands 1991). The result has to be an emphasis upon negotiation and consensus concerning the process of evaluation, and the conclusions reached.
Three key dimensions
Basically, evaluation is either about proving something is working or needed, or improving practice or a project (Rogers and Smith 2006). The first often arises out of our accountability to funders, managers and, crucially, the people are working with. The second is born of a wish to do what we do better. We look to evaluation as an aid to strengthen our practice, organization and programmes (Chelimsky 1997: 97-188).
To help make sense of the development of evaluation I want to explore three key dimensions or distinctions and some of the theory associated.
Programme or practice evaluation? First, it is helpful to make a distinction between programme and project evaluation, and practice evaluation. Much of the growth in evaluation has been driven by the former.
Programme and project evaluation. This form of evaluation is typically concerned with making judgements about the effectiveness, efficiency and sustainability of pieces of work. Here evaluation is essentially a management tool. Judgements are made in order to reward the agency or the workers, and/or to provide feedback so that future work can be improved or altered. The former may well be related to some form of payment by results such as the giving of bonuses for ‘successful’ activities, the invoking of penalty clauses for those deemed not to have met the objectives set for it and to decisions about giving further funding. The latter is important and necessary for the development of work.
Practice evaluation. This form of evaluation is directed at the enhancement of work undertaken with particular individuals and groups, and to the development of participants (including the informal educator). It tends to be an integral part of the working process. In order to respond to a situation workers have to make sense of what is going on, and how they can best intervene (or not intervene). Similarly, other participants may also be encouraged or take it upon themselves to make judgements about the situation. In other words, they evaluate the situation and their part in it. Such evaluation is sometimes described as educative or pedagogical as it seeks to foster learning. But this is only part of the process. The learning involved is oriented to future or further action. It is also informed by certain values and commitments (informal educators need to have an appreciation of what might make for human flourishing and what is ‘good’). For this reason we can say the approach is concerned with praxis – action that is informed and committed
|3 years ago. Rating: 1|