Abstract: The burgeoning performance management movement, with its emphasis on social program ‘results’ measured typically by a limited set of quantitative indicators, has developed a life of its own largely...

Abstract: The increasing use by central government in the UK of pilot programmes as a means of modernizing key public services marks a new style of policy formulation and implementation which in turn calls for a different kind of evaluation support. The ‘rational-objective’, ex post analyses favoured by ministers and officials over the last decade are no longer sufficient. Instead evidence from one of the earliest and most important of the new government's pilot initiatives (the national Best Value pilot programme) suggests that it will increasingly look to evaluators to act as change agents who are able to combine summative analysis of outputs and impacts with more formative approaches focused on developing a detailed understanding of processes. This presents important new challenges for both policy makers and evaluators.

Abstract: Performance measurement systems are cast as significant responses to contemporary public demands for government accountability for social spending and social programs. Performance measurements are then critiqued as inadequate measures of social program quality, as inadequate representations of program quality, and as at odds with evaluative processes that advance the ideals of deliberative democracy.

Abstract: This article examines the impact of cultural codes on the conduct and outcomes of evaluation. Cultural codes are defined as symbols and systems of meaning that are relevant to members of a particular culture (or subculture). These codes can be utilized to facilitate communication within the ‘inside group’ and also to obscure the meaning to ‘outside groups’. Drawing on the authors' combined experiences in Central and Eastern Europe, the article explores how cultural codes emerged in these contexts and the challenges they presented to the conduct and utilization of evaluation and to the evaluators themselves. Evaluators must be prepared, it is suggested, to unlock their own internal codes in relation to those of other cultures in order to create the appropriate socio-political relationships that are a prerequisite for learning and change. The article concludes with a discussion of the kind of partnerships and trust it is necessary to establish to underpin a mutual unlocking of codes.

Abstract: This article offers a deconstruction of global evaluative discourses concerning school effectiveness and improvement. It portrays these discourses in anthropological terms, as 'cultural performances', and examines the ways in which technical discourses obscure elements of ritual, philosophy, myth and shamanism. The author concludes that such discourses, especially in their mediatized forms - as league tables - are a form of contemporary 'spectacle'. They are our Olympic Games. The relation between these discourses and their cultural critique is then itself deconstructed in order to reflect on the nature and purpose of deconstruction as a means to contemporary cultural and educational insight.

Abstract: Two recent evaluation studies undertaken for the European Commission, related to the LEADER Community Initiative implemented throughout the 1 990s, have provided the basis for the issues discussed in this article. This programme was set up to encourage pilot experiences in the promotion of rural development in the problem areas of Europe, by introducing a set of distinctive features emphasizing the method by which local development is implemented, rather than individual actions. The article argues that development policies with a territorial approach, locally defined objectives, strategies and actions, and involving the participation of several levels of government, have a different set of underlying assumptions and cannot be evaluated with the same tools and concepts that are commonly used in conventional evaluations. Four main methodological problems are discussed: how to define and assess the distinctive features of LEADER individually and as a whole; how to relate such features to development factors;...

Abstract: This article reports on a mid-term evaluation of the impact of a recent anticorruption program developed by the Bank's Economic Development Institute (EDI). Central in EDI's approach is helping to develop and/or reinvigorate a country's National Integrity System (NIS). Integrity pillars, amongst others, are administrative reforms, watchdog agencies, Parliament, civil society, public awareness, the judiciary, the media and political will.The evaluation focuses on two African countries. It describes goals and instruments of the EDI approach, and puts these in an institutional context. The underlying ‘program logic’ is reconstructed. This reconstructed ‘logic’ is confronted with findings from a literature review, document analysis and on-site interviews in Uganda and Tanzania.This (realist evaluation) approach highlights the importance for evaluators to unravel (behavioural and social) mechanisms that underly programs.Conclusions are drawn about the program and its delivery including participants' assessment...

Abstract: Trying to build political dimensions into evaluation has blurred the distinction between policy analysis and policy evaluation. We do not dispute the importance of political context and values for ...

Abstract: Evaluation as an organizational element has spread across countries and organizational fields. Yet little attention has been paid to the comparative analysis of adoption, construction and implementation processes of evaluation within different organizational fields. The article is based on an empirical study of research evaluation and evaluation of education within the Danish university sector. It conceptualizes and explains similarities and differences in adoption processes and constructed evaluation standards within the two subfields. Three models are used for explaining differences and similarities: a stakeholder model, an institutional field model and an institutional heritage model. The article shows how evaluation as an organizational element is used simultaneously for processes of change and processes of reproduction of norms and values.

Abstract: This article probes the contrasting traditions of economics and evaluation and shows that intensified collaboration would benefit both professions. While evaluators have long been examining the impact of economic advice on public policy, economists have neglected evaluation both as a platform for testing economic policy hypotheses and as a topic for economic research in its own right. To encourage convergence between economics and evaluation, this article sketches the elements of an institutional economics of evaluation, considers the ongoing debate about the proper function of evaluation in society and presents cost benefit concepts and a supply demand framework to scrutinize the value and prospects of evaluation activities. Finally, it identifies performance measurement as a priority area for cooperation between evaluators and economists.

Abstract: The central concern of this article is with the question of how to evaluate and manage the performance of emerging university–industry research networks. It is argued that university–industry (UI) partnerships have, after more than a decade of promotion by the state, reached a point where both partners are actively seeking each other out. Despite the recognition of the potential benefits of co-operation by both sides, there are still several unanswered questions about how to assess and manage such co-operation. The article describes the national science system as one where a ‘central rule’ environment has been gradually replaced by forms of intervening R&D networks. It is further argued that such networks require new approaches to evaluate performance. The article concludes by proposing an aggregate of a number of indicators that may be used for dynamic network management.

Abstract: The utility of Guba and Lincoln's (1989) Fourth Generation Evaluation (FGE) method has been a source of debate in the field of evaluation (e.g. Adelman, 1996; Greene, 1996; Laughlin and Broadbent, ...

Abstract: The Debates, Notes and Queries section provides an opportunity for evaluation-related issues to be debated as well as for more genera! interchange. Debates can take the form of sustained arguments by the advocates of different approaches or of briefer thoughts or notes. Contributors may also wish to comment and raise questions about material that has previously appeared in the journal or simply use the section to draw readers' attention to relevant issues, ongoing research, evaluation activities and other events.

Abstract: This article presents a discussion about an emerging area of evaluation discourse, the ‘European spatial approach’. It examines the current policy framework at the European-wide level by exploring the alternative planning and public policy paradigms that underpin the case for rationality within this framework. By using such a theoretical review, the article seeks to argue that the current deployment of evaluation within strategic spatial policy and planning is poorly understood, and further work is needed to develop an understanding of the role of power and knowledge in rationality, and in particular the role they play within the pluralist models of evaluation that are emerging within the wider EU policy processes. This is illustrated by specific evaluation experiences within the EU Trans-European Transport Networks and Structural Funds.

Abstract: This article discusses, from an evaluator's point of view, the uses and utility of evaluation in inter-governmental contexts. Empirical data are drawn from the current evaluation of the European So...

Abstract: This article explores the relationship between the social construction of risk, uncertainty and evaluation in the field of development cooperation. It explores the assumption that risk and uncertai...

Abstract: Supreme Audit Institutions (SAIs) have become important producers of evaluations. Auditors have become evaluators and evaluators have become auditors. In this article, we discuss some of the main characteristics of the Office of the Auditor General of Norway and society's expectations of this institution. We investigate some of the driving forces behind the developments in evaluation activities in the Office of the Auditor General, focusing in particular on the dilemmas and challenges these new 'products' create for this kind of institution. In connection with this, we also list a number of issues that will require particular attention in order for the Office of the Auditor General to maintain its legitimacy in the future.

Abstract: This article reports on initial findings of the Template Project, a project researching ways of evaluating post-graduate courses offered through the medium of distance education. It has taken an evaluation tool from industry and re-designed it so that it can apply to distance education courses. The question as to whether or not a tool initially located in the service industries is appropriate for education is examined as are the alternative paradigms of evaluation. A Popperian approach to evaluation is offered as the paradigm within which the Template Project's approach to evaluation should be located.

Abstract: This article discusses active qualitative evaluation and modern sociotechnical design as parts of an approach for the evaluation of organizational change. ‘Intervention in interactions’ is the most important similarity between these parts when considering the evaluation of organizational change. The two parts are discussed in terms of their theoretical core elements, practical consequences for research and dilemmas. Because, in practice, intervening in a changing organization is always accompanied by some important paradoxes, the different parts of the approach are presented as supplementary in practical evaluation of organizational change. The leitmotif on the basis of the case of the Royal Netherlands Military Academy here is that a paradoxical situation such as intervening in a changing organization needs a paradoxical intervention strategy.

Abstract: In Volume 3 (1) we published a keynote speech delivered by Eleanor Chelimsky at the December 1996 Conference of the UK Evaluation Society. Hellmut Eggers, an active member of the evaluation community in Europe, wrote to partly challenge some of Eleanor Chelimsky's arguments. Dr Eggers is no stranger to the problems of making evaluations purposeful and useful: he was for many years responsible for evaluation within the General Development Directorate of the European Commission. We are pleased to publish below both Hellmut Eggers' contribution and Eleanor Chelimsky's response. Readers who wish to take this debate further may want to re-read Eleanor Chelimsky's keynote address published in this journal and also consider a more recent article by the same author in the American Journal of Evaluation 19 (1) 1998.

Abstract: The central questions of this article are when, how and what do governmental agencies learn from evaluations. A structural constructivist theoretical framework is developed and applied to two case studies, both of which take a report by the Dutch Court of Audit (CoA) as a starting point. A reconstruction is made of the intra- and interorganizational processes through which the impacts of these evaluations were socially constructed. It appears that an evaluation has hardly any direct effect that can be unequivocally ascribed to it. Rather, evaluations seem to support or counteract debates, tendencies and options already present (or 'under construction') in the interaction among the actors involved. Using a structural constructivist theoretical framework we identify mechanisms and conditions that enhance forms of learning processes.This article concludes with some hypotheses about the genesis of evaluation impact.

Abstract: Since the mid-1980s, the British government has put more emphasis on monitoring and evaluation of existing and new policy initiatives. There is, however, little evidence to show that new regenerati...

Abstract: This article examines the crucial role of scientific expertise and authority, in evaluation as well as in research decision-making. The case of the Indian Council of Agricultural Research (ICAR) is used to demonstrate how the bureaucratic imperatives in a public research system can thwart the cause of scientific authority and accountability. Research decision-making is a function delegated to different points, or ‘nodes’, vested with scientific expertise and the power to make decisions. Scientific expertise is the basic asset used by the nodes, which may be individual scientists and/or groups, boards or organizations. Patronage or decision-making in the ICAR is, for the most part, vested in bureaucratic nodes, marking the dichotomy in the organization between scientific and administrative or financial decision-making. The concluding section of this article highlights the social reproduction of bureaucratic nodes, which perpetuates the marginalization of evaluation. The nodes in the ICAR rely on bureaucrat...

Abstract: Evaluation has much to offer to policy makers, but policy makers rarely base new policies directly on evaluation results. Partly this is because of the compedting pressures of interests, ideologies...

Abstract: In this article we argue that evaluation is an event-driven and product-oriented activity needs to be reconsidered and reconceptualized in light of continuous changes organizations are experiencing...