Abstract: A common model for learning from evaluation assumes that learning occurs when evaluators transmit findings and conclusions to programme participants and stakeholders. Learning is then understood as a cognitive act, happening in the mind of an individual and separated from the rest of our activities. In recent years, this common model has been challenged by scholars who argue for ‘situated learning’ in ‘communities of practice’. This alternative conception emphasizes the context-bound nature of learning (versus learning from material abstracted from context) in relationships between people and implies an intimate connection between knowledge and action. Using as an example an evaluation of a coercion project in Dutch psychiatry, this article illustrates and highlights some of the implications of this conception in terms of the learning that occurs and the roles of the evaluator in developing communities of practice.

Abstract: `Theories of Change' has proved a popular approach for both evaluators and commissioners of complex social programmes, but the ways in which theories of change (ToC) evaluations have been implement...

Abstract: The search for increased effectiveness of development aid and improved accountability mechanisms has led donors to place more emphasis on evaluation as the tool to discern what works, when and why ...

Abstract: The use of the constructivist paradigm to undertake so called `fourth generation evaluation' (FGE) has been promoted by its initial proponents (Guba and Lincoln) since the late 1980s. Despite this its use is still rarely favoured over the conventional positivist scientific approach.This paper explores the applicability of some of the main principles of FGE in relation to an evaluation undertaken by the authors.

Abstract: This article argues that one of the reasons for unsatisfactory performance is the single-sighted and narrow scope of performance management models applied in most of the OECD countries. Performance...

Abstract: During the last few years, study programmes in the field of evaluation have gained importance throughout Europe. The aim of this article is to give an overview of evaluation courses in Europe. Fourteen study programmes in France, Ireland, Germany, Spain, England, Italy, Belgium, the Netherlands and Switzerland were identified by means of internet research and personnel communication. Nine of these are described here in detail. This description is based on written materials and answers given in a short questionnaire. As a result two types of university Masters programmes could be distinguished: consecutive programmes, which provide young graduates with specialized theoretical and practical education in evaluation, and in-service programmes for those with professional experience. This is the first article to provide systematic information on this topic. However, further in-depth investigations are needed to enhance and complete the picture.

Abstract: Using ‘evidence’ to falsify rather than verify patterns in data and searching for alternative explanations enables a better understanding of the circumstances that explain why and how a social programme works or does not work. An analysis of the extent to which a programme is meeting its aims and objectives to find out if it provides a solution to the policy problem, is more rigorous. The roles researchers adopt influence the quality of an evaluation; facilitating a better understanding of the theories embodied in programmes enhances an evaluation while being a ‘broker of compromise’ can limit access to information. Researchers have a valuable role in promoting learning. A robust evaluation framework integrates strategies for generalizing at the outset and identifying mechanisms of change or causal mechanism is a way forward. Examples are taken from recent evaluations conducted by the author and colleagues to illustrate the arguments.

Abstract: In 1996, Icelandic schools were mandated to evaluate their work.This article focuses on four schools that were selected for this case study, two primary schools and two secondary schools.They all received support from two researchers: one from Central Connecticut State University and another from the University of Iceland.These researchers coached school-evaluation teams in their self-evaluation efforts and assessed change in the schools. The schools received empowerment-based support, where the researchers taught staff to evaluate school work and take responsibility for development based on outcomes from longitudinal data collection and dissemination of evaluation information. Researcher findings include changes in teacher perceptions of administration facilitating their professional growth; systematic data collection; teacher ownership and collaboration; and shared decision-making for improvement efforts in the schools. Findings were triangulated, using different data collection methods. Results indicat...

Abstract: This article reflects on the relationship between evaluation and trust. Evaluation has become prominent in recent years as a way to control administrative action.The underlying assumption here is that administrative action, e.g. in the form of publicly funded programmes, requires control in order to gain or sustain its legitimacy.Two case studies of programmes of higher education reform in Germany, initiating so-called `virtual universities', are used to investigate how stakeholders experienced evaluation. Informants argued that there was too much evaluation, that confusion and competition arose about the roles of evaluation, and that little instrumental use occurred. This situation caused frustration and begs the question whether evaluation, intended to increase trust through systematic and transparent inquiry and rational decision-making, contradicted its own claims. It is argued that evaluation can be endangered by `inverse process use' or `process damage'.

Abstract: Governance through Institutionalized Evaluation : Recentralization and Influences at Local Levels in Higher Education in Sweden

Abstract: This article reports the results of a research project that aimed to identify the criteria upon which a foster care intervention may be considered as having been successful. The aim was to confirm the existence of shared evaluation criteria among the different groups involved in foster care and, moreover, to investigate possible differences in the meaning given to the foster care experience by the different stakeholders. The data collected from six focus groups was analysed in two ways: first, the criteria necessary to evaluate foster care were derived; second, a textual analysis was undertaken to investigate the associations and the frequency of use of some significant terms.

Abstract: Evaluation research mixes administrative objectives with social scientific aims.This article distinguishes between three types of evaluation research based on the role of the evaluator. First, soci...

Abstract: As part of a larger research project about right-wing extremism and its causes and countermeasures, this article assesses the quality of existing evaluation studies in Switzerland, Germany, the US and other countries in order to synthesize their results. It presents the results of a meta-evaluation of measures against right-wing extremism in areas including education and social work. The meta-evaluation follows the professional evaluation standards of the Swiss Evaluation Society (SEVAL Standards). Analysis of the quality of selected evaluation studies in the field of measures against right-wing extremism allows for an assessment of the value of their results. The meta-evaluation covers 12 evaluation reports, enabling a description of the current practice of evaluation of measures taken against right-wing extremism in selected countries. The article also presents evidence about the usefulness of the SEVAL Standards in an applied setting. Thus the article provides background information on both evaluation ...

Abstract: Medical education and training in the United Kingdom is undergoing profound change driven by a governmental policy that has attracted the title Modernizing Medical Careers.The role of piloting and ...

Abstract: Whenever policy logic and outcomes are uncertain, the traditional public management reform model, focused on efficiency and effectiveness, seems to be weak. Uncertainty requires not only devolution...

Abstract: This article offers for wider discussion a description of the approach used in a situation where the gold standard of a randomized controlled trial (RCT) was not immediately feasible. It explains how the methods and research design of a developmental project known as `WRAP' (Woundcare Research for Appropriate Products) evolved, and discusses their implications for complex wound care evaluations.The project was guided by the UK Medical Research Council (MRC) framework for the development and evaluation of RCTs (MRC and Board, 2000). It is proposed that dressing performance in chronic wound care may best be evaluated by a focus on the first two phases of the framework, theorizing and modelling, to design interventions and measurement tools, before proceeding to the third phase, exploratory trials.When the data generated in such trials are deemed valid and reliable, using specified criteria, alternative forms of evaluation to the randomized controlled trial, such as post-market surveillance studies, can foll...

Abstract: The ‘distinctiveness criteria’ are an instrument of peace and conflict impact assessment (PCIA) applied to the EU Programme for Peace and Reconciliation in Northern Ireland and the border counties of Ireland (Peace II). This article evaluates the capacity of the ‘distinctiveness criteria’ tool, as applied, to generate a peace-building impact for the fund. While the criteria are found to have influenced project design, their potential impact is weakened by a relatively narrow and quantitative application at assessment, monitoring and evaluation stages, and by the absence of independent conflict analysis. A stronger monitoring and evaluation culture at the project level could increase the impact of the criteria and, therefore, the peace-building impact of the programme, and could help the framework to avoid some of the weaknesses generally associated with PCIA.

Abstract: The thermodynamic concept of entropy, referring to the degradation of energy and resultant disorder, has been influential in a number of disciplines including information science, ecological economics and sociology, and has recently been posited as a potentially fruitful approach to the evaluation of labour market interventions. The origins and applications of the entropy concept are reviewed before an evaluation framework is applied to assess how entropy evaluation might differ from traditional methodologies. As a thought experiment, a recent large-scale programme evaluation is `re-engineered' using entropy ideas resulting in a radically different design. Whether entropy designs would perform better than traditional ones will remain open to speculation until an entropy evaluation is implemented. Before then, the idea deserves peer review and refinement or rejection as appropriate.

Abstract: This article is based on an exploratory research project on the treatment of patients with asthma and allergy.The aim of the project is to develop a model for bridge building (institutionalized cooperation) between medical doctors and alternative therapists.The model might be useful for developing cooperation among other kinds of professionals in areas of services other than health care.The assumption is that the professionals' understanding of disease includes a causal reasoning that legitimates specific treatment methods and specific `outcome optics'.The results, in the form of self-reported experience of learning among the professionals, are analysed on the basis of learning theory. The results indicate that differences in intervention models together with evaluative feedback from users'/recipients' assessments of the intervention can be used to stimulate learning in groups of professionals that hold very opposing views on the `right' treatment model.

Abstract: The lack of transparency in the process of qualitative data analysis has led to the suspicion that findings may not be robust enough to be used as evidence. Much qualitative data analysis software (QDAS) contains functions that facilitate the demonstration of reliability and validity, although the process is not straightforward. Given the scale and complexity of many evaluations, QDAS is a powerful tool that can be used to manage and analyse vast amounts of qualitative data. Its use in an evaluative context, however, can be fundamentally different from using it for other forms of qualitative analysis. This article explores the use of NUD*IST version 6 (N6), a widely used QDAS, to manage data from the qualitative aspects of a large-scale multi-component evaluation involving a large, multi-site team. It demonstrates that research and logistical imperatives can have concrete impact on the way technology is harnessed to produce qualitative findings, hence inevitably influencing resultant evidence.

Abstract: The past two decades have seen a drastic increase in the role and expectations of civil society organizations (CSOs) in international development, placing greater demands on the evaluation of CSOs and associated programs.While there is much potential in CSOs, it is important to recognize the distinctly political and economic realities in which they operate. CSOs face formidable challenges, the foremost being dependency on donor funding and the resultant threat to CSO autonomy and performance. Does donor-required monitoring and evaluation (M&E) enhance or compromise CSO performance? Drawing upon research from the United Nations Office of the Special Coordinator of Africa and the Least Developed Countries (UN/ OSCAL) and the United Nations Office of the Special Adviser on Africa (UN-OSAA), this article presents an overview of African CSOs, exploring key issues that can inform the international M&E community.

Abstract: This article presents an innovative evaluation design which was used to evaluate the Swiss Environmental Impact Assessment. The design is new in that it amalgamates the realistic approach to evaluation with the method of Qualitative Comparative Analysis (QCA), the two of which are conspicuously similar. They share a complex view of causality, a generative perspective, a theory-driven approach to empirical observation and a limited claim to generalization. These conceptual parallels, as derived from the literature, are described in the first section, after a short introduction to realistic evaluation and the method of QCA. The following empirical section exemplifies their joint application and tackles the problems encountered. Based on this experience, the initial theoretical parallels are then reviewed. The article concludes that, under certain conditions, realistic evaluation and QCA provide a powerful tandem to produce empirically well-grounded context-sensitive evidence on policy instruments.

Abstract: The integration of certain policy objectives into other policy sectors – i.e. policy integration of such issues as gender, regional development, employment and environmental protection – is frequently requested in order to improve public policy. The article discusses the importance of evaluating policy integration as well as the perspective that such evaluations should extend to the outputs and outcomes of policies. Two examples of evaluating policy integration are provided: the integration of environmental concerns into technology policy and the integration of innovation objectives into environmental policy. The findings show that the integration of environmental concerns into technology policies could be increased, especially with respect to promoting technologies that do not have explicit environmental intentions. Similarly, innovation objectives could be further integrated into environmental policies. Evaluations of policy principles, such as policy integration, are important for policy development.

Abstract: Integrating performance information into budgeting, managing and reporting has become a common component of good public and not-for-profit management. In many jurisdictions, efforts to do so have b...