This paper explores an approach to the implementation and evaluation of integrated health service delivery. It identifies the key issues involved in integration evaluation, provides a framework for assessment and identifies areas for the development of new tools and measures. A proactive role for evaluators in responding to health service reform is advocated.
This paper discusses a series of important methodological issues in developing targeted health-related quality of life measures in studies of the effects of medical interventions. Such measures cannot be developed unless the evaluator understands the life domains that medical interventions affect. Qualitative discovery methods are needed to obtain this understanding. Once domains are targeted for measurement, careful and systematic laboratory pilot work should be used to select initial scale items. Psychometric evaluation of response patterns in subsequent field tests is (...) needed to assess the measures. Less concern should be directed to internal consistency reliability of scales in the psychometric evaluation and more to the ability of short scales to reproduce total scale variance and to provide precise measurement within the range of the outcome where effects are expected. The paper closes with a discussion of modern methods of item response scaling that can be used to address these issues. (shrink)
This paper answers the question how pragmatic argumentation which occurs in a legal context, can be analyzed and evaluated adequately. First, the author surveys various ideas taken from argumentation theory and legal theory on the analysis and evaluation of pragmatic argumentation. Then, on the basis of these ideas, she develops a pragma-dialectical instrument for analyzing and evaluating pragmatic argumentation in a legal context. Finally she demonstrates how this instrument can be used by giving an exemplary analysis and evaluation (...) of pragmatic argumentation in a decision of the Dutch Supreme Court. (shrink)
The purpose of the present paper is: (1) to outline a conceptual framework useful for the analysis of ethical issues raised by goal-directed activities, (2) to apply this framework to nanoscale research, (3) identify some of the main challenges in the evaluation of such research, and (4) exemplify what is needed for a positive answer to the question “How can nanoscale research improve the quality of life?” A basic idea of the paper is that nanoscale research can improve the (...) conditions and quality of life of large groups in society, provided that: (a) this research is directed at certain generally accepted goals, (b) at least some of the opportunities are exploited for the good of mankind, (c) the key obstacles on the road are eliminated, reduced or circumvented, and (d) this is done in ethically acceptable ways. (shrink)
Corporate social performance (CSP) has become a widely applied concept, discussed in most large firms’ corporate reports and the academic literature alike. Unfortunately, CSP has largely been employed as a way of demonstrating corporate social responsibility (CSR) in practice, or to justify the business case for CSR in academia by relating some measure of CSP to some measure of financial performance. In this article, we discuss multiple shortcomings to these approaches. We argue that (1) CSR activities need to be managed (...) and measured as projects and aggregated to the business or corporate level using a project portfolio; (2) appropriate measures need to be identified that move away from reporting the firm’s activities toward quantifying actual social outcomes achieved; and (3) given the types of projects prevalent in CSR, statistical evaluation methods common in other fields (ideally, pre-test post-test control group designs, such as used in medicine or propensity score matching for ongoing or past projects) should be employed to properly measure outcomes. We make a first, albeit imperfect, attempt at using such an approach with data collected on behalf of the Patrimonio Hoy project, a well-publicized CSR initiative carried out by Cemex in Mexico. We show that the results from this data reinforce concerns voiced earlier in this article. (shrink)
Evaluation processes are a basic component of creativity. They guide not only the pure judgement about a new artefact but also the generation itself, as creators constantly evaluate their own work. This paper proposes a model for automatic story generation based on the evaluation of stories. A model of how quality in stories is evaluated is presented, and two possible implementations of the generation guided by this evaluation are shown: exhaustive space exploration and constrained exploration. A theoretical (...) model and its implementation are explained and validation of the evaluation function through comparison with human criteria is described. (shrink)
This study aims to propose a framework considering both economic issues and environmental effects in technology evaluation in order to provide firms' decision makers a useful reference in adopting technologies that will enable them to fulfill corporate social responsibilities and get competitive advantages at the same time. Recently, the demands for technology evaluation have increased with the flourishing development of technology licensing, technology transaction or joint venture on the one hand and with the pressing needs of environmental protection (...) for human beings' sustainable development on the other hand. Under such conditions, it thus goes without saying that firms' decision makers are propelled to take into account both economic benefits and environmental effects in evaluating technologies by choosing low or nonpolluting technologies for manufacturing products. Although technology evaluation is not a new and emerging subject currently besetting scholars in the field of management, previous research on this topic has unwittingly left behind the pressing issue of environmental effects. Based on this observation, this study purports to develop a new framework for technology evaluation by taking both economic benefits and environmental perspectives into consideration. In it, we seek to demonstrate that our proposed framework will not only be a workable model but also can serve as a useful point of reference for technology appraisers and firms' decision makers. (shrink)
waC provides a variety of different graphical notification mechanisms which can be coupled to specific working situations using the AREA model. We also report on the evaluation of the system under real-life conditions in a German federal ministry.
This paper examines a small-scale attempt to support collective evaluation of a transgenic potato variety. By mobilizing Laurent Thevénot’s ideas on the connectedness of the ontological and normative, it investigates how the controversial object was associated with coordinating perspectives or orders of worth in two focus groups. In these groups, the GM potato qualified for evaluation in relation to deterministic market forces. However, it was unclear whether the potato would operate as a beneficial market asset or merely as (...) an accelerator of ever tougher competition. The innovation also had a tendency to disappear out of sight or to receive capacities as a transgenic application of whatever kind. Hence, it was only with effort that the discussions delved into the specific realities and circumstances of the Finnish potato production. When they did, some particular demands posed by the blight resistant potato became visible and discussable. These scripts concerned the counterparts of contract production, possibly favoring the reorganization of producers into larger associations of highly specialized professionals. However, since practical implications and possibilities received little attention, the paper suggests that more attention needs to be put into organization and preparation of multi-stakeholder evaluations. The emergence of learning potential greatly depends on the abilities and willingness of the participants to engage with speculative experiments or reality tests. This is a risky strategy for anyone who hopes that a meeting will remain within a particular scope or that it will reach a particular conclusion. (shrink)
This paper is a portrayal of how social responsibility performance evaluation can act as an accounting measure of management efficiency. In fact, it has given much importance to socio-economic and socio-human obligations to others. The paper attempts to show that these days there is a great need to emphasise more clearly social responsibility, which the corporate sector can and should undertake. The theme of the paper is that the scope of corporate social responsibility encompasses not only economic well-being but (...) also the human aspects of life. In addition, if management of a corporation performs its social responsibility well, one may say that management has done its job efficiently. This study is based on mainly literature review. Analytical thinking is also another building block of this paper. However, the limitation of this study is that no data of the existing situation of Bangladesh or India pertaining to the subject matter referred to in the present paper has been used. (shrink)
This paper explores the issue of epistemic injustice in research evaluation. Through an analysis of the disciplinary cultures of physics and humanities, we attempt to identify some aims and values specific to the disciplinary areas. We suggest that credibility is at stake when the cultural values and goals of a discipline contradict those presupposed by official evaluation standards. Disciplines that are better aligned with the epistemic assumptions of evaluation standards appear to produce more "scientific" findings. To restore (...) epistemic justice in research evaluation, we argue that the specificity of a discipline's epistemic aims, values, and cultural identities must be taken into account. (shrink)
The effectiveness of information retrieval technology in electronic discovery (E-discovery) has become the subject of judicial rulings and practitioner controversy. The scale and nature of E-discovery tasks, however, has pushed traditional information retrieval evaluation approaches to their limits. This paper reviews the legal and operational context of E-discovery and the approaches to evaluating search technology that have evolved in the research community. It then describes a multi-year effort carried out as part of the Text Retrieval Conference to develop (...) class='Hi'>evaluation methods for responsive review tasks in E-discovery. This work has led to new approaches to measuring effectiveness in both batch and interactive frameworks, large data sets, and some surprising results for the recall and precision of Boolean and statistical information retrieval methods. The paper concludes by offering some thoughts about future research in both the legal and technical communities toward the goal of reliable, effective use of information retrieval in E-discovery. (shrink)
Recently, moral deliberation within care institutions is gaining more attention in medical ethics. Ongoing dialogues about ethical issues are considered as a vehicle for quality improvement of health care practices. The rise of ethical conversation methods can be understood against the broader development within medical ethics in which interaction and dialogue are seen as alternatives for both theoretical or individual reflection on ethical questions. In other disciplines, intersubjectivity is also seen as a way to handle practical problems, and methodologies have (...) emerged to deal with dynamic processes of practice improvement. An example is responsive evaluation. In this article we investigate the relationship between moral deliberation and responsive evaluation, describe their common basis in dialogical ethics and pragmatic hermeneutics, and explore the relevance of both for improving the quality of care. The synergy between the approaches is illustrated by a case example in which both play a distinct and complementary role. It concerns the implementation of quality criteria for coercion in Dutch psychiatry. (shrink)
This paper makes three points. First, empathy cannot be considered an epistemic basis for qualitative research and evaluation. Second, it is, however, a valuable method for understanding the private meanings of words and deeds. Third, this method is not completely reliable for purposes of what Popper called refutation, but is useful in what he dubbed scientific conjecture or the generation of theory. Basic researchers will need to take the necessary steps to subject empathetic hunches to critical examination. However, owing (...) to the exigencies of action settings and decision-making, disciplined conjectures are sometimes the most that evaluators can hope to record. (shrink)
Team science models are frequently promoted as the best way to study complex societal and environmental problems. Despite increasing popularity, there is relatively little research on the processes and mechanisms that facilitate the emergence of integration of interdisciplinary teams. This article evaluates a suite of recent team-building and grant-writing activities designed to address water management in the Western U.S. We use qualitative methods to document the emergence of integrative capacity at the individual, group, and institutional levels, with particular attention to (...) the role of graduate students and non-academic practitioners in a team science planning project. Our findings highlight the importance of social integration as a basis for conceptual integration and an ability to relate these concepts to real-world problems. The findings also demonstrate the value of qualitative evaluation measures of team readiness, capacity, and intellectual outputs to complement conventional evaluation indicators that rely on quantitative scientific outputs, particularly for team science projects still in the planning stages. (shrink)
An argument evaluation inventory distinguishing between different levels of theory-evidence differentiation was designed corresponding to the levels of argument observed in argument generation tasks. Five scenarios containing everyday theories about a social problem, and arguments to support those theories were presented to 170 participants from two age groups (15 and 22 years) and different educational tracks. Participants had to rate the validity of arguments proposed by a story figure, to support the theory, to choose the best argument, and to (...) justify their choice. The rating task proved to be very difficult for all age groups, with only 49% of the university students consistently rating valid evidence-based arguments higher than flawed arguments. Competence improved with age and educational level. In the choice task more than 80% of the adults preferred an argument that reflected theory-evidence differentiation over mere theory elaboration or flawed reasoning. However, only adults with a university education were able to also explicitly justify their choice. Overall, these findings imply that laypersons have similar conceptual problems in differentiating theory from evidence as it has been reported for evidence generation tasks (Kuhn, 1991). Performance on the choice task suggests that some implicit awareness of differences between theory and evidence may precede a full, explicit understanding. Implications for education are discussed. (shrink)
In this article the author develops a framework for a pragma-dialectical reconstruction of teleological argumentation in a legal context. Ideas taken from legal theory are integrated in a pragma-dialectical model for analyzing and evaluating argumentation, thus providing a more systematic and elaborate framework for assessing the quality of teleological arguments in a legal context. Teleological argumentation in a legal context is approached as a specific form of pragmatic argumentation. The legal criteria that are relevant for the evaluation of teleological (...) argumentation are discussed and translated in terms of critical questions that are relevant for the evaluation of the various forms of teleological argumentation. (shrink)
Clinical ethics committees have existed in Norway since 1996. By now all hospital trusts have one. An evaluation of these committees’ work was started in 2004. This paper presents results from an interview study of eight clinicians who evaluated six committees’ deliberations on 10 clinical cases. The study indicates that the clinicians found the clinical ethics consultations useful and worth while doing. However, a systematic approach to case consultations is vital. Procedures and mandate of the committees should be known (...) to clinicians in advance to ensure that they know what to expect. Equally important is bringing all relevant facts, medical as well as psychosocial, into the discussion. A written report from the deliberation is also important for the committees to be taken seriously by the clinicians. This study indicates that the clinicians want to be included in the deliberation, and not only in the preparation or follow-up. Obstacles for referring a case to the committee are the medical culture’s conflict aversion and its anxiety of being judged by outsiders. The committees were described as a court by some of the clinicians. This is a challenge for the committees in their attempt to balance support and critique in their consultation services. (shrink)
Introduction -- Need for a research and evaluation capability : becoming a high-performing organization -- Need for a research and evaluation capability : accreditation requirements -- Structure and scope of an office of institutional research : findings from interviews -- Lessons learned from organizations with training missions similar to that of METC -- Conclusions and recommendations.
This paper is a report on aconversation held between the authors andcentered on their shared interest inalternative methods of inquiry and evaluationin agriculture. The conversation was initiatedat the W. K. Kellogg Foundation and has evolvedthrough a series of long distanceconversations. Though not a verbatim transcriptof our conversations, this paper represents acomposite of both the face-to-face conversationand our stream of dialogue over the past year.Central to our discussion is an exploration ofthe parallels between the paradigm shift thatoccurred in evaluation in (...) the early 1980s andthe current agricultural paradigm shift beingpromoted by the W. K. Kellogg Foundation. Inthe course of this conversational paper, wesuggest not only that evaluators andresearchers should cultivate their capacity tohear and tell stories, but also thatagricultural programs and their long-termimpacts could benefit from different kinds ofevaluation efforts. From this perspective, theevaluation or research report is no longer anattempt to mirror reality, but rather it is anevocative story that asks the reader to engagethe storyline morally, emotionally,aesthetically, and intellectually, as well asfrom a social impact perspective. It is ourhope that this paper will serve as what Lather(1993) has called an ``incitement to discourse''in the disciplinary fields of agriculture andevaluation. (shrink)
Normal.dotm 0 0 1 66 378 Thorbjoern Mann Consulting 3 1 464 12.256 0 false 18 pt 18 pt 0 0 false false false Normal.dotm 0 0 1 80 459 Thorbjoern Mann Consulting 3 1 563 12.256 0 false 18 pt 18 pt 0 0 false false false Arguments commonly used in discussions about design, planning, policy-making issues have not been adequately analyzed in the literature. The structure of such ‘planning arguments’ is discussed. Based on the conceptual framework of the (...) ‘argumentative model of planning’ proposed by H. Rittel, an approach for their systematic and transparent evaluation by discourse participants is presented. Procedural implications for its application in the planning process are discussed, and the potential for information technology support for such processes explored. (shrink)
The centrality of argumentation in the judicial process is an age-old acquisition of research on legal discourse. Notwithstanding the deep insights provided by legal theoretical and philosophical works, only recently has judicial argumentation been tackled in its linguistic dimension. This paper aims to contribute to the development of linguistic studies of judicial argumentation, by shedding light on evaluation as a prominent aspect in the construction of the judge’s argumentative position. Evaluation as a deep structure of judicial argumentation is (...) studied from a discursive point of view entailing the analysis of a sample of authentic judicial language. Evaluative lexis is investigated within a single genre of judicial discourse, i.e. judgments, instantiated by a corpus of US Supreme Court judgments. Findings show that judges use diversified strategies to take stance as they organise their argumentative discourse: from easily recognisable verbal and adjectival tools to more finely-grained discourse elements such as the encapsulating pattern ‘this/these/that/those + labelling noun’. (shrink)