Here we present the framework of a new approach to assessing the capacity of research programs to achieve social goals. Research evaluation has made great strides in addressing questions of scientific and economic impacts. It has largely avoided, however, a more important challenge: assessing (prospectively or retrospectively) the impacts of a given research endeavor on the non-scientific, non-economic goals—what we here term public values —that often are the core public rationale for the endeavor. Research programs are typically justified in (...) terms of their capacity to achieve public values, and that articulation of public values is pervasive in science policy-making. We outline the elements of a case-based approach to public value mapping of science policy, with a particular focus on developing useful criteria and methods for assessing public value failure, with an intent to provide an alternative to market failure thinking that has been so powerful in science policy-making. So long as research evaluation avoids the problem of public values, science policy decision makers will have little help from social science in making choices among competing paths to desired social outcomes. (shrink)
[Dissertation summary] When performing intertemporal cost-benefit analyses of policies, both in terms of climate change and other long-term problems, the discounting problem becomes critical. The question is how to weight intertemporal costs and benefits to generate present value equivalents. This thesis argues that those best placed to answer the discounting problem are domain experts, not moral philosophers or the public at large. It does this by arguing that the discounting problem is a special case of an interesting class of problems, (...) those which are both what I call morally complex and quantitative. [Open access]. (shrink)
In some severely uncertain situations, exemplified by climate change and novel pandemics, policymakers lack a reasoned basis for assigning probabilities to the possible outcomes of the policies they must choose between. I outline and defend an uncertainty averse, egalitarian approach to policyevaluation in these contexts. The upshot is a theory of distributive justice which offers especially strong reasons to guard against individual and collective misfortune.
Citizens' juries are commended as a new technique for democratising health service reviews. Their usefulness is said to derive from a reliance on citizens' rational deliberation rather than on the immediate preferences of the consumer. The author questions the assertion of critical detachment and asks whether juries do in fact employ reason as a means of resolving fundamental disagreements about service provision. He shows that juries promote not so much a critically detached point of view as a particular evaluative framework (...) suited to the bureaucratic idiom of social welfare maximisation. Reports of jury practice reveal a tendency among juries to suppress by non-rational means the everyday moral language of health care evaluation and substitute for it a system of thought in which it can be deemed permissible to deny treatment to sick people. The author concludes that juries are chiefly concerned with non-rational persuasion and because of this they are morally and democratically irrelevant. Juries are no substitute for voting when it comes to protecting the public from zealous minorities. (shrink)
This paper defends the use of quasi-experiments for causal estimation in economics against the widespread objection that quasi-experimental estimates lack external validity. The defence is that quasi-experimental replication of estimates can yield defeasible evidence for external validity. The paper then develops a different objection. The stable unit treatment value assumption, on which quasi-experiments rely, is argued to be implausible due to the influence of social interaction effects on economic outcomes. A more plausible stable marginal unit treatment value assumption is proposed, (...) but it is demonstrated to severely limit the usefulness of quasi-experiments for economic policyevaluation. (shrink)
Many anti-obesity policies face a variety of ethical objections. We consider one kind of anti-obesity policy — modifications to food assistance programs meant to improve participants' diet — and one kind of criticism of these policies, that they are inequitable. We take as our example the recent, unsuccessful effort by New York State to exclude sweetened beverages from the items eligible for purchase in New York City with Supplemental Nutrition Support Program assistance. We distinguish two equity-based ethical objections that (...) were made to the sweetened beverage exclusion, and analyze these objections in terms of the theoretical notions of distributive equality and social equality. First, the sweetened beverage exclusion is unfair or violates distributive equality because it restricts the consumer choice of SNAP participants relative to non-participants. Second, it is disrespectful or violates social equality to prohibit SNAP participants from purchasing sweetened beverages with food stamps. We conclude that neither equity-based ethical objection is decisive, and that the proposed exclusion of sugar-sweetened beverages is not a violation of either distributive or social equality. (shrink)
As concerns about the negative health effects of unhealthy eating and overweight/obesity increase, so too do efforts to combat obesity. Both the federal government, as well as state and local governments, have proposed and implemented a variety of healthy eating and obesity prevention policies. Many of these policies are controversial, facing objections that range from the practical to the ethical. In this paper, we consider one such policy — restrictions on food assistance programs that are meant to improve participants’ (...) diet — and one criticism of these policies, that they are inequitable. We take as our primary example the recent, unsuccessful effort by New York State to exclude sweetened beverages from the items eligible for purchase in New York City with Supplemental Nutrition Assistance Program. (shrink)
Grand challenges stress the importance of multi-disciplinary research, a multi-actor approach in examining the current state of affairs and exploring possible solutions, multi-level governance and policy coordination across geographical boundaries and policy areas, and a policy environment for enabling change both in science and technology and in society. The special nature of grand challenges poses certain needs in evaluation practice: the need for learning at the operational, policy and, especially, system level; and the importance of (...) a wider set of impacts and behavioural change. The examination of the usefulness of evaluations as learning tools thus becomes relevant as does the way current evaluation practices address broader impacts and issues such as behavioural additionality. The suitability of existing evaluation contexts in meeting the specific issues posed by the ‘grand challenges’ orientation is also worth examining. The paper argues that learning at the policy and system levels is largely unaddressed while concepts such as behavioural additionality are still underexploited. (shrink)
To date, the concept of ecological justice, when applied to international climate change policy, has largely focused on the North-South dichotomy and has yet to be extended to Central and Eastern European countries. This article argues that current formulations of climate change policy cannot address potential issues of ecological injustice to Central and Eastern European countries. Several Central and Eastern European countries recently joined the European Union, but ecological justice discourse in the EU is shown to be underdeveloped. (...) Although the analysis argues that equal per capita emissions is a socially and ecologically useful starting point for EU climate policy, it is further argued that bringing pertinent regional concerns into the discussion can enhance efforts to realize just outcomes to the climate problem. (shrink)
This article aims to contribute to the application of ethical frameworks to public health policy. In particular, the article considers the use of the Nuffield Council on Bioethics stewardship model, as an applied framework for the evaluation of evidence within public health policymaking. The ‘Stewardship framework’ was applied to a policy proposal to restrict marketing of food and beverages to children. Reflections on applying the stewardship model as a framework are provided. The article concludes that the questions (...) used to apply the stewardship model usefully introduced ethical considerations into the evidence review. However, the real value will likely come from the type of policy process within which the framework is used, identifying competing value positions and capturing local value requirements. (shrink)
This paper argues that some educational policy slogans, particularly compound slogans, are inherently paradoxical, and that while this may have a strong motivational effect, in appealing to a wide range of ideals and aspirations, it renders both the implementation and the evaluation of certain policies problematic. The example is given of equal opportunities in relation to gender and subject choice.
A wide range of medical institutions have developed and implemented policies to mitigate the adverse consequences of conflicts of interest. These newly implemented policies, which include regulation of industry contact with physicians and hospitals, controls on gifts from industry, and greater transparency in industry sponsored activities, have generated considerable controversy.Formulating and evaluating policies in a neutral, unbiased fashion can be difficult for those personally affected. When people have a stake in an issue, they tend to process information in a selective (...) fashion that supports their personal interests, a phenomenon known as “motivated reasoning.” When decision makers with preexisting opinions are exposed to information, they are inclined to selectively use the information to arrive at conclusions that justify their prior beliefs. When confronted with information that contradicts existing views, people evaluate it with greater skepticism. Additionally, once decision makers have reached a decision, they are likely to evaluate subsequent evidence in a biased manner that supports their decision. (shrink)
Several different evaluation issuesare perceived as important by people involved withinnovative projects intended to improve local food andnutrition systems; particularly the establishment oflocal food policy coalitions. Several such coalitionshave been formed in North America, Europe, andAustralia with the goal of improving community foodsecurity and promoting sustainable local food systems.Pioneer coalitions have served as models, yet therehas been little systematic evaluation of thesemodels. A qualitative study was conducted to identifyfactors that may hinder evaluation efforts. In grouptelephone interviews, (...) we sought the views ofacademics, project organizers, and funders, a total of24 key informants. Pressures to evaluate were assessed differently bythe three groups of key informants. Academics felt thefocus of evaluation should be on the effectiveness ofthe process used to discuss issues and formulatepolicies and plans. Project organizers and fundersperceived a need to assess project impact andoutcomes. A lack of suitable evaluation models andmethods was viewed as a formidable barrier. The use ofinappropriate methods and premature impact evaluationwere noted as potential threats to projectsustainability. External constraints and resourcelimitations were also said to inhibit evaluationefforts. It appears that several other factors may also beimpeding progress in conducting more (and more useful)evaluations including: (1) the apparent negativeconnotation of evaluation and the limited benefitsexpected from evaluation by stakeholders, (2) a lackof consensus about important evaluation questions,(3) insufficient evaluation expertise among projectorganizers, and (4) inadequate appreciation ofincreasing accountability pressures. (shrink)
In the academic world, peer review is one of the major processes in evaluating a scholars contribution. In this study, we are interested in quantifying the merits of different policies in a peer review process, such as single-blind review, double-blind review, and obtaining authors feedback. Currently, insufficient work has been undertaken to evaluate the benefits of different peer review policies. One of the major reasons for this situation is the inability to conduct any empirical study because data are presently unavailable. (...) In this case, a computer simulation is one of the best ways to conduct a study. We perform a series of simulations to study the effects of different policies on a peer review process. In this study, we focus on the peer review process of a typical computer science conference. Our results point to the crucial role of program chairs in determining the quality and diversity of the articles to be accepted for publication. We demonstrate the importance of discussion among reviewers, suggest circumstances in which the double-blind review policy should be adopted, and question the credibility of the authors feedback mechanism. Finally, we stress that randomness plays an important role in the peer review process, and this role cannot be eliminated. Although our model may not capture every component of a peer review process, it covers some of the most essential elements. Thus, even the simulation results clearly cannot be taken as literal descriptions of an actual peer review process. However, we can at least still use them to identify alternative directions for future study. (shrink)
Physicians are affected by the conflict of interest (COI) policies they help formulate. This study examines whether physicians evaluate these policies impartially. One hundred and seventy-nine physicians, 224 financial advisors, and 1,430 members of the general public evaluated the fairness and efficacy of a COI policy in either a medical or financial context. Physicians were more critical of the medical COI policy compared to a financial COI policy, while financial professionals displayed the reverse pattern and control respondents (...) rated both policies similarly. This suggests a bias against COI policies by those who will be directly affected. (shrink)
“Female sexual dysfunction” is the type of contested disease that has sparked concern about the role of the pharmaceutical industry in medical science. Many policies have been proposed to manage industry influence without carefully evaluating whether the proposed policies would be successful. We consider a proposal for incorporating citizen stakeholders into scientific research and show, via a detailed case study of the pharmaceutical regulation of flibanserin, that such programs can be co-opted. In closing, we use Holman’s asymmetric arms race framework (...) as a tool for evaluating policies in industry-funded science. (shrink)
This book examines the role and limits of policies in shaping attitudes and actions toward war, violence, and peace. Authors examine militaristic language and metaphor, effects of media violence on children, humanitarian intervention, sanctions, peacemaking, sex offender treatment programs, nationalism, cosmopolitanism, community, and political forgiveness to identify problem policies and develop better ones.
This paper provides an evaluation of the spinoff of a for-profit company from the American Institute of Certified Public Accountants (AICPA), a nonprofit professional association. The evaluation is based on a review of the literature on public policy issues surrounding organizational conversions from nonprofit to for-profit legal status. Many criticisms of this for-profit spinoff were voiced by professional leaders and accounting regulators, and we demonstrate that these criticisms are grounded in widely recognized policy principles relating to (...) nonprofit conversions. The public policy issues raised by this study have implications for the governance of professional associations in all disciplines. (shrink)
Hospital ethics committees (HECs) and ethicists generally describe themselves as engaged in four domains of practice: case consultation, research, education, and policy work. Despite the increasing attention to quality indicators, practice standards, and evaluation methods for the other domains, comparatively little is known or published about the policy work of HECs or ethicists. This article attempts to open the ?black box? of this health care ethics practice by providing two detailed case examples of ethics policy reviews. (...) We also describe the development and application of an evaluation strategy to assess the quality of ethics policy review work, and to enable continuous improvement of ethics policy review processes. Given the potential for policy work to impact entire patient populations and organizational systems, it is imperative that HECs and ethicists develop clearer roles, responsibilities, procedural standards, and evaluation methods to ensure the delivery of consistent, relevant, and high-quality ethics policy reviews. (shrink)
This article explores the different uses of forms of direct verbal quotes in follow-up utterances delivered during the parliamentary debates after the inaugural speech of a new chancellor in the Austrian parliament and investigates their positioning effects for members of parliament who have the first opportunity of publicly ‘doing being a government or opposition MP’ in the new legislative term. Representing the first public confrontation between government and opposition MPs, the debates foreshadow topical and interpersonal aspects of the political frontstage (...) communication in the beginning legislative term. The main purpose of this article is to investigate the relations between formal properties of direct verbal quotes, recontextualization practices, their pragmatic function, and the specific aspect of the public personae speakers foreground by their use in the inaugural debates. Although direct quotations in follow-up utterances of MPs do not occur very often, the analyses demonstrate that this specific form of discourse representation deserves attention not only because it is mainly opposition MPs who use direct quotations in uptaking discourse units, but also because of the different semiotic aspects of direct quotes which are foregrounded in the investigated recontextualization practices. These practices result in MPs’ different public self-positionings toward their role of ‘being an opposition MP’, namely, either as policy-oriented arguers or as ironic evaluators. (shrink)
In 2004, a government-backed Japanese consortium signed an agreement with the government of Iran to develop the major Azadegan oil field. Not only has the project been given the go-ahead despite numerous political obstacles and poor prospects attributed it, but the agreement also appears to be in conflict with Japan's energy policy, materializing from the mid 1980s to date. Consequently it is important to evaluate Azadegan in terms of Japan's evolving oil policy. Three alternative arguments are proposed to (...) evaluate the quality of policy change: Japanspecial relationship’, bureaucratic factors in the energy policy-making process, and the rise of China. The conclusion emerging from this article is that the rise of China and the growing competition between Tokyo and Beijing in the Middle East and elsewhere are the primary factors in Japan's decision to conclude the Azadegan oil deal. The Azadegan case study therefore sets a new precedent for Japanese energy policy which emphasizes the political and strategic rather than economic factors leading this policy formulation. (shrink)
Public engagement is increasingly advocated and applied in the development and implementation of technological innovations. However, initiatives so far are rarely considered effective. There is a need for more methodological rigor and insight into conducive conditions. The authors developed an evaluative framework and assessed accordingly the effectiveness of a project of the Dutch Ministry of Health, Welfare and Sport in which the application of interactive policy making was piloted in medical biotechnology, among others, to increase the legitimacy and quality (...) of the policy agenda. Relevant societal actors, including citizens, were actively involved in an open-exchange, action and reflection process, integrating relevant knowledge. Although the project was overall evaluated as effective, some difficulties were faced. These were mostly related to the novel roles public engagement requires of principal actors. The article concludes that more research is needed on the position of the executing team and on overcoming tensions existing between different governance modes. (shrink)
BackgroundThe Transparency and Openness Promotion Guidelines describe modular standards that journals can adopt to promote open science. The TOP Factor is a metric to describe the extent to which journals have adopted the TOP Guidelines in their policies. Systematic methods and rating instruments are needed to calculate the TOP Factor. Moreover, implementation of these open science policies depends on journal procedures and practices, for which TOP provides no standards or rating instruments.MethodsWe describe a process for assessing journal policies, procedures, and (...) practices according to the TOP Guidelines. We developed this process as part of the Transparency of Research Underpinning Social Intervention Tiers Initiative to advance open science in the social intervention research ecosystem. We also provide new instruments for rating journal instructions to authors, manuscript submission systems, and published articles according to standards in the TOP Guidelines. In addition, we describe how to determine the TOP Factor score for a journal, calculate reliability of journal ratings, and assess coherence among a journal’s policies, procedures, and practices. As a demonstration of this process, we describe a protocol for studying approximately 345 influential journals that have published research used to inform evidence-based policy.DiscussionThe TRUST Process includes systematic methods and rating instruments for assessing and facilitating implementation of the TOP Guidelines by journals across disciplines. Our study of journals publishing influential social intervention research will provide a comprehensive account of whether these journals have policies, procedures, and practices that are consistent with standards for open science and thereby facilitate the publication of trustworthy findings to inform evidence-based policy. Through this demonstration, we expect to identify ways to refine the TOP Guidelines and the TOP Factor. Refinements could include: improving templates for adoption in journal instructions to authors, manuscript submission systems, and published articles; revising explanatory guidance intended to enhance the use, understanding, and dissemination of the TOP Guidelines; and clarifying the distinctions among different levels of implementation.Research materials are available on the Open Science Framework: https://osf.io/txyr3/. (shrink)
In his writings on school choice and educational justice, Harry Brighouse presents normative evaluations of various choice systems. This paper responds to Brighouse's claim that it is inadequate to criticise these evaluations with reference to empirical data concerning the effects of school choice.
The notion of social welfare was created by the paradigm shift from duty‐based to right‐based morality, in which the satisfaction of human needs is a right in line with preserving human dignity. This paper investigates Kant’s view on social welfare in light of redistribution policy. Kant bases his political philosophy on external freedom. Notwithstanding the ethical principles of his philosophy, he is the first prominent thinker to clearly emphasize the necessity of a redistribution policy by the government toward (...) providing for the needs of the poor and the needy. The important question remaining is whether or not the Kantian ideas of external freedom and redistribution for the sake of satisfying the right to social welfare can reach a compromise. It seems that Kant believes the redistribution policy to be not the right of the poor to be provided welfare by the state, but the right of the state, and as such, the states' right to task the people with providing for the welfare of the poor. Such a policy challenges the freedom of the wealthier class and apparently leads to an inconsistency between the two pillars of Kant’s sociopolitical philosophy. The current paper aims to find Kant’s response to this challenge by referring to his scattered arguments. (shrink)
Until now, however, little has been devoted to the results of various abortion policy changes. Legge examines the effects of abortion policy changes on maternal and infant health in the United States, Great Britain, and Eastern Europe.
Chloroquine-resistant plasmodium falciparum malaria is a serious public health threat that is spreading rapidly across Sub-Saharan Africa. It affects over three quarters (80%) of malarial endemic countries. Of the estimated 300-500 million cases of malaria reported annually, the vast majority of malarial-related morbidities occur among young children in Africa, especially those concentrated in the remote rural areas with inadequate access to appropriate health care services. In Liberia, in vivo studies conducted between 1993 and 2000 observed varying degrees of plasmodium falciparum (...) malaria infections that were resistant to chloroquine, including sulfadiazine-pyrimethamine. As the country emerges from a prolonged civil war, the health care delivery system may not be adequately prepared to implement an effective nation-wide malarial control strategy. As a result, the management of uncomplicated malaria in Liberia poses a significant public health challenge for the government-financed health care delivery system. Therefore, based on extensive literature review, we report the failure of chloroquine as an effective first-line drug for the treatment of uncomplicated plasmodium falciparum malaria in Liberia and recommend that national health efforts be directed at identifying alternative drug(s) to replace it. (shrink)
Unreasonable expectations about the nature and character of scientific knowledge support the widespread political assumption that predictive scientific assessments are a necessary precursor to environmental decision making. All too often, the practical outcome of this assumption is that scientific uncertainty becomes a ready-made dodge for what is in reality just a difficult political decision. Interdisciplinary assessments necessary to address complex environmental policy issues invariably result in findings that are inherently contestable, especially when applied in the unrestrained realm of partisan (...) politics. In this article, the authors argue that predictive scientific assessments are inherently limited in the extent to which they can guide policy development and that rigorous scientific assessments can be much more valuable in the role of ex post policyevaluation than they can in the context of ex ante policy formulation. (shrink)
The understanding of teachers? perception of new educational policy is crucial since this perception shapes the policy?s implementation. However, quantitative research in this area is scarce. This article draws on empirical data to investigate whether the school leader might influence his teachers? perception of the new teacher evaluationpolicy. The conceptualisation of teachers? perception consists of three policy characteristics: practicality, need and clarifying function. Our results indicate that school leadership influences teachers? policy perception. More (...) specifically, the structure a school leader provides in a school and the amount of trust teachers have in the school leader have a significant impact on teachers? perception of the practicality of the new teacher evaluationpolicy. (shrink)
: In February 2004, South Korean researchers became the first in the world to successfully harvest stem cells and establish a stem cell line from a cloned human embryo. This is just one of eight possible policy options concerning human embryonic stem cell research. In practice, every kind of stem cell research can be done in one country or another. This paper evaluates the eight policy options concerning human embryonic stem cell research in light of the arguments and (...) decisions behind them. (shrink)
Why do policies fail? How can we objectively choose the best policy from two (or more) competing alternatives? How can we create better policies? To answer these critical questions this book presents an innovative yet workable approach. Avoiding Policy Failure uses emerging metapolicy methodologies in case studies that compare successful policies with ones that have failed. Those studies investigate the systemic nature of each policy text to gain new insights into why policies fail. -/- In addition to (...) providing intriguing directions for research, this book also suggests a bold new standard for evaluating policies. While this method is broadly generalizable, specific examples are provided showing how to develop better Economic Policy, Military Policy, and Constitutional Organizations. This book shows scholars, researchers, and policy analysts how to develop more effective policies so that we may achieve our highest aspirations and avoid the horrendous failures of the past. (shrink)
Surveillance in public health is the means by which people who are responsible for preventing or controlling threats to health get the timely, ongoing, and reliable information they need about the occurrence, antecedents, time course, geographic spread, consequences, and nature of these threats among the populations they serve. “Policy surveillance” is the ongoing, systematic collection, analysis, and dissemination of information about laws and other policies of health importance.
Governments are increasingly using randomized controlled trials (RCTs) to evaluate policy interventions. RCTs are often understood to provide the highest quality evidence regarding the causal efficacy of an intervention. While randomization plays an essential epistemic role in the context of policy RCTs however, it also plays an important distributive role. By randomly assigning participants to either the intervention or control arm of an RCT, people are subject to different policies and so, often, to different types and levels of (...) benefits. In this paper, I identify one necessary condition as well as a set of sufficient conditions for the permissible use of random assignment by government agencies. I argue first that random assignment is permissible only if it is consistent with governments’ duty to realize morally important outcomes. I argue second that random assignment is permissible in cases where investigators are in a state of genuine equipoise regarding all arms of the experiment and the policy to which people have a claim of justice. Finally, I defend a set of conditions under which random assignment is permissible in cases where one or more arms of a policy RCT are reasonably expected to be either superior or inferior to this policy. (shrink)
Research agendas and academic evaluation are inevitably linked. By means of economic incentives, promotion, research funding, and reputation academic evaluation is a powerful influence on the production of knowledge; moreover, it is often conceived as a universal instrument without consideration of the context in which it is applied. Evaluation systems are social constructions in dispute, being the current focus of international debates regarding criteria, indicators, and their associated methods. A universalist type of productivity indicators is gaining centrality (...) in academic evaluation with profound effects on the content of research that is conducted everywhere. Specifically, evaluation systems based on this type of indicators are sending negative signals to scientists willing to conduct research on contextualized agendas, particularly those negotiated with non scientists. On the basis of theoretical and empirical studies documented on the specialized literature and extensive personal engagement with university research policy in Uruguay, we argue that the consolidation of evaluation practices of alleged universal validity deteriorates and discourages a type of research which is undeniably important in developing contexts. (shrink)
Does law matter regarding public health outcomes? Regardless of what one may think about the answer to this age-old question, in recent years the public health community has increasingly demonstrated and recognized the roles that public health laws and policies play in effectuating long-lasting and broad-based population-wide changes. Public health laws and policies have been instrumental in the following ways: reducing smoking prevalence; reducing underage alcohol-related drinking, driving, crashes, and fatalities; reducing exposure to second-hand smoke; eliminating vaccine–associated paralytic poliomyelitis ; (...) increasing seat-belt use and reducing traffic fatalities; reducing dental carries; and reducing access to and consumption of unhealthy foods and beverages sold in schools and to reductions in caloric intake and overweight. In fact, in a review of the ten greatest public health achievements in the 20th century, all were influenced by policy change. (shrink)