The use of sensitivity analysis is routine in some fields of empirical econometrics, although econometric theorists have generally taken a critical attitude towards it. This paper presents a framework in which arguments for and against such analysis can be evaluated. It appears that sensitivity is not necessarily a bad, nor sturdiness necessarily a good.

This paper examines the experiment in macroeconometrics, the different forms it has taken and the rules that have been proposed for its proper conduct. Here an ?experiment? means putting a question to a model and getting an answer. Different types of experiment are distinguished and the justification that can be provided for a particular choice of experiment is discussed. Three types of macroeconometric modelling are considered: the Cowles (system of equations) approach, the vector autoregressive model approach and the computational experiment. (...) JEL Classifications: B41, C5, E37. (shrink)

This article seeks to provide a characterization of theory prevalent in economics and found in many areas of social and natural science, particularly those that make increasing use of rational choice perspectives. Four kinds of theoretical project are identified in which empirical evidence plays a relatively small role in theory acceptance. The paper associates the minor role of evidence in theory formation and acceptance to a need to answer counterfactual questions and argues that is not necessarily incompatible with accounts of (...) science that emphasise truth in the development of theory. However, this view of economics does highlight factors that play a central role in theory acceptance which have not featured very strongly, if at all, in the philosophy of science literature. The article goes on to discuss four areas of economics that illustrate the consequences of using theory acceptance procedures which give relatively little weight to empirical testing. Benefits and costs of counterfactual science as it has developed in economics are discussed. It is concluded that there are good reasons why scientists may not use evidence in theory acceptance, even if an unintended consequence has been to strip economics of valuable empirical sensibilities still evident in natural sciences. (shrink)

This essay examines the role of data and program?code archives in making economic research ?replicable.? Replication of published results is recognized as an essential part of the scientific method. Yet, historically, both the ?demand for? and ?supply of? replicable results in economics has been minimal. ?Respect for the scientific method? is not sufficient to motivate either economists or editors of professional journals to ensure the replicability of published results. We enumerate the costs and benefits of mandatory data and code archives, (...) and argue that the benefits far exceed the costs. Progress has been made since the gloomy assessment of Dewald, Thursby and Anderson some 20 years ago in the American Economic Review, but much remains to be done before empirical economics ceases to be a ?dismal science? when judged by the replicability of its published results. (shrink)

Gul and Pesendorfer (2008) argue that neuroeconomics is evidentially and explanatorily irrelevant to economics, because neuroeconomics and economics ask different questions and utilize different abstractions. They suggest neuroeconomics is only relevant as a source of inspiration for economists. The present paper accepts their basic premise and asks whether the fact that neuroeconomics and economics ask different questions implies that neuroeconomics is irrelevant. The paper argues that Gul and Pesendorfer overlook some important respects in which neuroeconomics is relevant for economics. First, (...) neuroeconomics can improve singular explanations in economics. Second, and more importantly, it improves our understanding of economic phenomena. And finally, it helps us assess the plausibility of our conjectures concerning economic phenomena. It may be true that neuroeconomics will not revolutionize economics (at least in the short run), but it is more than a source of inspiration. (shrink)

Traditionally, evidence in economics has been seen in the context of theory choice. Much of recent methodological debate on the role of evidence has turned on the recognition that the status and role of evidence is somewhat more involved in economics than the conventional wisdom suggests. Rather than approaching this question in general terms from a starting point of philosophy of science or even science studies, our aim in this introduction to a symposium of articles on evidence in economics is (...) to approach the question from the perspective of applied economics. (shrink)

This survey of the symposium papers argues that the problem of data mining should be of interest to both practicing econometricians and specialists in economic methodology. After summarizing some of the main points to arise in the symposium, it draws on recent work in the philosophy of science to point to parallels between data mining and practices engaged in routinely by experimental scientists. These suggest that data mining might be seen in a more positive light than conventional doubts about it (...) imply. (shrink)

This paper outlines some of the main methodological issues to arise in macroeconomics, making the case that the methodological issues arising in macroeconomics are just as important as those arising in microeconomics and that they merit more attention. Focusing on the symposium to which it forms the Introduction, the paper discusses three such issues: can macroeconomic theories be tested? Do macroeconomic theories change in response to evidence? Is contemporary macroeconomics in good methodological shape?

A neglected critique of social science laboratories alleges that they implement phenomena different to those supposedly under investigation. The critique purports to be conceptual and so invulnerable to a technical solution. I argue that it undermines some economics designs seeking to implement features of real societies, and counsels more modesty in experimental write?ups. It also constitutes a plausible argument that laboratory economics experiments are necessarily less demonstrative than natural scientific ones. More radical sceptical conclusions are unwarranted.

Journal of Economic Methodology, Volume 18, Issue 4, Page 427-432, December 2011.

While it is widely alleged that economists do not like or use questionnaire surveys, the facts are considerably more complicated. Economists make extensive use of survey information on such things as prices and employment, and the use of ?contingent valuation? surveys has exploded recently. The paper reviews the historical debate that led to economists? seeming distrust of surveys. It then investigates why there is extensive use of surveys in the face of methodological strictures against survey use. To do this, the (...) paper develops a typology of kinds of information that can be gathered by surveys and considers whether anti-survey arguments apply with equal force to the various categories. Differing methodological uses for survey data are then considered, using a number of actual survey literatures as illustrations. Finally, the paper presents strategies for improving the use of survey data in economics. (shrink)

The Representational Theory of Measurement conceives measurement as establishing homomorphisms from empirical relational structures into numerical relation structures, called models. There are two different approaches to deal with the justification of a model: an axiomatic and an empirical approach. The axiomatic approach verifies whether a given relational structure satisfies certain axioms to secure homomorphic mapping. The empirical approach conceives models to function as measuring instruments by transferring observations of an economic system into quantitative facts about that system. These facts are (...) evaluated by their accuracy and precision. Precision is achieved by least squares methods and accuracy by calibration. For calibration standards are needed. Then two strategies can be distinguished. One aims at estimating the invariant (structural) equations of the system. The other strategy is to use known stable facts about the system to adjust the model parameters. For this strategy, the requirement of models as homomorphic mappings has been dropped. (shrink)

The metrology literature neglects a strong empirical measurement tradition in economics, which is different from the traditions as accounted for by the formalist representational theory of measurement. This empirical tradition comes closest to Mari's characterization of measurement in which he describes measurement results as informationally adequate to given goals. In economics, one has to deal with soft systems, which induces problems of invariance and of self-awareness. It will be shown that in the empirical economic measurement tradition both problems have been (...) on the agenda for a long while, and that the proposed solutions to these problems provide clues for the directions in which one could develop a measurement theory that takes account of soft systems. (shrink)

The contest between rational and behavioral finance is poorly understood as a contest over 'testability' and 'predictive success.' In fact, neither rational nor behavioral finance offer much in the way of testable predictions of improving precision. Researchers in the rational paradigm seem to have abandoned testability and prediction in favor of a scheme of ex post 'rationalizations' of observed price behavior. These rationalizations, however, have an unemphasized relevance for behavioral finance. While behavioral finance advocates may justly criticize rationalizations as unlikely (...) to lead to a science of financial economics with improving predictive power, rational finance's explanatory power plays a key role supporting the limits of arbitrage arguments that make behavioral finance possible. (shrink)

In a recent paper, Cavusoglu and Tebaldi (2006) provided an evaluation of neoclassical and endogenous growth theories according to Lakatos's methodology of scientific research programmes. This paper offers three criticisms of their contribution as well as a rival Lakatosian appraisal of growth theories. First, we hold that Cavusoglu and Tebaldi do not provide a proper structure of theory comparison in their contribution. Second, we argue that they use an inadequate version of Lakatos's appraisal criterion. Third, against the claim (...) of the authors, we show that there are seminal endogenous growth models, which predict income convergence among countries. Finally, in contrast to Cavusoglu and Tebaldi, our analysis suggests that by Lakatos's standards, Schumpeterian variant of endogenous growth theory is both theoretically and empirically progressive over neoclassical growth theory. (shrink)

Understanding the factors determining economic growth has been a major concern for economists and governing bodies for many years. The Solow growth model and the endogenous growth models are the main theories tested and used in the growth literature. This paper discusses the main contributions to economic methodology and uses Lakatos's scientific research program framework to evaluate the main theoretical contributions to growth theory. Based on Lakatos's ideas, Solovian models are both empirically and theoretically progressive. Endogenous growth models, on the (...) other hand, are not empirically corroborated, and thus not progressive in Lakatosian sense. However, there are many reasons for rational growth economists to continue working in this field, even facing empirical refutations of an important theoretical prediction. Endogenous growth models are promising in terms of generating enormous gains that will help us better understand the mechanisms of economic growth. (shrink)

Davis argues that Suppe's semantic conception provides a better understanding of the problem of theory?data confrontations. Applying his semantic methodology to the LSE (London School of Economics) approach of econometrics, he concludes that the LSE approach fails to address the issue of bridging the theory?data gap. This paper suggests two other versions of the semantic view of theories in the philosophy of science, due to Suppes and van Fraassen, and argues that the LSE approach can be construed under these two (...) versions of the semantic view in terms of structure and representation. (shrink)

In 2007?2008, the world economy came perilously close to a systemic failure in which a financial system collapse almost undermined the entire world economy as we know it. These events have led some to fault the economics profession for its failure to predict the crisis, and to ask whether the crisis will lead the economics profession to change its ways. In this paper, I will discuss these two issues, and then turn to some suggestions for institutional changes in the economics (...) profession that might lead to better outcomes in the future. (shrink)

A positive view of data-mining has been recently presented in a Journal of Economic Methodology ( JEM ) symposium. This is in stark contrast to the stance normally taken. In this note consideration of the Bayesian philosophy of science literature and the impact of data revision extends the analysis of data-mining. Introduction of these issues is seen to provide support for the arguments presented in the JEM symposium.

The philosophy of science literature has played an increasing role in discussion of econometric methodology in recent years, and the Hendry methodology in particular has received much attention. Despite this, the encompassing principle has been overlooked in the methodological literature. This paper addresses this by examining the major methodological implications of the principle.

The puzzle: why do so many economists in principle acknowledge the importance of creative destruction, and yet in practice give so little attention to creative destruction in what they teach and what they research? The answer lies, in part, in the difficulty of obtaining what is viewed as ?hard? evidence in support of some of the central claims. For example, one such claim is that new products contribute more to consumer well-being than price competition on old products. The only kind (...) of evidence accepted by much of the profession is the testing of econometric hypotheses generated from formal models. The sort of evidence found in persuasive sources such as DeLong's ?Cornucopia? consists of historical examples and raw time series. I argue that in the short run, a more pluralistic methodology would be better, and that in the long run, we should seek to understand which methods work best under which circumstances. (shrink)

Ever since the formation of the field of economic methodology in the 1990s, doubts have been raised about its discursive closure from both inside and outside the field. Rather than embarking on a programmatic discussion, I present a historical narrative regarding the conditions of the formation of the field, which may have necessitated this closure. These conditions are found in the role methodological reflections played in the formalist revolution of the 1950s and in its critique in the 1970s. Both episodes (...) gave occasion to, but did not require the import of philosophy of science in the mid-1970s. Since the 1980s, when the field became separately established through post-Popperian methodology, it remains an open question whether this imposition still holds. (shrink)

Economists often try to make plausible inferences from a sizable empirical literature addressing a particular measurement, direction-of-effect, or testing issue. There are serious methodological problems associated with drawing such inferences. This article sets out some of these problems in order to make a case for their importance. After discussing these problems, the paper presents three case study examples of inference difficulties in specific literatures. It then proposes a new hypothesis about the time pattern of publication bias in empirical economics literatures. (...) As support for this hypothesis, it presents evidence that ?reversals in findings? in empirical literatures in economics are not uncommon. Similarities are pointed out between the focus on inference problems in this paper, and the meta-analysis literatures in psychology and medical clinical trials. (shrink)

Some empirically minded philosophers of science argue that the evidence should choose the best theory from among theoretical rivals. However, the evidence may not speak clearly, a problem of 'underdetermination of theory by data'. We examine this problem in a concrete setting, rival theories of smoking behaviour. We investigate whether several uncontested pieces of empirical evidence allow us to choose between two competing theoretical perspectives on smoking, rational choice and non-rational choice, respectively. Next, we develop a more refined taxonomy of (...) smoking theories, and consider the consequences for theory testing. Finally, we examine some normative aspects of theory choice involving the appropriate scope of government action. (shrink)

Does empirical work in economics both provoke and test theoretical models, or does model development proceed according to a theory-oriented research program, with little interaction with empirics? Robert Solow and Richard Lipsey have articulated different visions of this relationship. This paper: (i) describes these competing Solow versus Lipsey views; (ii) presents examples illustrating each view; and (iii) draws inferences about factors promoting a close relation between empirics and modeling. Three examples are examined in detail: the ?nursing shortage? literature; Lind's analysis (...) of recent rent control models; and a wide-ranging evaluation of ?is there too little theory in development economics?? by leading development economists. Various factors promoting or inhibiting a close connection between modeling and empirics are identified. (shrink)

This paper examines some economic forecasts made in late 1930 that were intended to predict economic activity in the United States in order to shed light on several methodological issues. We document that these forecasts were extremely optimistic, predicting that the recession in the US would soon end, and that 1931 would show a recovery. These forecasts displayed egregious errors, because 1931 witnessed the largest negative growth rate for the US economy in any year in the twentieth century. A specific (...) question is what led forecasters to make such serious and substantial empirical errors. A second more general issue involves the methodology of forecasting. The 1930 forecasts were sometimes based on explicit analogies with previous serious business cycles. Modern forecasting approaches are based on techniques that may not be recognized as analogies. Using the 1930 forecasts, we examine the implicit?analogy content of forecasts, and what might render such implicit analogies valid or invalid. This 1930 forecast example also resonates beyond the confines of economic methodology because forecasts about the Great Depression are of continuing interest to the profession at large, and we produce a forecast series not previously available. (shrink)

This paper argues classical statistics and standard econometrics are based on a desire to meet scientific standards for accumulating reliable knowledge. Science requires two inputs, mining of existing data for inspiration and new or 'out-of-sample' data for predictive testing. Avoidance of data-mining is neither possible nor desirable. In economics out-of-sample data is relatively scarce, so the production process should intensively exploit the existing data. But the two inputs should be thought of as complements rather than substitutes. And we neglect the (...) importance of out-of-sample testing in the production of reliable knowledge. Avoidance of data-mining is not a substitute for tests conducted in new samples. The problem is not that data-mining corrupts the process, the problem is our collective neglect of out-of-sample encompassing, stability and forecast tests. So the data-mining issue diverts us from the crucial margin. (shrink)

This paper considers whether game theory can be tested, what difficulties experimenters face in testing it, and what can be learned from attempts to test it. I emphasize that tests of game theory rely on fallible assumptions concerning particular features of the strategic situation and of the players. These do not render game theory untestable in principle, but they create serious problems. In coping with these problems, experimenters may use game theory to learn what games experimental subjects are playing.

In this paper I examine Don Ross’s application of unificationism as a methodological criterion of theory appraisal in economics and cognitive science. Against Ross’s critique that explanations of the preference reversal phenomenon by the ‘heuristics and biases’ programme is ad hoc or ‘Ptolemaic’, I argue that the compatibility hypothesis, one of the explanations offerd by this programme, is theoretically and empirically well-motivated. A careful examination of this hypothesis suggests several strengths of a procedural approach to modelling cognitive processes underlying individual (...) decision making, compared to a multiple-agent approach which Ross promotes. I argue that the debate between economists and psychologists are both theoretical and empirical, but cannot be resolved by appealing to the ideal of unification. (shrink)

This article assesses two major conceptual arguments against theories of choice.The first argument concerns the circularity of belief-desire psychology, on which decision theory is based. The second argument concerns the normativity arising from the concept of rationality. Each argument is evaluated against experimental practice in economics and psychology, and it is concluded that both arguments fail to establish their skeptical conclusion that there can be no science of intentional human actions.

In this paper I study how the theoretical categories of consumption theory were used by Milton Friedman in order to classify empirical data and obtain predictions. Friedman advocated a case by case definition of these categories that traded theoretical coherence for empirical content. I contend that this methodological strategy puts a clear incentive to contest any prediction contrary to our interest: it can always be argued that these predictions rest on a wrong classification of data. My conjecture is that this (...) methodological strategy can contribute to explain why Friedman’s predictions never generated the consensus he expected among his peers. (shrink)

In this paper I explore a positivist methodological tradition in early demand theory, as exemplified by several common traits that I draw from the works of V. Pareto, H. L. Moore and H. Schultz. Assuming a current approach to explanation in the social sciences, I will discuss the building of their various explanans, showing that the three authors agreed on two distinctive methodological features: the exclusion of any causal commitment to psychology when explaining individual choice and the mandate to test (...) the truth of demand theory on aggregate data by statistical means. However, I also contend, from an epistemological point of view, that the truth of demand theory was conceived of in three different ways by our authors. Inspired by Poincaré, Pareto assumed that many different theories could account for the same data on individual choice, coming close to a kind of conventionalism -though I prefer to refer to this position as theoreticism. Moore was himself akin to Pearson's approach, which could be named descriptivist insofar as it resolved scientific laws into statistical descriptions of the data. Finally, Schultz tried to reconcile both approaches in an adequationist stance with no success, as we shall see. (shrink)