Abstract
Are the categories used to study the social world and acting on it “real” or “conventional”? An empirical answer to that question is given by an analysis of the debates about the “quality” of statistics produced by the European National Institues of statistics in the 1990s. Six criteria of quality were then specified: relevance, accuracy, timeliness, accessibility, comparability and coherence. How do statisticians and users of statistics deal with the tension produced by their objects being both “real” (they exist before their measurement) and “conventionally constructed” (they are in a way, created by these conventions)? In particular, the technical and sociological distinction between the criteria of relevance and accuracy implies a realistic interpretation, desired by users, but that is nonetheless problematic.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Some are surprised that the list of criteria contains no allusion to ethical issues such as data confidentiality, neutrality or independence from political pressure. The apparent reason is that the list was prepared from a customer-oriented standpoint focused on the needs of statistics users. The list therefore represents only a part of what, more generally, society is entitled to expect from its statistical system.
The only exception is an allusion to the crucial distinction between the uses of statistics as a “public good” (or “universal service”), i.e., free of charge and intended for “citizens of a democratic society,” and as a “private good,” billed to “customers” in the context of a market relationship. This distinction also occurs in the debates on the privatization (or transformation into “agencies”) of certain public services.
On the relationship between the two aspects of industrial quality—production and customers—see Duclos (1996).
This analysis is less true for France than for other countries. The distinction is less clearcut in France, where the “economist-statisticians” graduating from the National School of Statistics and Economic Administration (École Nationale de la Statistique et de l’Administration Économique: ENSAE) and National School of Statistics and Information Analysis (École Nationale de la Statistique et de l’Analyse de l’Information: ENSAI) receive training that provides them—so it is assumed—with both competencies. In practice, they often work in both areas. French universities, as well, offer programs in mathematical statistics, but their graduates have few contacts with the official statistical system. Their counterparts in the Anglo-Saxon countries often work under contract for NSIs or Eurostat, but that is seldom the case in France.
One of the incentives devised by the promoters of the TQM procedure is a series of awards to the best European NSIs for their successful compliance with these criteria (the awards are handed out in very formal ceremonies). We find a similar use of quantification to compare and score high schools and hospitals in the ranking tables published by the French press since the 1990s.
The division of labor is often more complex: methodologists are responsible for the theoretical design of the survey (sampling plan, sample selection, checks, adjustments, estimation of variables and their confidence intervals, etc.), but data collection in the field is, in some countries, often subcontracted to specialized private firms. This further increases the need for written pre-objectivation of “quality criteria”.
These problems encountered by statisticians in their work, and the sometimes bitter controversies they elicit, are comparable to the problems faced by journalists, who are also often torn between conflicting requirements (Lemieux 2000).
French government bodies use the English term (which may be rendered in French as étalonnage or référence) to denote this construction of a European equivalence space whose aim is to achieve convergence in NSI performances (most notably for publication timeliness).
Traders see higher unemployment as a short-term signal of a decline in the “inflation risk”—a risk that would prompt the Fed to lift its key rates, thereby slowing economic activity. In the longer run, higher unemployment is seen as putting downward pressure on wages and hence driving up profits.
Some material on this may be found, for example, in the work of Armatte (1992), Morgan (1990), and Klein (1997). Klein’s book, Statistical Visions in Time: A History of Time Series Analysis, 1662–1938, offers an extraordinary survey of the evolution of these forms of rhetoric and of these tools for untangling realities from the flow of time.
On this topic, see the debate organized by Florence Weber in Genèses, 9, October 1992, pp. 90–119. In this case, the epistemological compromise put forward by the challenged authors consists in falling back on the notion of “order of magnitude:” the convention concerns only the accuracy of the measurement, but not an intrinsic reality, which is assumed to be optimally proxied by the conventional algorithms proposed. This type of argument should be studied systematically, by viewing it as one of a range of formulations available for managing the tension between realism and conventionalism.
The Dutch and Scandinavian NSIs have long chosen to develop their official statistics from administrative records, reducing the share of direct surveys.
The notion of order of magnitude is ambiguous, being midway between the imprecise measurement, which pertains to the field of classical metrology, and the indicator, which separates the measurement and the thing measured. It does not make a clear choice between the two alternatives.
We shall not return here to the previously mentioned hypothesis that a statistic is all the more real as it is more solidly built and anchored in stable institutions (Desrosières 1998). This “compromise solution” helps to understand many historical situations and to study them empirically.
References
Akerlof, George. 1970. The market for ‘lemons’: Quality uncertainty and the market mechanism. Quarterly Journal of Economics 84: 488–500.
Anderson, Margo. 1988. The American census: A social history. New Haven: Yale University Press.
Armatte, Michel. 1992. Conjonctions, conjoncture et conjecture. Les baromètres économiques (1885–1930). Histoire et mesure VII-1/2: 99–149.
Armatte, Michel. 1995. Histoire du modèle linéaire. Formes et usages en statistique et économétrie jusqu’en 1945. Doctoral dissertation, EHESS, Paris.
Bayart, Denis. 1998. Statistique mathématique et gestion de la qualité. Recherches sur l’histoire de cognition dans l’organisation industrielle, Cahier du CRG (École Polytechnique), no. 14, May 1998.
Blondiaux, Loïc. 1998. La fabrique de l’opinion. Une histoire sociale des sondages. Paris: Seuil.
van Bochove, Cornelis. 1996. From assembly line to electronic highway junction: A twin-track transformation of the statistical process. Netherlands Official Statistics 11: 5–36.
Boltanski, Luc, and Eve Chiapello. 2007. The new spirit of capitalism. London: Verso.
Cicourel, Aaron. 1964. Method and measurement in sociology. New York: The Free Press of Glencoe.
Conein, Bernard, and Eric Jacopin. 1994. Action située et cognition. Le savoir en place. Sociologie du travail, XXXVI 4: 475–500.
Desrosières, Alain. 1998. The politics of large numbers: A history of statistical reasoning. Cambridge, Mass: Harvard University Press.
Desrosières, Alain. 1999. La statistique aux Pays-Bas. Informatisation et intégration, un projet futuriste, Courrier des statistiques, no. 91–92, December 1999, 51–59. Paris: INSEE http://www.insee.fr/fr/ffc/docs_ffc/cs91g.pdf.
Didier, Emmanuel. 2000. De l’échantillon à la population. Sociologie de la généralisation par sondage aux États-Unis avant la seconde guerre mondiale. Dissertation, Centre de Sociologie de l’Innovation, École Nationale Supérieure des Mines de Paris.
Duclos, Laurent. 1996. L’exigence de qualité suffit-elle pour porter la parole du client?. Les cahiers de recherche, GIP Mutations industrielles 70: 27–36.
Elissalt, François. 2001. La statistique communautaire au tournant du XXIème siècle. Nouveaux enjeux, nouvelles contraintes. Courrier des statistiques, no. 100, December 2001, 41–51. Paris: INSEE, http://www.insee.fr/fr/ffc/docs_ffc/cs100i.pdf (in English: “European statistics at the dawn of the twenty-first century,” Courrier des statistiques, English series, no. 8, 2002 [forthcoming], Paris: INSEE).
Eymard-Duvernay, François. 1989. Conventions de qualité et formes de coordination. Revue Économique 40(2): 329–359.
Favret-Saada, Jeanne. 1977. Les mots, la mort, les sorts. Paris: Gallimard.
Grais, Bernard. 1998. Harmonisation statistique et qualité: le cas des statistiques sociales. Paper at Eurostat Seminar in Mondorf on “The Future of European Social Statistics” (4th session, March 26–27, 1998).
Hacking, Ian. 2000. Façonner les gens: Le seuil de pauvreté. In L’ére du chiffre. Systèmes statistiques et traditions nationales, ed. J.P. Beaud, and J.G. Prévost, 17–36. Montreal: Presse de l’Université du Québec.
Hopwood, Anthony, and Peter Miller (eds.). 1994. Accounting as social and institutional practice. Cambridge (UK): Cambridge University Press.
Klein, Judy. 1997. Statistical visions in time: A history of time series analysis, 1662–1938. Cambridge (UK): Cambridge University Press.
Lemieux, Cyril. 2000. Mauvaise presse. Une sociologie compréhensive du travail journalistique et de ses critiques. Paris: Métailié.
Marchand, Olivier, and Claude Thélot. 1991. Deux siècles de travail en France. Paris: INSEE.
Martin, Olivier. 1997. La mesure de l’esprit. Origines et développements de la psychométrie, 1900–1950. Paris: L’Harmattan.
Morgan, Mary. 1990. The history of econometric ideas. Cambridge (UK): Cambridge University Press.
Patriarca, Silvana. 1996. Numbers and nationhood. Writing statistics in nineteenth-century Italy. Cambridge (U.K.): Cambridge University Press.
Platek, Richard, and Sarndal, Carl-Eric. 2001. Can a statistician deliver?. Journal of Official Statistics 17: 1–20, no. 1, March, Stockholm: Statistics Sweden. http://www.jos.nu/Contents/issue.asp?vol=17&no=1.
Porter, Theodore. 1994. Making things quantitative. In Accounting and science, ed. M. Power, 36–56. Cambridge (UK): Cambridge University Press.
Thevenot, Laurent. 1983. “L’économie du codage social,” Critiques de l’économie politique, no. 23/24, April-Sept. 1983; Théorie économique et pratiques sociales, La Découverte/Maspero, pp. 188–222.
Thomas, Ray. 1996. Statistics as organizational products. Sociological Research Online 1, no. 3. http://www.socresonline.org.uk/socresonline/1/3/5.html.
Vanoli, André. 2005. A history of national accounting. Amsterdam: IOS Press.
Weber, Florence, et al. 1992. “Histoire et statistique. Questions sur l’anachronisme des séries longues”, Genèses, 9, 90–119. Paris: Belin.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Desrosières, A. How to be Real and Conventional: A Discussion of the Quality Criteria of Official Statistics. Minerva 47, 307–322 (2009). https://doi.org/10.1007/s11024-009-9125-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11024-009-9125-3