1 Introduction

By an experiment, in the common sense of the term, we mean a procedure with two major components. An intervention is performed, usually under specified prior conditions. In connection with it, observations of changes (or lack of changes) are made. This is done in order to discover or verify what changes (if any) follow regularly after performance of this type of intervention under similar conditions. Since experiments are performed in order to gain knowledge about regular patterns of events, such a combination of intervention and observations has to be repeatable (replicable) in order to fulfil the epistemic function of an experiment.Footnote 1

We associate experiments with science, but the practice of making experiments is much older than modern science or its recognized precursors. Unfortunately, the experimental traditions that are prior or parallel to science have escaped philosophical attention. A major reason for this may be that their purpose differs from that of the experiments that have mostly been at focus in the philosophy of science. With respect to purpose, there are two major types of experiments, epistemic and directly action-guiding experiments.Footnote 2 (Hansson 2015) Discussions in the philosophy of science have predominantly focused on the former category of experiments, whereas experiments in traditions outside of (recognized) science belong almost exclusively to the latter category.

An epistemic experiment aims at providing information about the workings of the world we live in. Therefore, the regularities looked for are such that can reveal mechanisms and propensities of the study objects. In typical cases, the intervention introduces or eliminates some factor that is believed to contribute causally to some effect, and the outcome is taken to confirm or disconfirm a cause-effect relationship between the (type of) intervention and the (type of) effect under the given (type of) circumstances. In contrast, a directly action-guiding experiment has a practical purpose. It is performed in order to find out whether some intervention can be used to achieve a specified practical purpose. In this case as well, a successful experiment is typically taken to establish a cause-effect relationship. However, the demands on that relationship differ. In an epistemic experiment it has to serve explanatory or other epistemic purposes. That does not apply in a directly action-guiding experiment, but instead the effect of the cause-effect relationship has to be some desirable practical attainment.

We can define a directly action-guiding experiment as the performance of some action X in order to determine whether, or to what extent, a desirable result Y follows, with the intention that this information should be useful on future occasions when the attainment of Y is desired. The reference to future usefulness is important since it marks the difference between performing an experiment and (merely) trying to do something.Footnote 3 This definition also provides a basic justification for requiring that these experiments should be repeatable: Their purpose is to tell us how certain effects can be obtained in the future, in other words to deliver useful action recipes. This is only possible if the experiment discloses what will happen in general, under circumstances that can be specified.

The X and Y of the definition can be of many different types. Experimentation can aim at highly general knowledge that can be applied in a wide range of circumstances, or it can aim at local knowledge that is only intended for use in some particular place or situation.Footnote 4

The vast majority of experiments in basic (curiosity-driven) science are epistemic. This is true of the famous experiments that are often referred to in the philosophy of science, such as the experiments by Mikhail Lomonosov and Antoine Lavoisier showing the conservation of mass in chemical reactions, Léon Foucault’s pendulum experiment, which confirmed the earth’s rotation, Gregor Mendel’s experiments with crossing of garden peas, and Edward Morley’s experimental demonstration that light has the same speed irrespective of its direction in space. Many discussions on the history and methodology of experimentation only refer to this type of experiments. However, a large part of the experiments performed in modern science are directly action-guiding. Major examples of this are clinical trials, agricultural field trials, tests of technological constructions, and various forms of policy experimentation.

As already mentioned, directly action-guiding experiments differ from epistemic experiments in having a long-standing history that reaches back far beyond modern science and its forerunners in classical antiquity. There is ample evidence that craftspeople have experimented long before there was a learned experimental tradition. (Hansson 2015) The same applies to farmers.Footnote 5 It is inconceivable that agriculture could at all have arisen and evolved without extensive experimentation. Indeed, the evidence suggests “that people everywhere began by experimenting with the cultivation of plants they were already collecting in the wild” (Bray 2000). Experimental practices have been documented in traditional agriculture all around the world. In Europe, agricultural experimentation was almost exclusively driven by the farmers themselves until the mid nineteenth century, when scientists gradually took over. (Pretty 1991) The experimental traditions in indigenous agricultural communities were scarcely noticed by Western scholars until the 1970s, when anthropological studies of experimenting farmers began to appear. (Johnson 1972; Richards 1979).

Due to extensive field work performed by anthropologists and also by agricultural researchers, there is now a large literature on farmers’ experiments. Most of it is focused on practical issues, such as the potential for fruitful co-operation between experimenting farmers and agricultural scientists, but it also contains a wealth of information on how farmers in different environments and under different social conditions perform their experiments. None of the other traditions of directly action-guiding experiments outside of modern science, such as those of various crafts and trades, seem to have been documented to a comparable degree. Therefore, the literature on farmers’ experiments is a unique and untapped reservoir of information for studies of directly action-guiding experiments outside of modern science.

The extension of the scope of study objects for the philosophy of experimentation that is proposed here and in Hansson (2015) can be seen as a further step in a process that has already been going on for about four decades. As was noted for instance by Galison (1987, p. 2), Steinle (2002, p.2), and Karaca (2013, p.2), the major empiricist philosophers of science, including Hempel, Braithwaite, Nagel, Popper, and Kuhn, treated experimentation as exclusively devoted to the testing of scientific hypotheses. This means that the experiments they considered are a proper subset of the epistemic experiments performed in science. Ian Hacking’s (1983) book Representing and Intervening was the starting-point of a new movement, which is now known as the “New Experimentalism” (Ackermann 1989). Major tenets of this movement were that experiments are in need of a more careful study than what they had previously received (Franklin 1986) and that experimentation “enjoys a certain amount of autonomy from theory” and “should be studied for its own sake, and not merely as subordinate to scientific theorizing” (Karaca 2013, pp. 126 and 95). Starting in the late 1990s, several participants in this movement turned their attention to exploratory experiments, i.e. experiments searching for empirical regularities, performed without theory-based expectations on which regularities might be found. (Burian 1997; Steinle 1997, 2002; Franklin 2005a, b; O'Malley 2007; Karaca 2013, 2017; Colaço 2018) However as can be seen from numerous definitions and descriptions of the research area, this literature still has an exclusive focus on epistemic experiments. Directly action-guiding experiments performed by scientists, such as clinical trials, are rarely mentioned, and I have seen no signs of awareness of any of the experimental traditions that prosper in communities outside of modern science. (Ackermann 1988, 1994; Franklin 1986, pp. 103–104; Franklin 2005a, b, p. 3; Galison 1987, pp. ix-x; Gooding et al. 1989; Hacking 1983; O'Malley 2007, p. 339) The present contribution aims at a further widening of the scope of philosophical studies of experimentation to cover all kinds of experiments, both epistemic and directly action-guiding, whether or not they are part of a scientific tradition.

In what follows, farmers’ experiments will be treated on par with modern scientific experiments in a discussion of some of the major methodological issues that are relevant for the reliability of experiments as guidance to practical action. Section 2 briefly summarizes the experimental activities performed by farmers: their extent, purpose and forms, as well as the characteristics of the farmers who perform them. Section 3 describes and analyses the means that traditional farmers have to identify potential practices that are promising enough to be worth experimenting with. Section 4 is devoted to the planning and actual conduct of experiments. It relates farmers’ experimental activities to well-known requirements on scientific experiments such as controls, randomization, and replication. Both similarities and differences between farmers’ experiments and modern scientific experiments are identified. In Section 5, the differences are further discussed, and farmers’ experiments are found to have both advantages and disadvantages, compared to the field trials conducted by agricultural scientists. Section 6 provides some examples of how investigations of traditional issues from the philosophy of science can gain from including the experimental traditions of farmers in philosophical studies of experimentation. Section 7 concludes.

2 The prevalence and importance of farmers’ experiments

As already mentioned, farmers’ experiments have been reported from all over the world, not least from traditional cultures that have had very limited contact with modern science. In many of these cultures, experimentation is recognized as a specific type of activity. Several languages have special words for “experiment”, such as “hugo” or “saini” in Mende (spoken in Sierra Leone), “shifleli” in Bambara (spoken in Mali), and “igerageza” in Kinyarwanda (spoken in Rwanda and neighbouring countries). (Richards 1985, p. 98; Stolzenbach 1999; den Biggelaar 1996).Footnote 6

In farming, experimentation is a necessity, not a luxury. This is because nature changes and evolves. Due to natural evolution, pests and weeds never cease to pose new challenges. The properties of cultivated soil change significantly over the years, often as a result of the farmer’s own actions. Therefore, in order to feed one’s family, it is not sufficient to rely on the knowledge of one’s forefathers. New methods and new seeds have to be tried out continuously, just to keep pace with changes in the local conditions. (Box 1988, p. 65) In sharp contrast to the common misconception of traditional farming as unchanging, farming has always and everywhere been in an unending process of change, largely driven by experimentation.Footnote 7

Experimentation by traditional farmers is often an integrated component of a sophisticated risk management strategy. Contrary to modern commercial farming, traditional subsistence farming aims at minimizing the risk of food shortage, rather than maximizing the expected yield. (Johnson 1972, pp. 150–151; van Beek 1993, p. 55) By constantly experimenting with many crops and varieties, as well as locations and practices, farmers spread the risks and make it as unlikely as possible that everything will go wrong at the same time. This strategy has been reported from many parts of the world. (Altieri 2002, p. 3; Brookfield and Padoch 1994, pp. 41–42; Mokuwa et al. 2014, p. 14) Agricultural experimentation appears to be particularly intensive in harsh and capricious climates where the need for well-developed risk management is larger than elsewhere. For instance, traditional Andean agriculture employs a “massive parallelism” in response to fickle natural conditions. It is not uncommon for a family to have around thirty fields in different locations, each bearing several crops. Experimentation aiming to find crops suitable for different soils and climate conditions is an integral part of this form of farming. (Earls 2011) Farmers in the Himalayas have developed similar strategies of diversity and intensive experimentation in response to similar natural circumstances. (Schroeder 1985).

One of the most consistent features of indigenous agricultural experiments all over the world is that novelties are first tried out on a small scale. New crops, varieties, or methods are first tested on a small plot. (de Schlippe 1956, pp. 220–222; Lightfoot 1987, p. 81; Sumberg and Okali 1997, p. 38; González 2001, pp. 220–221; Bentley et al. 2005, p. 99; Stone 2007, p. 80; Leitgeb et al. 2008, p. 14; Stone 2016, p. 5) This minimizes the consequences for food supply if the innovation fails. Furthermore, many more experiments can be performed in this way than if larger areas were used. In many cases, the experimental plots are close to the home, which facilitates surveillance of the experiment. (Johnson 1972, p. 155; Richards 1985, pp. 98 and 145; Pretty 1991, p. 139; Bentley 2006, pp. 452–453; Kummer 2011, p. 68) Importantly, the use of small plots confirms that this is experimentation and not “merely trying”. An attempt to increase this year’s harvest would have to be large enough to make a noticeable difference in the total harvest if it succeeds. In contrast, an experiment is intended to provide knowledge that can be used to increase harvests in coming years. Therefore, an experiment can be so small that it does not contribute significantly to this year’s harvest.

There is ample evidence that experimentation is ubiquitous among farmers. It is something that almost every farmer does, not something performed only by a few, especially dedicated farmers. (Bentley 1994, p. 143; Sumberg and Okali 1997, pp. 45–46; Nielsen 2001, p. 97; Bentley 2006; Leitgeb et al. 2014; Vogl et al. 2015, p. 140) Experimentation seems to be performed spontaneously, rather than as the result of some exceptional stroke of genius. In one case, extensive experimental traditions developed among a previously pastoral people when circumstances forced them to take up farming. (Boven and Morohashi 2002, pp. 93–103) That being said, there are also many examples of individual farmers who have taken a leading role in local experimental activities, for instance by keeping a large number of crop varieties, bringing seeds and ideas from other communities, or inventing new work methods. (Harold Brookfield and Padoch 1994, p. 40; Cleveland et al. 2000, p. 382; Dixon 2005, pp. 313–314; Beckford and Barker 2007, pp. 124–125; Leitgeb et al. 2011, p. 359) Since women perform much of the agricultural work in many traditional societies, it should be no surprise that women are often the principal experimenters, not least in experiments concerning breeding and seed selection. (Abay et al. 2001; Nasr et al. 2001; Ong’ayo et al. 2001; Saad 2001, pp. 9–10; Boven and Morohashi 2002, pp. 69–71).

The ubiquity of agricultural experiments does not mean that they are equally well developed everywhere. As already mentioned, experimental activities tend to be particularly vigorous in regions where agriculture is more difficult and precarious than elsewhere. The social conditions of experimentation are also important. Just like scientists, experimenting farmers need freedom of thought and innovation. These conditions can be threatened in a hierarchical system whose members are socially expected to do as their elders, and discouraged from trying something new. At least one example is documented of a traditional culture where experimentation has been hampered by such limitations. (Nielsen 2001, pp. 101–102).

3 Hypothesis formation

Directly action-guiding experiments can only be successful if there are plausible action recipes available for testing. There are many examples in the literature showing that causal and mechanistic understanding can guide farmers in the construction of experiments. Perhaps most obviously, knowledge that plants need water has informed the development of various techniques for watering and irrigation, including more unusual methods such as planting in deep holes to reach groundwater. (Ong’ayo et al. 2001, p. 116) Often, observations of the effects of non-experimental actions and events have led to hypotheses about intervention–effect associations that have then been tested and shown to be valid. As early as 2000 B.C.E., farmers observed that the mahogany species Indian lilac (neem, Azadirachta indica) has repellant and antifeedant effects on many insects. This led to a range of useful agricultural practices, such as planting Indian lilac trees close to crops in need of protection, and spraying crops with preparations made from the trees. (Snively and Corsiglia 2001) A couple of recent such discoveries have been recorded. Ramani Abe-bieli, a Ghanaian farmer, discovered that his cereals (millet and sorghum) were much less infested by striga (witchweed) after he had interrupted the repeated cultivation of cereals on a field by one year of onion. He hypothesized that a preparation from dried onion leaves could curb the growth of striga. Reportedly, the new method turned out to be effective, and it was soon adopted by other farmers in his community. (Tambo 2015, p. 120) In another case, farmers in Kenya, who had recently started to grow rice, cleared water fern (Azolla spp) from their rice fields. Having done so, they discovered that plants grew more vigorously than elsewhere around the heaps of weed that they had collected. This inspired them to make experiments, and as a result they started to use water fern to fertilize their kale and tomatoes as well as their rice fields. (Kamau and Almekinders 2009) This was a rediscovery; water fern, which has a high nitrogen fixation capacity, has been used in Chinese rice cultivation at least since the sixth century C.E. (Shi and Hall 1988).

These examples probably represent the way in which various methods for soil improvement have been discovered and rediscovered by observant and experimenting farmers around the world. Farmers have developed a keen understanding of soil quality problems and a wide range of efficient measures to deal with them. These methods are similar across the world: adding organic matter such as manure, weed, rotting stumps, or wood ash, swiddening, fallowing, crop rotation, intercropping including leguminous crops, terrace-building to reduce erosion on slopes, slowing down running waters to enhance sedimentation, etc. (Winklerprins 1999; Jodha 1980) Such improvements have been obtained with the help of generalizations based on careful empirical observations, such as “this crop grows better if we add wood ash to the soil”. A generalization like this can be classified as causal knowledge, albeit of a different nature than modern biochemical understanding of the same process, which refers to the role of potassium in plant metabolism. Empirically based causal generalizations can be efficiently action-guiding even if they do not refer to the objects invisible to the naked eye that are referred to in modern scientific accounts of the same phenomena.

However, there are also examples showing that lack of mechanistic knowledge can hamper the development of useful intervention hypotheses for experimental testing. Most obviously, this applies to lack of knowledge about pathogenic organisms that cannot be seen with the unaided eye. Without microscopes and other laboratory equipment it is difficult if not impossible to develop distinctions between different plant diseases. Whereas traditional taxonomies for plants and vertebrates are usually both detailed and much in line with modern scientific terminology, traditional vocabularies for plant pathology usually contain few distinctions. Plant diseases are typically not perceived as distinct entities, and the characteristic symptoms of specific diseases are not recognized. (Bentley 1989; Bentley 1994; Trutmann et al. 1996; Bentley and Thiele 1999; Stone 2016) This is no surprise; the scientific classification of plant diseases was highly uncertain until their connections with microorganisms were detected. (Stevens 1934).

Without knowledge about microorganisms that cause diseases, it is difficult to develop useful hypotheses about measures to cope with these diseases. For instance, many Nigerian farmers have discovered that a blight disease in cassava (Cassava Bacterial Blight) is particularly prevalent after heavy rain. This has led to the erroneous conclusion that the damage on the plants is caused by the rain, whereas it is in fact caused by a species of bacteria that are more easily spread in rainy conditions. (Richards 1979, p. 29) This mistaken belief hampers the conception of plausible hypotheses on how to prevent the disease. (It can be prevented with crop rotation followed by planting of new cassava from uninfected cuttings.) (Lozano 1986) Although farmers have in some cases discovered the usefulness of removing disease-affected plants, the idea of doing so is much less obvious if one does not know that the disease is caused by a microorganism spreading from plant to plant. Therefore, such (phytosanitary) measures are often not taken when they would have been efficient. (Trutmann 1996, p. 69) Similar problems apply to animal health. Without knowledge of the causal agents of animal disease, it is difficult for livestock owners to devise reasonable hypotheses on how to treat diseased animals. (McCorkle 1989).

Knowledge about insects is usually limited in traditional cultures.Footnote 8 Although most insects are visible to the naked eye, it is difficult to divide them into species. Traditional (“folk”) classifications tend to be vague and to operate with much fewer categories than there are biological species. (Bentley 1989) Furthermore, the reproduction and development of insects (sexual reproduction, usually followed by a series of three or four stages of metamorphosis with very different body structures), is typically not part of traditional knowledge. Farmers who have not received modern biological education are usually not aware that a caterpillar eating their crops is the same species as an adult insect, which they also observe on the farm. (Bentley 1989, pp. 27–28; Price and Gurung 2006) Farmers in third world countries often believe that pestilent insects are generated spontaneously from the plants they infest. (Bentley 1989, pp. 26–27; Bentley et al. 1994, p. 178) (In European science, the issue whether or not spontaneous generation can take place was considered open until the 1860s, when it was settled by specific experimental evidence and better understanding of biological processes.) (McMullin 1987) The common misconception that all insects are harmful has often led to indiscriminate and harmful use of insecticides. (Bentley 1989, pp. 28–29).

Not having access to microbiological and entomological knowledge obviously makes it more difficult to develop promising hypotheses for pest control experiments. Consequently, plant diseases have sometimes been attributed to completely unrelated factors, such as lightning, halos around the sun, planting in the wrong phase of the moon, or – worse – the “evil eye” of menstruating women. (Bentley and Thiele 1999, p. 78; Sillitoe 2000, p. 4) Such beliefs have also persisted in Europe, where witches and people with an “evil eye” were accused of causing crop failures. (Lykiardopoulos 1981; Behringer 1995) Magical ideas and practices were recorded in American agriculture in the 1940s, and they still have a central role in biodynamical agriculture, which is mostly practiced in Europe. (Passin and Bennett 1943; Smith and Barquín 2007; Chalker-Scott 2013).

However, there is considerable experience showing that traditional farmers tend to be interested in scientific information for instance about microbes and insect pests. They also use such knowledge efficiently to develop new methods of crop protection. (Sherwood 1997) For instance, knowing an insect’s metamorphosis makes it possible to control it in another stage than the one that attacks the plants. In one case, Bolivian farmers had problems with grubs attacking stored potatoes. After learning that these grubs are the offspring of weevils, they developed methods to control this pest. (Bentley 1994, p. 145) In many cases, when farmers have learned that insects such as wasps and fire ants, which they previously believed to be harmful, are in fact predators of pest insects, they have stopped killing them. (Bentley 2000, p. 284) Instead, they have often invented and experimented with ingenious methods to attract these insects to pest-infested fields. (Bentley et al. 1994; Bentley and Andres 1996; Sherwood 1997, p. 182).

Most of the action recipes that farmers try out in experiments are not innovations in the strict sense of new methods that have never been used before. Instead they often consist in trying something on one’s own farm that other farmers have already used elsewhere. In other words, the vast majority of experiments test adopted technologies rather than newly invented ones.Footnote 9 Therefore, learning from others is essential for the construction or choice of action recipes to be tried out in experiments. Farmers observe what others do, discuss with them, and exchange seeds. They learn from neighbours, but also from people they meet when travelling. (Boster 1986, p. 433; Sumberg and Okali 1997, p. 51; Maertens and Barrett 2013, p. 356) Obviously, this learning process works best if there is a free flow of information. Ideally, a village can function as a unit of collective learning through experimentation. However, farming communities differ much in the extent to which information is freely shared. In some communities, the information flow is rather severely restricted.Footnote 10 (Conley and Udry 2001; Maertens and Barrett 2013) In particular in hierarchical societies, there is a risk of prestige bias in social learning. For instance, some Indian farmers may be more inclined to learn from a high-caste than a low-caste farmer. (Stone et al. 2014).

4 The planning and conduct of experiments

The anthropological literature provides material for interesting methodological comparisons between the experiments performed by farmers and those performed in modern science. Two important caveats should be kept in mind when such comparisons are made. First, it would be grossly misleading to compare farmers’ practices with the ideals of modern science. It is much more interesting to compare farmers’ practices with the actual practices of modern scientists, which may not always coincide with the ideals promoted in the philosophical and methodological literature. Secondly, it should not be taken for granted that the same methodological requirements are relevant in the experiments performed by farmers as in those performed by scientists. The two experimental traditions differ in important respects, for instance in the extent to which experiments are replicated and in the intended application area for the experimental results (this farm vs. a large geographical region). In this section, six methodological precepts from modern science will be used as descriptive tools to characterize the methodological practices in farmers’ experiments, namely using controls, changing one variable at a time, following the original plan, randomizing, blinding, and replicating.

4.1 Controls

In the directly action-guiding experiments of modern science, such as clinical trials, it is considered obligatory to perform a control experiment (control arm of the experiment). In order to find out whether performing X will lead to Y, we do not only perform X to see whether (or to what extent) Y will then take place. We also investigate whether (or to what extent) Y takes place under similar circumstances but without X. This enables us to determine whether X really makes a difference. In some cases, we have background knowledge from previous experience, telling us that without X, Y will not happen (or only happen to some limited extent). However, such so-called historical controls are considered to be unreliable, since circumstances may have changed in ways we are not aware of. Therefore, it is strongly recommended to include a control arm in all experiments.

In agricultural experiments, it is usually easy to include a control arm, in the form of a control field. This method has decisive advantages over historical controls, since the two fields will be similar in terms of weather conditions, pest infestations, etc. In addition, the memory problems that can arise with historical controls will be avoided in this way. In fact, much of the methodology used in modern controlled experiments developed out of agricultural experiments that followed this pattern. (Conniffe 1990; Pretty 1991).

Judging by anthropological studies, most experiments performed by farmers involve some form of comparison. According to a large survey of farmers’ experiments in Africa, about 39% of the experiments involved “a direct, side-by-side comparison” between adjacent fields or two parts of the same field. (Sumberg and Okali 1997, p. 98) The prevalence of control fields has been observed by many researchers, also on other continents. (Johnson 1972, p. 154; Gupta 1989, p. 88; Richards 1989, p. 19; Bentley 1994, pp. 143–144; Richards 1994, p. 168; Lyon 1996, p. 43; Quiroz 1999, p. 120; Bentley et al. 2010, p. 131) However, it is an equally common observation that farmers frequently use historical controls. This can either take the form of comparisons with a particular plot on a particular year, or, more typically, reliance on “the accumulated understanding of past farming performances and major influencing factors such as rainfall”. (Kummer 2011, p. 68) According to agricultural researchers who cooperated with female Rwandan bean farmers, some of these farmers did not use control plots since they “‘carry the check in their heads’, that is, they know how the plot should yield under given conditions.” (Sperling et al. 1993, p. 512. Cf. Leitgeb et al. 2010, p. 745) The limitations of this method are not unknown to farmers. Sumberg and Okali quote one Zimbabwean farmer who used the historical control method. He argued that the years were similar enough, in particular concerning the amount and timing of rainfalls, for such comparisons to be reliable enough. However, he added that the peculiar pattern of rainfall in the year when the interview was made would make the comparison precarious for his latest experiments. (Sumberg and Okali 1997, p. 99).

Another common method is to compare one’s own harvests to those of neighbouring farmers growing the same type of crop. (Lyon 1996, p. 42; Leitgeb et al. 2014, p. 58) However, in spite of the visibility of other farmers’ crops and work methods, such comparisons tend to be less reliable due to lack of information on the neighbour’s farming practices.

4.2 Changing one variable at a time

The old dictum that one should change only one variable at a time seems still to have a considerable hold over scientists. Indeed, “testing of only one variable at the same time” has quite recently been described as one of the criteria that a scientific field trial has to satisfy. (Kummer 2011, p. 69. Cf. Röling and Brouwer 1999, p. 156) In projects involving cooperation between farmers and scientists, scientists have sometimes been “frustrated” with farmers whose experiments have not satisfied the one-variable requirement. (Ramisch 2014, p. 127) Reportedly, this is “one of the points that has [led] research station scientists to dismiss farmer innovation”. (Saad 2001, p. 14. Cf: Bentley 2006, pp. 455–456; Bentley et al. 2006, p. 100).

From a methodological point of view, these attitudes among scientists are somewhat surprising. It is known since long that the advice to change only one variable at a time is oversimplified and in many cases highly inadequate. Both in epistemic and directly action-guiding experiments, if the variables are not independent, then the one-variable precept is misguided. The American plant ecologist Frank Egler explained this as follows: “[I]n the physical sciences one can generally isolate one variable at a time for study, whereas in biology and even more in vegetation science, this is rarely possible. In the physical sciences differences among replicate measurements are generally attributable to deficiencies of technique; in biology differences are to a great extent due to fluctuations in variables assumed to be constant.” (Egler 1960, p. 235).

If the variables are independent, then the two types of experiments differ in terms of the one-variable precept. In an epistemic experiment, it will usually be an advantage to determine the effects of each individual variable separately.Footnote 11 This can provide us with fine-grained information about the causal mechanisms. However, this argument does not apply to a directly action-guiding experiment, since its goal is to obtain some desired practical outcome rather than to elucidate mechanisms. If we have a credible hypothesis according to which we can obtain the desired result by doing A + B + C, then we should test this combination, rather than each of A, B, and C separately. This is also how directly action-guiding experiments, such as clinical trials, are performed in practice. For instance, suppose that there are mechanistic reasons to believe that a combination of three drugs, which is already used with success against AIDS, might also be efficient against some other autoimmune disease. Then it is the combination of the three drugs that should be tested in clinical trials, rather than each of the three component drugs by itself. Similarly, suppose that a farmer has obtained a few seeds from a variant of beans that grows very well in another region, which has larger rainfalls in the growing season. (S)he may then form the hypothesis that an improved harvest can be obtained by sowing this variety and also increasing the amount of watering. There are good reasons to test this action recipe, which involves a change in two variables (variety and water), rather than just testing the new seeds without increased watering. In other words, there may be good reasons for experimenting farmers not to follow the dictum “change only one variable at a time”.

4.3 Following the original plan

As was noted by Arthur Stolzenbach, farmers performing experiments differ from agricultural scientists in that they “do not pin down the design and execution of their experiments but adjust them during the run of the cropping-period.” (Stolzenbach 1999, p. 171) Whereas such changes would be considered faults in a typical (epistemic) experiment in science, from the farmer’s viewpoint they may be part of what makes the experiment a success. Basically, the farmer’s purpose with the experiment is to find a way to obtain a certain desirable result. If observations that (s)he makes during the experiment indicate that an additional intervention is needed, then making that intervention can serve the purpose of the experiment. For instance, if it becomes obvious during an experiment with a new crop variety that it needs more water than the standard variety, then it is sensible to water it, in order to see whether the revised intervention “new variety + more water” yields the desired effect. If it does, then the experiment can be a success. Refraining from extra watering in deference to the original plan (just “new variety”) would be tantamount to a guaranteed failure to obtain directly useful information from the experiment.

Again, it is important to compare farmers’ experiments with directly action-guiding experiments performed by scientists, rather than with their epistemic experiments. Clinical trials are commonly required to have a mechanism for early termination if it becomes clear before the end of the trial that patients gain from being included in one of the arms of the trial rather than the other. (O'Brien and Fleming 1979) It is highly plausible that similar practices are prevalent also in other areas where scientists perform directly action-guiding experiments. For instance, suppose that a group of technological scientists are testing a new mechanical device. Halfway through the experiment they discover that the device will break down unless it is lubricated in a way not foreseen in their original plans. We would expect them to lubricate it, rather than sticking to the original protocol. In doing so they would not be irrational or unscientific, and neither are the farmers who adjust their original action recipes during an experiment in order to increase the chances of a useful outcome.

4.4 Randomization

Randomization is one of the cornerstones of modern experimental methodology. It is an efficient countermeasure, not only against known confounding factors, but also against as yet unidentified confounders. However, its introduction in modern science was surprisingly late. In the nineteenth century, there was sporadic use of some precursors of the modern method of randomization, such as alternate allocation, in which the first, third, fifth etc. patients in a clinical trial are assigned to one group and the second, fourth, sixth etc. to the other. (Hansson 2015, pp. 106–107) The modern method, in which each participant is randomized to one of the groups, was developed by Ronald Fisher (1890–1962) in the 1920s when assigning cultivars randomly to fields in agricultural trials. (Conniffe 1990) The first medical study employing the method was published in 1948. (Doll 1998; Marshall et al. 1948) It is now, after some initial resistance, accepted as part of the “gold standard” for a directly action-guiding experiment in medicine: the randomized controlled trial (RCT). (Hansson 2014).

Judging by the published literature, randomization does not take place within the traditional practices of agricultural experimentation. To the contrary, in co-operations between agricultural scientists and farmers, the former have often had to give up their plans for randomization in order to achieve practical goals such as “keeping all plots of a given legume together in one set or placing all the unfertilized treatments on one side of a footpath.” (Ramisch 2014, p. 128) This should be no big surprise, given how late and reluctant the introduction of randomization has been among professional scientists.

4.5 Blinding

Blinding is a highly useful countermeasure against the effects that the experimenter’s beliefs and expectations (and those of the human subjects, if there are any) can have on the outcome of an experiment. The first scientific study with blinded evaluators seems to have been the investigation of Franz Mesmer’s alleged animal magnetism that was conducted in 1784 by a commission of the French Academy of Sciences. (Sutton 1981; Lopez 1993) In the nineteenth century, blinding was employed by critics of deviant belief systems, such as dowsing and homeopathy, as means to disclose scams and self-deception. Towards the end of that century, some researchers began to use it as a means to improve the accuracy of observations in experiments where “real” effects were expected. After World War II, blinding became more common due to increased awareness of the effects of bias in research. It has been part of the standard requirements on clinical trials since the 1940s. (Kaptchuk 1998; Hansson 2015, pp. 102–104).

Extensive experience, in particular from modern medical and psychological research, confirms that blinding is an efficient tool to prevent misinterpretation of experiments due to bias and other confounding factors. But nevertheless, the introduction of blinding has been remarkably slow in many areas of science. Still today, many experiments that could easily have been improved with blinding are performed unblinded. One example is the evaluation of histopathological material from toxicity studies, which many pathologists prefer to conduct without blinding. (Holland 2001; Holland and Holland 2011) Another example is field trials in agricultural science. A trial involving two varieties of seed that are indistinguishable to the naked eye can easily be organized so that neither those who sow and grow the seeds nor those who evaluate the harvests know which seed is which. However, in practice this is not how agricultural scientists normally perform field trials. A recent study indicates that blinding may have a large impact on the outcomes of agricultural field trials. (Bulte et al. 2014).

There does not seem to be any trace in the published literature of farmers using blinding in their experiments. This is much less surprising than that agricultural scientists, with their access to methodological literature on blinding and its advantages, have not either adopted it.

4.6 Replication

As mentioned in Section 1, experiments have to be repeatable (replicable) in order to fill their intended function, which in the case of directly action-guiding experiments is to discover and confirm useful action recipes. Strictly speaking, an agricultural experiment cannot be exactly replicated, since the conditions for farming change from growing season to growing season. However, with a reasonably wide understanding of the notion of a replication, an agricultural experiment is replicated every time someone performs a new experiment with the same seed variety or the same work methods under similar conditions. In that perspective, farmers’ experiments are replicated to a massive extent that is unknown in academic science. This is because, as already mentioned, farmers around the world tend to try new methods experimentally on their own farms, even if others have tested them elsewhere under similar conditions.

Agricultural researchers have often noted, not without surprise, that farmers treat new seeds or methods obtained from researchers in the same way as new seeds or methods obtained from a neighbour or an acquaintance. Before adopting the new practice on a full scale, farmers try it out on small experimental plots. In this way “every change of the system introduced by an outside organization will be first screened and tested by the farmer”. (Dries 1991, pp. 227–228) This is a remarkably ubiquitous pattern that has been observed all around the world. (Ryan and Gross 1943, p. 18; Brookfield and Padoch 1994, p. 41; Lyon 1996; Katanga et al. 2007, p. 120; Waters-Bayer et al. 2009, p. 240) This massive experimentation might seem superfluous, under the assumption that the field trials and other investigations performed by agricultural scientists provide all the information that farmers need. However, this assumption may not hold. The agricultural methods proposed by scientists may have to be adjusted to the specific conditions on the individual farm. As noted by the Scottish agricultural writer James Caird already in 1852, “the detail is everywhere varied by the judicious agriculturalist to suit the necessities and advantages of the particular locality”. (Caird 1852, p. 502).

The ubiquity of such local adaptations can be seen from the introduction of a “new” method for storing seed potatoes, namely to keep them in natural diffused light rather than in complete darkness. This form of storage has been used traditionally by farmers in Peru. In the mid 1970s, researchers confirmed that it improved quality, after which it was introduced in some 25 countries. An extensive study showed that the vast majority of the farmers who adopted this technology modified it in various ways to suit their own facilities and budgets. (Rhoades and Booth 1982).

5 Evaluation of the methodologies

Both farmers’ experiments and the field trials performed by agricultural scientists are directly action-guiding experiments. They also both have basically the same goals, namely to improve agricultural practices in respects that are important to farmers, such as yield (per area and per work hour), resistance to pests and climatic stresses, product quality, and sustainability. Therefore, comparisons between these two types of experiments are of particular interest.

One important factor that needs to be taken into account in such comparisons is the immense scale of farmers’ experiments. Well over 800 million people are working on farms.Footnote 12 Probably, most of them perform experiments. Most of these experiments are not particularly innovative. They may for instance be tests of new seeds obtained on the market or as gifts, small changes in established work methods, or minor adjustments of crop rotation schemes. As noted by Jeffery Bentley, “these experiments are essential for adapting techniques in constantly evolving farming systems. Yet each individual experiment is not very novel or useful. It is their aggregate effect over the long run that gives them value.” (Bentley 2006, pp. 451–452).

The massive extent of farmers’ experiments impart to them two considerable advantages over the field trials conducted by agricultural scientists. First, since farmers perform their experiments on their own farms, they obtain information that is fully adjusted to the local conditions. Secondly, farmers’ experiments satisfy the demand for replication to a much higher degree than any type of experiments performed by professional scientists. But on the other hand, farmers’ experiments also have features that put them at disadvantage compared to the field trials of agricultural scientists. Scientists use control fields more systematically, and only seldom refer to historical controls. They also apply modern statistical and observational methods that reduce the risk of incorrect conclusions for instance due to the influence of confounding factors.

Can the advantageous features of farmers’ experiments compensate for their less stringent methods for evaluation and for their less developed control of confounding factors? And conversely, can the advantages of scientific field trials compensate for their lack of local specificity and their much lower rates of replication? These are empirical issues that cannot be solved with armchair methodological considerations. However, something can be said about the character of the empirical issues that are involved in this comparison.

Agricultural scientists aim for fairly general action recipes, for instance new crop varieties or work methods that perform well at least in a whole region. The extent to which this succeeds will depend, among other factors, on how strong the correlation is between what works under these different local conditions. A strong correlation will put agricultural science at advantage, whereas a weak correlation will put it at disadvantage. Obviously, the strength of this correlation may differ significantly between different crops or types of agricultural practices. It will also depend on the natural and agricultural variabilities within the region that the recommendations of agricultural scientists aim at serving.

6 Connections with the philosophy of science

It should be clear from the previous sections that farmers’ experiments give rise to interesting epistemological and other philosophical issues. As I will now proceed to show with three examples, there are also strong connections with several prominent topics in the philosophy of science. This is, of course, an argument to pursue the philosophy of experimentation in a unified manner, rather than separately for the different experimental traditions.

Since Hanson’s (1958) seminal remarks on the inherent presuppositions of experiments, the theory ladenness of experiments has been a contested issue in the philosophy of science. (Karaca 2013; Franklin 2015) Heidelberger (2003) argued that exploratory experiments can be theory-free, but this was contested by Radder (2003). Several authors, including Franklin (2005a, b) and Karaca (2013), have proposed to resolve this disagreement by distinguishing between different senses or degrees of theory ladenness. This debate has been focused on epistemic experiments, but chances of finding examples of experiments that are theory-free, or as close to that as possible, would seem to be greater if the search is extended to directly action-guiding experiments. (Hansson 2015, pp. 91–93) In particular, this applies to directly action-guiding experiments performed outside of scientific traditions, such as farmers’ experiments.

As mentioned above, much of the attention that philosophers of science have paid to experiments has concerned their role in the testing of hypotheses. The question how the outcomes of experiments can rationally change the epistemic status of hypotheses (verify, falsify, corroborate etc.) is still a standard topic in undergraduate courses in the philosophy and methodology of science. Contributions by the New Experimentalists have made it clear that experiments can have other roles than determining the rational epistemic status of hypotheses, but no one seems to have denied that experiments can legitimately be used for that purpose, and that this is in practice one of the major uses of experiments.Footnote 13 A large category of hypothesis-testing experiments have been left out of these discussions, namely directly action-guiding experiments. A directly action-guiding experiment, such as a farmer’s experiment, always has a (practical) hypothesis in the form of an intervention, such as a new seed or a method to ward of harmful insects, which is put to test in the experiment. In studies of the logic of theory testing, it would be of considerable interest to compare the testing of traditional, theoretical, hypotheses in science to that of practical hypotheses such as “this new seed yields a larger harvest than the old one”.

Communication and decision-making at the science-policy interface give rise to an abundance of complex epistemic and value-related issues. One of these concerns the evaluation of experimental evidence for practical purposes, i.e. the use of experiments to determine whether a proposed action-type, such as a medical treatment or a method in social work, produces the desired results. Here it is crucial to observe that the information needed for practical decision making is what effects the measure has under what circumstances. Information about the mechanism underlying a putative effect (“how-information”) is useful for this purpose only to the extent that it contributes to answering the question whether this effect is actually produced by the intervention. (Hansson 2014, p. 46) Unfortunately, this has often been misunderstood, in particular concerning clinical trials, which are typically performed by medical scientists who are also deeply involved in mechanistic studies and discussions. Prominent philosophical disparagement of clinical trials and associated evidence-based methodologies is based on the misconception that practical decision-making primarily requires knowledge about causal laws connecting the proposed measure with the desired effect. (Cartwright 2012, p. 315; Cartwright and Stegenga 2011, pp. 295–296 and 313) If this approach were applied to farmers’ experiments, it would be incomprehensible how successful the action guidance obtained from these experiments can be. The discussion on clinical trials and evidence-based policies would gain much from a clear distinction between epistemic and directly action-guiding experiments and from the inclusion of farmers’ experiments as examples of how the latter type of experiments can serve their purpose without yielding information about mechanisms or laws of nature.

7 Conclusion

We have seen that farmers in all parts of the world have traditionally performed experiments to a massive extent, and they continue to do so. Anthropological studies show that many of these experimental traditions are fairly sophisticated; for instance they involve control fields and long-term trials. These experimental practices give rise to important philosophical issues, some of which have been discussed above. We should see the philosophy of experimentation as a philosophical discipline of its own. Since some but not all of the practices it studies are parts of science (in the common sense of that word), the philosophy of experimentation overlaps with the philosophy of science but is not one of its subdisciplines.

Farmers’ experiments are directly action-guiding, i.e. they aim at finding work methods (action recipes) that have desirable practical effects. They should therefore be compared to directly action-guiding experiments performed by scientists (clinical trials, agricultural field trials, etc.), rather than to scientific experiments aimed at finding out the causes and mechanisms of natural phenomena. In comparison to the field trials performed by agricultural scientists, farmers’ experiments have the advantages that many more replications are performed and that results pertaining to many more specific local conditions are obtained. On the other hand, farmers’ experiments often have less stringent controls and less precise evaluation methods. In some cases, the formation of plausible hypotheses for experimentation is hampered by lack of knowledge for instance about the causes of plant diseases (microorganisms, life cycles of insects, etc.). Thus, each of the two experimental traditions has advantages that the other lacks. Neither of them can replace the other, but there is a potential for mutual and respectful learning.

The anthropological literature on farmers’ experiments is extensive and rich in empirical detail. As this article has hopefully shown, it provides useful material for investigations in the philosophy of experimentation. However, there is also a need for empirical studies of farmers’ experiments that put focus on methodological issues such as the choice of topics and hypotheses for testing, the use control fields, how confounding factors are conceived and dealt with, and the process of evaluating and drawing conclusions from experiments.